Job Title - Automation Engineer
Location - Washington DC
Job Description:
• Design and build end-to-end AI testing frameworks covering unit, functional, integration, and regression testing including automated test case generation and model output validation against defined quality benchmarks.
• Develop load and performance testing simulating high-concurrency scenarios to validate scalability, latency SLAs, and platform stability under peak load.
• Build LLM evaluation pipelines covering relevance, groundedness, and hallucination metrics integrated into CI/CD workflows to enforce automated quality gates before every model or agent deployment.
• Own regression testing strategy across the AI platform maintaining automated suites that validate agent behavior, RAG pipeline accuracy, and API contracts after every release to prevent quality degradation.