Source description
About the role
Responsibilities Build and own eval infrastructure for production AI systems, golden datasets, regression suites, and LLM-as-judge harnesses. Own API and backend test automation across microservices and async pipelines, not the UI layer. Write, co-write, and review test design documentation; drive code review for automation frameworks. Drive the design/code review process for test automation, seeking and providing constructive criticism. Lead observability design so anyone can answer did the AI get worse this week with a chart, not a gut feel. Own quality communication across sprint and release cycles. Requirements 2-5 years QA/SDET with a minimum of 1.5-2 years of backend/API testing. Strong coding in Java, Python, or TypeScript. Hands-on with REST Assured, Pytest, or equivalent coded API frameworks. Understands microservices, async systems, and event-driven architecture. Builds automation frameworks from scratch, not just uses tools. CI/CD integration experience. Systems thinker - reasons about failure modes, retries, latency, contracts. AI/LLM testing is a strong plus, RAG, hallucination detection, and eval frameworks. Experience working with Web and API Testing - both manual and automation. Performance testing exposure preferred (JMeter, k6) and Agile experience. This job was posted by S M Nandakishore from CAW Studios.
More at Caw Networks