Senior QA Engineer Conversational AI (Delhi)

Senior QA Engineer Conversational AI & Agentic Systems Location: Gurgaon (Hybrid / In-office) Experience: 69 Years Employment Type: Full time Job Overview We are looking for a Senior Testing Engineer with 6 to 9 years of experience specializing in Conversational AI. You will build the testing framework for advanced AI chatbots and multi-agent systems, ensuring accuracy, safety, and seamless user experiences. Key Responsibilities - Chatbot Evaluation: Test intent recognition, entity extraction, fallback handling, and contextual memory. - Agentic Testing: Validate multi-agent workflows, tool execution accuracy, and guardrail compliance. - LLM Evaluation: Implement RAG (Retrieval-Augmented Generation) testing metrics like faithfulness and relevance. - Automation: Build automated test suites using Python, Pytest, or specialized Conversational AI testing tools. Required Skills & Qualifications - 6 to 9 years in software testing, with 2+ years dedicated to LLMs, chatbots, or voicebots. - Familiarity with prompt injection vulnerability testing and semantic similarity evaluation. - Python, CI/CD pipelines, API testing (Postman/RestAssured), and performance testing tools. Preferred Skills - Experience with RAG evaluation frameworks (Ragas, TruLens, DeepEval). - Knowledge of AI safety frameworks (NeMo Guardrails, Llama Guard). - Experience testing multi-agent/agentic systems with tool-calling validation. - Hands-on experience with semantic similarity scoring and cosine similarity libraries. Ideal Candidate Profile A strong candidate should be able to: - Build automation frameworks for non-deterministic AI outputs. - Design adversarial test suites for prompt injection and jailbreak detection. - Implement programmatic RAG evaluation metrics beyond traditional pass/fail assertions. - Validate multi-agent tool-calling workflows and prevent infinite agent loops. Work Location: Gurgaon Work Mode: Hybrid / In-office .