Associate Lead - Testing (QA + MLOps)

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! Role: Lead/Associate Lead – QA + MLOps & Generative AI Experience: 10+ years Location: Mumbai/Bangalore (Hybrid) Key Responsibilities AI/ML & GenAI Testing Strategy (AWS Ecosystem) Define testing approaches for AI systems built on AWS services such as: Amazon SageMaker Amazon Bedrock AWS Lambda Amazon API Gateway Amazon Kinesis AWS Glue Amazon S3 Amazon CloudWatch Design Validation Frameworks Covering Model accuracy & performance validation Data drift & concept drift detection Hallucination detection for LLMs Prompt robustness testing RAG validation (retrieval accuracy + grounding) Bias & fairness validation Safety & toxicity testing MLOps Quality Engineering (AWS-Centric) Validate The End-to-end ML Lifecycle Including Data ingestion & feature pipelines Model training & hyperparameter tuning Model versioning & registry Deployment validation Canary & blue/green release validation Work With AWS-native Services Such As SageMaker Pipelines SageMaker Model Monitor SageMaker Feature Store Bedrock model evaluation workflows CloudWatch-based observability Implement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools. GenAI & Agentic AI Testing Define Quality Engineering Approaches For LLM-based applications using Amazon Bedrock Prompt engineering validation Multi-agent orchestration testing Chatbot & Voice bot conversational testing Intent classification validation Conversation drift & fallback validation API contract validation for LLM integrations Build Reusable Evaluation Harnesses For BLEU / ROUGE scoring Embedding similarity scoring Response consistency Safety scoring frameworks Framework & Capability Development Design reusable AI testing accelerators Create AWS-aligned AI test automation frameworks (Python-first) Develop synthetic data generation strategies Establish AI quality scorecards Build an internal AI QA Center of Excellence Client Engagement & Leadership Lead AI/ML quality strategy workshops Perform AI risk & readiness assessments Present quality architecture to CXOs Drive QA transformation programs Mentor QA teams on AWS-based AI testing Own delivery for AI testing engagements end-to-end Must Have Skills Testing Expertise 8–12+ years in Quality Engineering Strong test strategy, automation & governance experience Experience leading QA transformation initiatives Experience building frameworks from scratch AI/ML & GenAI Expertise Deep understanding of ML lifecycle Experience testing ML models (NLP preferred) Hands-on experience validating LLM applications Strong understanding of: Prompt engineering RAG architecture Embeddings Bias & explainability AWS AI/ML Expertise Hands-on experience with: Amazon SageMaker (training, deployment, monitoring) Amazon Bedrock (LLM integration & evaluation) S3-based data pipelines AWS IAM (security validation) CloudWatch monitoring Lambda & API Gateway integrations AWS CI/CD (CodePipeline / CodeBuild preferred) Understanding Of Infrastructure as Code (Terraform / CloudFormation) Observability in AI systems Cost monitoring for ML workloads Technical Skills Python (mandatory) Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch) Experience with LLM frameworks (LangChain, etc.) API & automation testing frameworks Git-based workflows Leadership & Communication Strong client-facing communication Experience leading QA teams Ability to create strategy decks & solution proposals Strong stakeholder management If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us !