Source description
About the role
About CredServ CredServ is building the AI Operating Layer for Embedded Receivable Workflows . We are transforming B2B trade by deploying specialized Agentic AI to automate the entire receivables lifecycle—from Partner Onboarding and Credit Underwriting to Voice Collections and Digital Legal recovery. The Role: Lead AI Quality Engineer You are not just testing AI; you are using AI to redefine how testing is done. We are looking for a pioneering Lead QA Engineer who bridges the gap between deterministic software and non-deterministic AI Agents. You will use AI copilots to hyper-accelerate automation, migrate legacy suites to self-healing frameworks, and build the infrastructure that ensures our core services and autonomous agents operate flawlessly, safely, and within strict financial regulations. What You Will Do (Core Responsibilities) Next-Gen Automation & Core QA AI-Assisted Scripting & Regression: Utilize AI copilot tools (e.g., Cursor, GitHub Copilot) to rapidly generate, maintain, and scale automated regression, integration, and functional test suites. Modern Frameworks: Architect and scale automation using Playwright or Cypress, transitioning away from and migrating legacy Selenium suites to AI-native, self-healing frameworks. Visual & API Testing: Implement Visual AI testing (e.g., Applitools) to catch UI anomalies across multiple devices. Own the end-to-end API testing strategy for our deterministic rule-based logic and core banking integrations. Root Cause Analysis: Perform deep triage and root cause analysis of pipeline failures and flaky tests using AI-powered log analysis and observability tools. Agentic AI Testing Strategy Onboarding Agent: Test conversational KYC flows, document extraction (OCR/NLP) accuracy, and self-serve onboarding logic. Underwriting Assistant: Validate the AI's ability to draft accurate credit summaries and reason codes for Human-in-the-Loop (HITL) limit reviews. Voice Collection Bot: Measure voice latency (Time-to-First-Token), intent recognition, and compliance-grade empathy for recovery calls. Digital Legal Module: Verify the automated triggering and tracking of real-time legal notices and case updates. Non-Deterministic QA & Guardrails LLM-as-a-Judge: Implement hallucination metrics, context precision (RAG testing), and semantic similarity checks to ensure AI outputs never breach regulatory guardrails. Red Teaming & Security: Write adversarial test cases to evaluate prompt injection vulnerabilities and data leakage prevention. Who You Are (Requirements) Experience: 5+ years of proven experience as an SDET or QA Automation Engineer, with at least 1+ years specifically testing LLMs, RAG systems, or deploying AI-assisted QA workflows. Automation Mastery: Deep expertise in Playwright or Cypress. Strong understanding of core QA principles, including the Page Object Model (POM), exhaustive regression testing, and robust CI/CD integration (GitHub Actions/GitLab CI). Coding Proficiency: Strong programming skills in Python (for the AI evaluation stack) and/or Node.js/TypeScript (for UI/API automation). AI QA Tooling: Hands-on experience with AI coding assistants (Cursor), Visual AI (Applitools), and LLM evaluation frameworks (e.g., LangSmith, DeepEval or Ragas). Financial Precision: Deep understanding of testing Human-in-the-Loop workflows, complex reconciliation logic, and exception-clearing AI where accuracy is non-negotiable. Why Join CredServ You will be defining how global supply chains and enterprises evaluate and trust Artificial Intelligence. If you want to work at the bleeding edge of Next-Gen Automation, Voice AI, and B2B trade, this is your place. Competitive Salary & Equity Flexible Work Options Comprehensive Health & Wellness Benefits