Sr. AI Data Analyst-Agentic Systems & GenAI

What makes You an ideal candidate? Validate readiness of Agentic AI use cases for production deployment. Track deployment success metrics and post-production performance. Identify gaps between expected vs. actual outcomes. Define metrics for: Accuracy and response quality, Task completion success rates, Hallucination and failure cases, Latency and throughput. Build evaluation datasets and validation pipelines. Analyze: Agent workflows and decisions, Prompt-response chains, Tool usage and orchestration behavior. Develop observability dashboards using telemetry and logs. Detect and escalate production issues and anomalies. Data Analysis & Reporting. Perform root cause analysis on failures and performance issues. Deliver executive-level reporting on AI system effectiveness. Provide actionable insights to improve system design and outcomes. Work closely with: Lead Architects for feasibility alignment AI/ML engineers for model/system improvements Product teams for use case refinement. Translate technical findings into clear business insights. Advanced SQL, Python (Pandas, NumPy), or similar tools. Data visualization platforms (Power BI, Tableau). Strong experience in data validation, anomaly detection, and statistical analysis. Familiarity with: LLM workflows and prompt engineering, RAG pipelines and evaluation strategies, Agent orchestration and tool integration. Understanding of AI failure modes (hallucinations, drift, inconsistency). Experience with: Logging, tracing, and telemetry systems AI evaluation tools and frameworks Monitoring production systems (Azure Monitor, Application Insights). Strong working knowledge of Azure ecosystem, including: Azure OpenAI / AI services Azure Databricks Data platforms (Azure SQL, Cosmos DB) Monitoring tools (Log Analytics, App Insights). Strong analytical and problem-solving skills in complex AI-driven systems. Ability to connect system behavior with business outcomes. Expertise in translating data into actionable insights. High attention to detail in validation, quality, and accuracy. Strong communication skills across technical and non-technical stakeholders. Ability to thrive in fast-evolving AI environments. Ability to wrangle large datasets, structured and non-structured data, including data mining and manipulation.

Work Experience & Education 6-8 years experience in data analytics, data science, or AI Systems analysis or similar role required. Experience supporting AI/ ML or GenAI systems in production environments preferred. Auto finance experience preferred, cross functional Agile team experience preferred. Bachelor’s Degree in Data Science, Computer Science, Engineering or related quantitative field preferred. Master’s Degree in related quantitative field preferred.

What We Offer : Generous benefits package available on day one to include: 401K matching, bonding leave for new parents (12 weeks, 100% paid), tuition assistance, training, GM employee auto discount, community service pay and nine company holidays. Our Culture: Our team members define and shape our culture — an environment that welcomes innovative ideas, fosters integrity, and creates a sense of community and belonging. Here we do more than work — we thrive. Compensation: Competitive pay and bonus eligibility. Work Life Balance: Hybrid work environment, 2-days a week in office. The office locations for this role can be Irving, TX or Ft. Worth, TX NOTE: We are unable to consider candidates who require visa sponsorship for this position This position is not open to agency submissions #LI-hybrid #LI-MH1 #GMFJobs