Senior Machine Learning Engineer AI Assisted Data Annotation

About the Role We are seeking a Senior Machine Learning Engineer AI-Assisted Data Annotation to own the automated annotation track within ABBYYs Document AI Data team. This role sits at the intersection of large model capabilities and production data engineering, leveraging LLMs and vision-language models to generate high-quality training data at scale. You will design and build AI-assisted annotation pipelines, ensuring outputs are accurate, measurable, and reliable for downstream model training. This is an ideal role for engineers who combine deep model expertise with strong system-building instincts and thrive in fast-moving, experimental environments. Key Responsibilities Technical Development & Innovation Design and implement AI-powered annotation pipelines using large models to generate ground truth labels at scaleDevelop and refine prompting strategies, few-shot examples, and fine-tuning approaches to improve accuracy and consistencyBuild systems for label verification, confidence scoring, and quality validationEvaluate which tasks are suitable for automated annotation vs. human review, and define decision criteriaCreate evaluation frameworks to benchmark automated annotations against human-labeled dataContinuously improve annotation quality using feedback from human review workflowsProject Ownership & Leadership Own the automated annotation track end-to-end, from architecture through production monitoringDrive technical decisions across model selection, pipeline design, and validation strategiesDefine integration points with platform infrastructure and model serving systemsCollaborate with Data Operations to design human-in-the-loop workflows for efficient reviewContribute to roadmap planning with Principal-level technical leadershipInfrastructure & Scale Build and optimize large-scale inference pipelines for processing millions of documentsImplement monitoring and alerting for quality degradation and system failuresDesign batching, caching, and fallback mechanisms to balance cost, throughput, and accuracyCollaborate with Platform teams on model serving, APIs, and infrastructure scalingMaintain clear documentation of annotation strategies, metrics, and known limitationsQualifications Education & Experience MS or PhD in Computer Science, Engineering, Mathematics, or related field5+ years of experience in Machine Learning / AI, with focus on:Large Language Models (LLMs)Vision-Language Models (VLMs)Data annotation or labeling systemsDemonstrated success using large AI models to automate annotation at production scaleStrong background in evaluation design and quality measurementTechnical Expertise Deep expertise in LLMs and VLMs, including prompting, instruction tuning, and output evaluationStrong understanding of document understanding tasks (classification, extraction, layout analysis, semantic parsing)Experience designing label quality metrics, confidence scoring, and agreement analysisStrong programming skills in Python and proficiency with PyTorch or similar frameworksExperience with large-scale inference pipelines and model serving systemsFamiliarity with human-in-the-loop annotation systems and automation trade-offs About the Role We are seeking a Senior Machine Learning Engineer AI-Assisted Data Annotation to own the automated annotation track within ABBYYs Document AI Data team. This role sits at the intersection of large model capabilities and production data engineering, leveraging LLMs and vision-language models to generate high-quality training data at scale. You will design and build AI-assisted annotation pipelines, ensuring outputs are accurate, measurable, and reliable for downstream model training. This is an ideal role for engineers who combine deep model expertise with strong system-building instincts and thrive in fast-moving, experimental environments. Key Responsibilities Technical Development & Innovation Design and implement AI-powered annotation pipelines using large models to generate ground truth labels at scaleDevelop and refine prompting strategies, few-shot examples, and fine-tuning approaches to improve accuracy and consistencyBuild systems for label verification, confidence scoring, and quality validationEvaluate which tasks are suitable for automated annotation vs. human review, and define decision criteriaCreate evaluation frameworks to benchmark automated annotations against human-labeled dataContinuously improve annotation quality using feedback from human review workflowsProject Ownership & Leadership Own the automated annotation track end-to-end, from architecture through production monitoringDrive technical decisions across model selection, pipeline design, and validation strategiesDefine integration points with platform infrastructure and model serving systemsCollaborate with Data Operations to design human-in-the-loop workflows for efficient reviewContribute to roadmap planning with Principal-level technical leadershipInfrastructure & Sca