BioTech Data Engineer

Job Title: Biotech Data Engineer

Job Type: FTE

Work Mode: Remote

The Biotech Data Engineer focuses on designing, building, and maintaining scalable Azure Databricks-based data pipelines and architectures that enable analytics, AI/ML, and reporting across commercial functions. It involves leading data engineering initiatives, ensuring governance and compliance, optimizing performance and costs, and collaborating across teams to advance the company’s data and AI strategy. The role reports to the Director of Business Intelligence.

Roles & Responsibilities

Design, build, and maintain scalable, reliable, and cost-efficient data pipelines using Azure Databricks in support of analytics, machine learning, data science, and operational use cases

Lead data engineering initiatives that enable AI/ML model development, LLM integrations, and AI-driven applications while ensuring scalability and alignment with enterprise and business priorities

Architect and implement data integration frameworks across diverse Adtech and Martech ecosystems, incorporating Google Analytics, media campaign data, and third-party marketing APIs like Salesforce

Manage and optimize data ingestion, transformation, and storage processes using SQL, Python, and PySpark to integrate structured and unstructured data sources

Design and maintain API integrations with internal and external systems, including Python- and PySpark-based services and AI/LLM-powered APIs for advanced analytics and automation

Administer and maintain Azure-based data tools and platforms (e.g., Databricks, ADF) to ensure operational excellence, reliability, and security

Collaborate with internal stakeholders and external partners to evolve data platform design and architecture that supports advanced analytics, personalization, marketing intelligence, and marketing automation

Execute against the company’s data and AI strategy by translating strategic goals into technical architecture, design and requirement documents, and implementation roadmaps

Ensure data quality, integrity, and consistency through robust validation, monitoring, and alerting mechanisms within Azure Databricks

Implement and enforce data governance, security, and compliance standards in collaboration with IT, InfoSec, and data governance teams

Partner with analytics, marketing, commercial operations, and core technology/cybersecurity teams to deliver fit-for-purpose commercial data products

Monitor and optimize data infrastructure costs, performance, and scalability across Azure cloud environments

Develop and maintain architecture documentation, pipeline specifications, and design diagrams for transparency and knowledge sharing with technical and business stakeholders

Participate in architecture discussions, design reviews, and CI/CD workflows to ensure high-quality engineering and deployment practices as part of an Agile engineering team

Continuously evaluate and recommend new Azure services, designs, improvements, frameworks, and AI integration tools to enhance our data platform

Drive automation, observability, and standardization across data workflows to improve efficiency and reduce manual intervention

Complete all job duties in compliance with company policy, SOPs, safety rules, and applicable federal, state, and local regulations

Education & Licenses And Experience

Bachelor of science degree required. A minimum of 5 years transferable working experience in the area of operational support of a Data Engineering, Data Architecture, or Cloud Platforms function preferred. The ideal candidate will have recent and relevant experience in the pharma or biotech industry.

Required Competencies & Skills

Experience with Azure Cloud (Databricks, DevOps, DataFactory)

Pharma / biotech domain experience, specifically within the commercial data space (sales, market access / payer, marketing)

Strong hands-on python, pyspark, and SQL skills

Direct experience with building and leveraging API integrations in ETL pipeline development

Experience integrating with current best-in-class AI models and APIs like OpenAI API, Databricks AI models, etc.

Self-driven with ability to independently design end-to-end data pipelines while ensuring architectural best practices

Ability to collaborate with a broad set of stakeholders to evaluate the business need and construct a technical design that can enable stakeholder priorities

Travel up to 20%

Preferred Competencies & Skills

Strong knowledge of and experience with maximizing business value using Databricks Unity Catalog or Databricks One capabilities

Familiarity with tools and practices in the Martech and Adtech landscape

Knowledge of Consumer, Patient, or HCP data ecosystems

Experience with identity providers like LiveRamp, Acxiom, Experian, etc.

Experience with custom-built or third-party Customer Data Platforms (CDP)

Experience with marketing automation tools