Data Engineer - SQL/Python

We are looking for a skilled Data Engineer, who can design and maintain robust data pipelines, work with large-scale structured and semi-structured pharma datasets, and collaborate closely with Data Scientists and Python Developers to deliver clean, validated, production-grade data infrastructure. Key Responsibilities : Pipeline Design & Development : - Design and build scalable ETL/ELT pipelines for ingesting, transforming, and loading pharmaceutical data from multiple sources. - Process indication-level market forecast data from Excel workbooks into structured JSON and DB-ready formats for frontend visualisation. - Build and maintain data extraction pipelines from clinical trial registries, conference abstracts, and drug databases. - Implement multi-step validation workflows to ensure pipeline output accuracy and completeness before downstream handoff. Data Modelling & Architecture : - Design normalised relational schemas in SQL Server to support pharmaceutical analytics use cases. - Collaborate with Data Scientists on data preparation and feature engineering pipelines. - Maintain data dictionaries, schema documentation, and lineage records for all active pipelines. - Contribute to architecture decisions on data storage, processing strategies, and API integrations. Quality, Validation & Reliability : - Own data quality across assigned pipelines implement automated checks, alerting, and audit trails. - Investigate and resolve silent data errors including missing fields, incorrect labels, zero-value nodes, and schema mismatches. - Write thorough unit and integration tests for all pipeline components. Collaboration & Standards : - Work within an agile delivery framework participate in sprint planning, estimation, reviews, and retrospectives. - Follow and contribute to team-wide GitHub standards: branching strategy, PR reviews, naming conventions, and documentation. - Actively use AI coding tools (GitHub Copilot, Claude, etc.) as part of everyday development. - Produce clear technical documentation and handoff notes for every delivery. Required Skills & Experience : Technical Skills Tools & Platforms : - Python (3+ years, production pipelines) - SQL Server (primary) - SQL complex queries, stored procedures, optimisation - Git / GitHub (branching, PRs, code review) - ETL/ELT design and implementation - Pandas, SQLAlchemy, Pydantic - Data modelling and schema design - Excel / openpyxl for source data handling - REST API integration and data extraction - Docker or containerised environments (desirable) - Data validation and testing practices - Selenium / web scraping tools (desirable) - JSON / XML data processing - Airflow or equivalent orchestration (desirable) Nice to Have : - Experience working with pharmaceutical or life sciences data clinical trials, market forecasting, drug pipelines, or therapy area analytics. - Familiarity with NLP pipelines or LLM-assisted data processing workflows. - Knowledge of multi-agent AI architectures or experience building pipelines that feed AI/ML models. - Experience building Streamlit or lightweight data exploration tools. What We Expect From You : - You think before you code research, design, and plan before diving into implementation. - You take end-to-end ownership of your pipelines from requirement to optimisation. - You are comfortable flagging issues early rather than discovering them post-handoff. - You write code that your teammates can read, understand, and maintain. - You embrace AI tools as a productivity multiplier, not a threat. - You contribute to a team culture of quality, transparency, and continuous improvement. (ref:hirist.tech) We are looking for a skilled Data Engineer, who can design and maintain robust data pipelines, work with large-scale structured and semi-structured pharma datasets, and collaborate closely with Data Scientists and Python Developers to deliver clean, validated, production-grade data infrastructure. Key Responsibilities : Pipeline Design & Development : - Design and build scalable ETL/ELT pipelines for ingesting, transforming, and loading pharmaceutical data from multiple sources. - Process indication-level market forecast data from Excel workbooks into structured JSON and DB-ready formats for frontend visualisation. - Build and maintain data extraction pipelines from clinical trial registries, conference abstracts, and drug databases. - Implement multi-step validation workflows to ensure pipeline output accuracy and completeness before downstream handoff. Data Modelling & Architecture : - Design normalised relational schemas in SQL Server to support pharmaceutical analytics use cases. - Collaborate with Data Scientists on data preparation and feature engineering pipelines. - Maintain data dictionaries, schema documentation, and lineage records for all active pipelines. - Contribute to architecture decisions on data storage, processing strategies, and API integrations. Qual