Source description
About the role
Role Title: Azure Data Engineer
BU/Segment: Healthcare Analytics
Responsibilities
-
Design, develop, test, and deploy robust and scalable data pipelines using Azure Databricks, PySpark, and Spark SQL for processing large volumes of structured and unstructured data.
-
Implement workflow orchestration using Azure Data Factory and Databricks Workflows to automate ETL/ELT processes.
-
Implement and maintain CI/CD pipelines for Databricks notebooks, scripts, and configurations using Azure DevOps or GitHub Actions, promoting code across dev, test, and prod environments.
-
Develop and maintain healthcare data models, including data entities, relationships, and data flows, to support healthcare analytics and decision-making.
-
Design and implement enterprise-wide BI architectures. Develop and maintain data warehouses and data marts.
-
Develop and maintain reports, dashboards, and data visualizations. Collaborate with stakeholders to understand business needs and provide data-driven insights
-
Person attributes
-
6+ years of experience as a Data Engineer with a proven track record of building data pipelines in the cloud.
-
3+ years of hands-on experience with Azure Databricks
-
Strong experience with core Azure data services – Azure Data Lake Storage (Gen 2), Azure data factory.
-
Proficiency in Python and expert-level skills in SQL
-
Solid understanding of data warehousing concepts, dimensional modeling (star/snowflake schemas), and data architecture principles
-
Experience with CI/CD pipelines (Azure DevOps, Jenkins, GitHub Actions) and version control systems (Git).
-
Demonstrated experience and knowledge of strategic problem solving and frameworks, and project management skills
-
Excellent written and verbal communication with the ability to establish credibility and strong relationships with stakeholders
More at EXL