Data Engineer

Education: • Bachelor’s degree in Computer Science, Computer Engineering, Data Engineering, or a relevant field. Required Experience: • 5–8+ years of solid understanding in data analytics concepts (data ingestion, data warehouse, data lakes, reporting, etc.). • 5+ years of experience delivering production ETL pipelines. • 2+ years of hands-on experience with Databricks (Azure Databricks preferred). • Built, optimized, and maintained ETL pipelines using Databricks for large-scale data processing. • Strong experience with PySpark and SQL; proficiency in Python. • Strong experience in Azure cloud and data platform services including Data Factory, Databricks, databases, ETL/ELT concepts, IAM, and security controls. • Strong understanding of cloud technologies, data management, big data solutions, machine learning, and AI frameworks. • Experience with ADF for orchestration and operational monitoring. • Experience implementing CI/CD practices using GitHub Actions. • Knowledge/experience in DevOps and Agile methodologies. • Collaborate with cross teams, infrastructure, and security teams for best and timely delivery. Preferred Qualifications: • Experience with Spark/Delta optimization (partitioning, file sizing, performance tuning). • Experience in healthcare/Life Sciences is preferred

Technical Skills: • Azure Databricks (workspace management, cluster configuration, notebooks, jobs) • PySpark and Spark SQL for large-scale data processing • Python for scripting, automation, and data engineering tasks • Unity Catalog for data governance, access controls, and asset organization • Azure Data Factory (ADF) for pipeline orchestration, scheduling, and monitoring • Lakehouse architecture patterns (Bronze/Silver/Gold data layers) • Delta Lake format and optimization techniques • CI/CD pipelines using GitHub Actions • Git-based version control and collaborative development workflows • Strong SQL skills for data transformation and validation • Knowledge of data quality and testing frameworks • Knowledge of DevOps and Agile methodologies Knowledge, Skills & Abilities: • Strong analytics and problem-solving skills with experience across a broad spectrum of analytics • Strong written and verbal communication skills • Strong team player working on cross-functional deliverables