Source description
About the role
Lead the design and implementation of end-to-end data pipelines using Azure Databricks and Spark Architect data lakehouse solutions leveraging Delta Lake, Azure Data Lake, and Synapse Define and enforce data engineering best practices, coding standards, and governance policies Collaborate with business and analytics teams to translate requirements into technical solutions Oversee data ingestion, transformation, and modeling processes Mentor and guide junior data engineers and analysts Ensure performance tuning, cost optimization, and security compliance Drive CI/CD automation and DevOps practices for data workflows Stay current with Azure and Databricks platform updates and innovations Mandatory skills 8+ years of experience in data engineering, with 3+ years in Azure Databricks Expertise in Apache Spark, Scala, and Delta Lake Strong proficiency in Azure Data Factory, Azure Data Lake, Azure Synapse Deep understanding of data modeling, ETL/ELT, and data architecture Experience with version control (Git) and CI/CD tools (e. g. , Azure DevOps) Familiarity with data visualization tools (Power BI, Tableau) Excellent communication, leadership, and stakeholder management skills Desired/ Secondary skills Good experience in CI/CD pipelines for data engineering workflow
More at CLIFYX INC