Lead Azure Data engineer

Job Title: Lead Azure Data Engineer Location: Bangalore, Chennai, Hyderabad, Pune Expereinece: 8-15 Years Work Type: Contract (Extendable) Job Description We are looking for an experienced Lead Data Engineer with 8 + years of experience in designing, developing, and maintaining enterprise-scale data engineering solutions. The ideal candidate should have strong expertise in Microsoft Fabric, Databricks, PySpark, SQL/T-SQL, Delta Lake, Azure Data Factory, and Power BI . You will play a key role in modernizing legacy data platforms by building scalable Bronze, Silver, and Gold Medallion Architecture pipelines and migrating SQL Server ETL workloads to Microsoft Fabric. Roles & Responsibilities Design, develop, and maintain scalable Bronze, Silver, and Gold data pipelines using Microsoft Fabric and Databricks. Implement Cascade 1.0 and Cascade 2.0 merge logic within the Silver layer, managing schema evolution, new attributes, and null value handling. Migrate legacy SQL Server objects including tables, views, stored procedures, and ETL processes into Microsoft Fabric Pipelines, Fabric Notebooks, and Databricks. Develop high-performance PySpark transformations and build optimized Delta Lake tables for enterprise data processing. Write and optimize complex SQL/T-SQL queries, stored procedures, views, functions, CTEs, and performance-tuned scripts. Design and develop Azure Data Factory (ADF) pipelines for orchestrating enterprise data ingestion and transformation. Build and maintain scalable Lakehouse solutions using Microsoft Fabric and OneLake. Perform data ingestion, cleansing, transformation, validation, and reconciliation to ensure data quality and consistency. Establish data quality checks, lineage, reconciliation (apple-to-apple validation), and governance across enterprise datasets. Develop and optimize semantic models and support Power BI datasets for enterprise reporting and analytics. Collaborate with Data Engineers, BI Developers, QA teams, and business stakeholders to deliver high-quality data solutions. Optimize pipeline performance, resource utilization, and overall processing costs. Participate in Agile ceremonies including sprint planning, daily stand-ups, sprint reviews, retrospectives, and code reviews. Prepare technical documentation for data pipelines, migration activities, architecture, and operational support. Required Skills Microsoft Fabric (Lakehouse, OneLake, Fabric Pipelines) Databricks PySpark SQL / T-SQL Delta Lake Azure Data Factory (ADF) Power BI (Datasets & Semantic Models) Data Engineering Medallion Architecture (Bronze, Silver, Gold) ETL Development Data Migration Data Quality & Data Lineage Agile / Scrum