Source description
About the role
We are looking for 4-7 years of experienced Data Engineer Databricks to design and develop scalable data platforms using the Databricks Lakehouse architecture. The ideal candidate should have strong hands-on experience with PySpark, SQL, Delta Lake, Auto Loader, Spark Declarative Pipelines (DLT/SDP), Databricks Workflows, and Unity Catalog. Key Responsibilities Strong proficiency in PySpark and SQL for large-scale data processing.Practical experience with Delta Lake / Delta Tables (ACID transactions, schema enforcement, versioning).Experience building ingestion pipelines with Auto Loader (cloudFiles, schema inference/evolution).Hands-on experience with SDP (Spark Declarative Pipelines / DLT) for building declarative data pipelines.Experience orchestrating pipelines with Databricks Workflows (jobs, tasks, triggers).Solid understanding of Unity Catalog for governance, permissions, and metastore management.Good grasp of data warehouse concepts dimensional modeling, data marts, slowly changing dimensions.Experience building or supporting AI/BI dashboards (Databricks SQL Dashboards, Genie, or similar BI tools).Integrate Databricks with cloud storage and cloud-native services across Azure, AWS, or GCP. .
More at Talentmatics