Databricks Data Engineer

Job Description & Summary: We are seeking an experienced Data Engineer Databricks to design, build, and operate scalable, high-performance data pipelines on the Databricks Lakehouse Platform. The role involves hands-on development using Apache Spark, Databricks notebooks, Delta Lake, and cloud-native services, along with close collaboration with analytics, AI/ML, and business teams Job Position Title: Senior Associate_ Data Engineer Databricks_Data and Analytics_Advisory_Gurgaon & Bangalore Responsibilities: Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL)Implement batch and incremental data processing using Delta Lake and multi-hop architectureDevelop and optimize Databricks notebooks, jobs, and workflowsIngest, transform, and curate large-scale structured and semi-structured datasetsSupport analytics, reporting, and downstream data consumption use casesEnsure data quality, reliability, lineage, and governanceCollaborate with data scientists, analysts, and architects on AI/ML workloadsOptimize Spark jobs for performance and cost efficiencyAdhere to enterprise security, access control, and compliance standardsProvide production support and troubleshoot data pipeline issuesDocument technical designs, data flows, and operational runbooksMentor junior engineers and contribute to best practices Mandatory skill sets: 5+ years of experience as a Data Engineer with strong Databricks expertiseHands-on experience with Apache Spark, PySpark, and Spark SQLStrong knowledge of Delta Lake and Lakehouse architectureAdvanced SQL skillsExperience with ETL/ELT patterns and data warehousing conceptsExposure to at least one cloud platform (Azure / AWS / GCP)Understanding of distributed computing conceptsExperience working in Agile teams Preferred skill sets: Good to have working knowledge of RAG, Vector DB, LLM and Agentic AI based implementation within Databricks. Experience with Unity Catalog and data governanceExposure to Auto Loader or streaming frameworksCI/CD for data pipelinesPython for data engineering and automationDatabricks certification (Associate / Professional) Years of experience required: 5 to 12 years Education qualification: Bachelors or Masters degree in Computer Science, Engineering, or related field (60% above) Job Description & Summary: We are seeking an experienced Data Engineer Databricks to design, build, and operate scalable, high-performance data pipelines on the Databricks Lakehouse Platform. The role involves hands-on development using Apache Spark, Databricks notebooks, Delta Lake, and cloud-native services, along with close collaboration with analytics, AI/ML, and business teams Job Position Title: Senior Associate_ Data Engineer Databricks_Data and Analytics_Advisory_Gurgaon & Bangalore Responsibilities: Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL)Implement batch and incremental data processing using Delta Lake and multi-hop architectureDevelop and optimize Databricks notebooks, jobs, and workflowsIngest, transform, and curate large-scale structured and semi-structured datasetsSupport analytics, reporting, and downstream data consumption use casesEnsure data quality, reliability, lineage, and governanceCollaborate with data scientists, analysts, and architects on AI/ML workloadsOptimize Spark jobs for performance and cost efficiencyAdhere to enterprise security, access control, and compliance standardsProvide production support and troubleshoot data pipeline issuesDocument technical designs, data flows, and operational runbooksMentor junior engineers and contribute to best practices Mandatory skill sets: 5+ years of experience as a Data Engineer with strong Databricks expertiseHands-on experience with Apache Spark, PySpark, and Spark SQLStrong knowledge of Delta Lake and Lakehouse architectureAdvanced SQL skillsExperience with ETL/ELT patterns and data warehousing conceptsExposure to at least one cloud platform (Azure / AWS / GCP)Understanding of distributed computing conceptsExperience working in Agile teams Preferred skill sets: Good to have working knowledge of RAG, Vector DB, LLM and Agentic AI based implementation within Databricks. Experience with Unity Catalog and data governanceExposure to Auto Loader or streaming frameworksCI/CD for data pipelinesPython for data engineering and automationDatabricks certification (Associate / Professional) Years of experience required: 5 to 12 years Education qualification: Bachelors or Masters degree in Computer Science, Engineering, or related field (60% above)