Source description
About the role
Summary We are looking for a highly skilled Databricks Data Engineer to design, develop, and optimize scalable data pipelines and enterprise-grade data solutions. This role involves working on Azure cloud platforms, ensuring data quality, reliability, and enabling data-driven decision-making across business functions. Location Bangalore / Gurgaon Your Future Employer A globally recognized analytics and operations management firm, known for delivering data-driven insights and advanced analytics solutions across industries including insurance, banking, healthcare, and more. The organization fosters innovation, learning, and growth, working with large-scale, cutting-edge data ecosystems Responsibilities 1. Design, develop, and optimize scalable data pipelines using Azure Databricks and PySpark. 2. Build and manage ETL/ELT workflows, ensuring efficient data ingestion, transformation, and processing. 3. Implement data quality frameworks, validation checks, and monitoring mechanisms for reliable data pipelines. 4. Perform performance tuning for Spark jobs, workflows, and data platforms to ensure cost and efficiency optimization. 5. Collaborate with cross-functional teams (architects, analysts, product teams) to deliver business-aligned data solutions. 6. Ensure governance, compliance, and best practices using tools like Unity Catalog and Azure data services. 7. Mentor junior team members and drive technical excellence across the team. Requirements 1. 512 years of experience in Data Engineering with Databricks & Azure Cloud 2. Solid hands-on expertise in PySpark, Python, and SQL 3. Experience in ETL/ELT pipeline development, data modelling, and data warehousing. 4. Proficiency in Azure components such as ADF, Databricks, Azure Data Lake. 5. Experience with Delta Lake, Delta Live Tables, Autoloader, Unity Catalog 6. Strong analytical, problem-solving, and communication skills What is in it for you - Work in a fast-paced, innovative analytics environment with global exposure - Opportunity to collaborate with highly experienced data professionals. - Continuous learning and development on advanced data and analytics tools Reach us If you think this role is aligned with your career, kindly write me an email along with your updated CV a for a confidential discussion on the role. Disclaimer Crescendo Global specializes in Senior to C-level niche recruitment. We are passionate about empowering job seekers and employers with an engaging and memorable hiring experience. Crescendo Global does not discriminate on the basis of race, religion, gender, or any protected status. Note We receive a large number of applications daily, and it may not be possible to respond to every applicant. If you do not hear back within one week, please assume your profile has not been shortlisted. [Web-Ad_JD Format new | PDF] Profile Keywords Crescendo Global, Databricks Jobs, Azure Data Engineer, Data Engineering Jobs, PySpark, ETL Developer, Data Pipeline, Azure Data Factory, Delta Lake, Big Data, SQL, Python, Analytics Jobs, India Jobs Summary We are looking for a highly skilled Databricks Data Engineer to design, develop, and optimize scalable data pipelines and enterprise-grade data solutions. This role involves working on Azure cloud platforms, ensuring data quality, reliability, and enabling data-driven decision-making across business functions. Location Bangalore / Gurgaon Your Future Employer A globally recognized analytics and operations management firm, known for delivering data-driven insights and advanced analytics solutions across industries including insurance, banking, healthcare, and more. The organization fosters innovation, learning, and growth, working with large-scale, cutting-edge data ecosystems Responsibilities 1. Design, develop, and optimize scalable data pipelines using Azure Databricks and PySpark. 2. Build and manage ETL/ELT workflows, ensuring efficient data ingestion, transformation, and processing. 3. Implement data quality frameworks, validation checks, and monitoring mechanisms for reliable data pipelines. 4. Perform performance tuning for Spark jobs, workflows, and data platforms to ensure cost and efficiency optimization. 5. Collaborate with cross-functional teams (architects, analysts, product teams) to deliver business-aligned data solutions. 6. Ensure governance, compliance, and best practices using tools like Unity Catalog and Azure data services. 7. Mentor junior team members and drive technical excellence across the team. Requirements 1. 512 years of experience in Data Engineering with Databricks & Azure Cloud 2. Solid hands-on expertise in PySpark, Python, and SQL 3. Experience in ETL/ELT pipeline development, data modelling, and data warehousing. 4. Proficiency in Azure components such as ADF, Databricks, Azure Data Lake. 5. Experience with Delta Lake, Delta Live Tables, Autoloader, Unity Catalog 6. Strong analytical, problem-solving, and communication skills
More at Crescendo Global Leadership Hiring India