Source description
About the role
Job description: We are seeking a highly experienced Senior Databricks Data Engineer to design, build, and optimize our data lake house environment. You will leverage your 10+ years of expertise to develop complex data pipelines, ensure data quality, and drive innovation using Databricks and Apache Spark. Responsibilities: Design and implement scalable and robust data pipelines using Databricks and Apache Spark. Architect and optimize data lake house solutions for performance and reliability. Develop and maintain complex ETL/ELT processes for large-scale data ingestion and transformation. Write and optimize high-performance SQL and Spark code for data processing and analysis. Implement data quality checks, data validation, and data governance practices. Troubleshoot and resolve complex data processing issues and performance bottlenecks. Collaborate with data scientists, analysts, and other engineering teams to deliver data-driven solutions. Provide technical leadership and mentorship to junior engineers.
More at E4 Software Services