Source description
About the role
Lead end-to-end design, development, and optimisation of data pipelines on Azure Databricks Manage large-scale ETL/ELT workflows using Delta Lake, PySpark, and Azure Data Factory Implement scalable data models and ensure high-performance processing for analytics use-cases Drive Azure cloud architecture decisions across Databricks, ADLS, ADF, Synapse, and related services Optimise Spark jobs for performance, cost efficiency, and best-practice coding standards Collaborate with data scientists, analysts, and business stakeholders to deliver robust data solutions Ensure data quality, governance, security, and compliance across platforms Troubleshoot production issues and perform root-cause analysis with proactive monitoring Mentor junior engineers and contribute to solution design reviews and technical roadmaps Experience in Git, CI/CD, DevOps, and Agile methodology is mandatory