Source description
About the role
Manage and monitor ETL/ELT pipelines, analytics reporting jobs, ML workflows across Azure services, ensuring SLA compliance and the timely resolution of Incidents, failures and performance bottlenecks. · Proactively monitor platform health, performance, and SLAs using Databricks monitoring, logging and alerting, to minimize downtime and data issues. · Perform root cause analysis and implement long term fixes that prevent recurring issues and improve overall operational reliability. · Partner with data engineers, ML engineers, and BI engineers to troubleshoot issues, implement improvements, automate operations, and drive continuous enhancement of data/ML platform reliability. · Contribute to platform automation, CI/CD for data/ML pipelines. · Provide support for tools that are owned by Enterprise Data Management. · Develop and maintain code enhancements, automation scripts that support both data engineering and operational needs. · Provide after-hours support for critical systems and applications. · Develop and maintain documentation, runbooks, and operational standards. · Strong experience in SQL, Python and PySpark · Extensive hands-on experience with Azure Databricks with Unity Catalog, ADF, ADLS · Excellent problem-solving skills and attention to detail. · Strong communication and collaboration abilities across technical and non-technical teams.
More at Tavant