Source description
About the role
As a Senior/Lead Databricks Developer, you will be responsible for designing and developing enterprise-scale data pipelines using Databricks. Your key responsibilities will include: - Building and optimizing Lakehouse architectures with Bronze, Silver, and Gold layers. - Developing robust ETL/ELT frameworks and data quality solutions. - Driving performance tuning, monitoring, and operational excellence. - Implementing CI/CD best practices and automated testing frameworks. - Collaborating with cross-functional teams to deliver analytics-ready data products. - Mentoring junior engineers and contributing to engineering standards and best practices. You should possess the following required skills: - Proficiency in Databricks, Python, PySpark/Spark SQL, SQL/PL-SQL, Delta Lake/Lakehouse Architecture, Azure Cloud Services, Azure DevOps & Git, and ETL/ELT Tools (ADF, Fivetran, or equivalent). - Experience in CI/CD Pipeline Management. Preferred qualifications include: - Experience with Terraform and Unity Catalog. - Databricks Certification(s). - Exposure to Pharmaceutical, Healthcare, Life Sciences, or other data-intensive industries. As a Senior/Lead Databricks Developer, you will be responsible for designing and developing enterprise-scale data pipelines using Databricks. Your key responsibilities will include: - Building and optimizing Lakehouse architectures with Bronze, Silver, and Gold layers. - Developing robust ETL/ELT frameworks and data quality solutions. - Driving performance tuning, monitoring, and operational excellence. - Implementing CI/CD best practices and automated testing frameworks. - Collaborating with cross-functional teams to deliver analytics-ready data products. - Mentoring junior engineers and contributing to engineering standards and best practices. You should possess the following required skills: - Proficiency in Databricks, Python, PySpark/Spark SQL, SQL/PL-SQL, Delta Lake/Lakehouse Architecture, Azure Cloud Services, Azure DevOps & Git, and ETL/ELT Tools (ADF, Fivetran, or equivalent). - Experience in CI/CD Pipeline Management. Preferred qualifications include: - Experience with Terraform and Unity Catalog. - Databricks Certification(s). - Exposure to Pharmaceutical, Healthcare, Life Sciences, or other data-intensive industries.
More at Spectral Consultants