Azure Data Engineer Manager

Key Responsibilities - Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Services. - Build and optimize data ingestion and transformation workflows utilizing Azure Data Factory (ADF) and Azure Databricks. - Develop data processing solutions employing PySpark and Delta Lake architecture. - Implement and manage data storage solutions using Azure Data Lake Storage (ADLS). - Create and manage data pipelines using Delta Live Tables (DLT). - Establish and enforce data governance, security, and access controls utilizing Unity Catalog. - Monitor, troubleshoot, and enhance data platform performance and reliability. - Collaborate with business stakeholders, architects, and development teams to understand and deliver on data requirements. Overview - The Azure Data Engineer Manager role is pivotal in leading the development and optimization of scalable data solutions on the Azure cloud platform. - Situated in Bangalore, this role requires a seasoned professional with 9-12 years of experience, including at least 8 years of relevant work with Azure data services. - In a consulting capacity, the Azure Data Engineer Manager will oversee the design and execution of advanced data engineering projects, ensuring high performance and reliability of data solutions. - The individual will act as a key advisor, providing strategic insights into data architecture and governance. - This role demands a strong technical background, particularly in Azure Data Factory, Azure Databricks, PySpark, and other modern data engineering tools. - The manager will lead a team of engineers, fostering a collaborative environment to drive innovation and efficiency. - As an expert in data engineering concepts and ETL frameworks, the Azure Data Engineer Manager will collaborate closely with business stakeholders to align data strategies with business objectives. - The successful candidate will demonstrate strong leadership skills, with the ability to manage a team and communicate effectively with both technical and non-technical stakeholders. Requirements - 9-12 years of experience in data engineering with at least 8 years of relevant experience in Azure data services. - Expertise in Azure Data Factory, Azure Data Lake Storage, Azure Databricks, and PySpark. - Proficiency in Delta Live Tables, Unity Catalog, and SQL for data management and optimization. - Strong communication skills with the ability to explain technical concepts to non-technical stakeholders. - Experience with CI/CD implementation using Azure DevOps/GitHub Actions. - Knowledge of Python and Scala programming languages. - Azure or Databricks certifications are highly desirable.