Senior Software Azure Data Engineer

Role Overview: You are seeking a highly experienced Senior Azure Data Engineer with 4-6 years of experience to join the growing data team in Bangalore/Chennai. In this role, you will be responsible for designing, developing, and operationalizing large-scale data processing systems on the Azure cloud platform. The shift timing for this role is 1.00 - 10.00 PM for 3 days a week working from the office. Key Responsibilities: - Design, build, and maintain scalable data pipelines for processing large volumes of structured and unstructured data. - Develop and optimize high-performance Spark jobs using PySpark and Spark SQL in Azure Databricks. - Implement data storage solutions using Azure Data Lake Storage (Gen2) following Medallion Architecture (Bronze Silver Gold layers). - Collaborate with data architects, analysts, and business stakeholders to translate data requirements into technical solutions. - Implement robust data security measures including RBAC, ACLs, encryption, and data masking. - Perform data modeling for scalable analytics in both batch and real-time use cases. - Monitor, troubleshoot, and optimize pipelines and Databricks workloads for performance and cost efficiency. - Automate deployment using CI/CD pipelines (Azure DevOps, GitHub Actions) and IaC tools (Terraform, Bicep). - Establish and enforce data quality governance and best practices across the platform. - Mentor junior engineers and drive engineering excellence within the team. Qualifications Required: - 5 years of experience in data engineering with enterprise-scale data solutions. - 3 years of hands-on experience with Microsoft Azure data services. - Core Technical Expertise in Azure Databricks, Azure Data Lake Storage (Gen2), and Medallion Architecture. - Expert programming skills in Python, advanced PySpark expertise, and strong SQL skills. - Solid understanding of data modeling and experience with orchestration tools like Azure Data Factory or Apache Airflow. - Proficiency with Git and collaborative development workflows. - Preferred qualifications include Microsoft Azure Data Engineer Associate (DP-203) certification, experience with Delta Lake, real-time processing, knowledge of Azure ecosystem tools, DevOps/DataOps practices, and familiarity with visualization tools. Company Details: At CGI, life is rooted in ownership, teamwork, respect, and belonging. The company values ownership from day one, innovative solutions development, building relationships, and career growth opportunities in a supportive environment. CGI is one of the largest IT and business consulting services firms globally, offering a platform to scale ideas, embrace new opportunities, and benefit from industry expertise. Role Overview: You are seeking a highly experienced Senior Azure Data Engineer with 4-6 years of experience to join the growing data team in Bangalore/Chennai. In this role, you will be responsible for designing, developing, and operationalizing large-scale data processing systems on the Azure cloud platform. The shift timing for this role is 1.00 - 10.00 PM for 3 days a week working from the office. Key Responsibilities: - Design, build, and maintain scalable data pipelines for processing large volumes of structured and unstructured data. - Develop and optimize high-performance Spark jobs using PySpark and Spark SQL in Azure Databricks. - Implement data storage solutions using Azure Data Lake Storage (Gen2) following Medallion Architecture (Bronze Silver Gold layers). - Collaborate with data architects, analysts, and business stakeholders to translate data requirements into technical solutions. - Implement robust data security measures including RBAC, ACLs, encryption, and data masking. - Perform data modeling for scalable analytics in both batch and real-time use cases. - Monitor, troubleshoot, and optimize pipelines and Databricks workloads for performance and cost efficiency. - Automate deployment using CI/CD pipelines (Azure DevOps, GitHub Actions) and IaC tools (Terraform, Bicep). - Establish and enforce data quality governance and best practices across the platform. - Mentor junior engineers and drive engineering excellence within the team. Qualifications Required: - 5 years of experience in data engineering with enterprise-scale data solutions. - 3 years of hands-on experience with Microsoft Azure data services. - Core Technical Expertise in Azure Databricks, Azure Data Lake Storage (Gen2), and Medallion Architecture. - Expert programming skills in Python, advanced PySpark expertise, and strong SQL skills. - Solid understanding of data modeling and experience with orchestration tools like Azure Data Factory or Apache Airflow. - Proficiency with Git and collaborative development workflows. - Preferred qualifications include Microsoft Azure Data Engineer Associate (DP-203) certification, experience with Delta Lake, real-time processing, knowledge of Azure ecosystem tools, DevOps/DataOps practices, and familiarity with visualization too