Azure Data Engineer - SQL / Python (Bengaluru)

Main Roles and Responsibilities : - Design, develop, and implement ETL pipelines for the continuous deployment and integration. - Collaborate with other data engineers and data scientists to understand data requirements and optimize data solution processes. - Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements - The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.) - Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications - Build data pipelines to collectively bring data together - Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse - Implement best practices for pipeline building and governance. - Troubleshoot and resolve issues related to pipeline deployment and performance. - Ensure compliance with security and data privacy standards - Solution design using Microsoft Azure services and other tools Required skills and experience : - 2-3 years of experience in Azure Data Analytics, Synapse, DevOps. - Proficiency in programming languages such as Python, SQL, Py-Spark, Spark-SQL. - Experience with Azure: ADLS, Synapse, Stream Analytics, SQL DW, Analysis Services, Serverless Architecture, ARM Templates - Experience with cloud platforms like Azure, AWS or and their respective services. - Knowledge of CI/CD pipelines, automation tools and version control systems like Git. - Robust problem-solving skills and ability to troubleshoot complex issues. - Experience with monitoring tools. - Ability to work collaboratively in cross-functional teams. Nice to have: - Should have Data Engineer certification from any discipline - Azure, AWS. - Azure Synapse Studio, Data bricks, Fabric - Familiarity with agile software development lifecycle (SCRUM, Kanban, etc.) Location - Hyderabad,Ahmedabad,pune,chennai,kolkata. Main Roles and Responsibilities : - Design, develop, and implement ETL pipelines for the continuous deployment and integration. - Collaborate with other data engineers and data scientists to understand data requirements and optimize data solution processes. - Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements - The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.) - Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications - Build data pipelines to collectively bring data together - Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse - Implement best practices for pipeline building and governance. - Troubleshoot and resolve issues related to pipeline deployment and performance. - Ensure compliance with security and data privacy standards - Solution design using Microsoft Azure services and other tools Required skills and experience : - 2-3 years of experience in Azure Data Analytics, Synapse, DevOps. - Proficiency in programming languages such as Python, SQL, Py-Spark, Spark-SQL. - Experience with Azure: ADLS, Synapse, Stream Analytics, SQL DW, Analysis Services, Serverless Architecture, ARM Templates - Experience with cloud platforms like Azure, AWS or and their respective services. - Knowledge of CI/CD pipelines, automation tools and version control systems like Git. - Robust problem-solving skills and ability to troubleshoot complex issues. - Experience with monitoring tools. - Ability to work collaboratively in cross-functional teams. Nice to have: - Should have Data Engineer certification from any discipline - Azure, AWS. - Azure Synapse Studio, Data bricks, Fabric - Familiarity with agile software development lifecycle (SCRUM, Kanban, etc.) Location - Hyderabad,Ahmedabad,pune,chennai,kolkata.