Data Pipeline Engineer

Role Overview: You will be joining as an Azure Data Engineer in an agile team, where your main responsibility will be designing, developing, and maintaining scalable data pipelines and workflows using Microsoft Azure cloud services. Your role will involve collaborating with data scientists, analysts, and stakeholders to ensure data solutions align with business objectives. You will also be responsible for optimizing data performance, ensuring data quality, integrity, and security, as well as troubleshooting and resolving data pipeline issues. Key Responsibilities: - Design, develop, and maintain scalable data pipelines and workflows using Azure Data Factory, Azure Databricks, and other relevant tools. - Implement and optimize data storage solutions in Azure. - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and implement solutions that align with business objectives. - Ensure data quality, integrity, and security in all data-related processes and implementations. - Work with both structured and unstructured data, implementing data transformation and cleansing processes. - Optimize and fine-tune data solutions' performance to meet real-time and batch processing requirements. - Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and optimal performance. - Stay updated with industry trends and best practices, proactively recommending improvements to existing data infrastructure. - Conduct data analysis, validation, and rectification of master data inconsistencies. Qualifications Required: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Proven experience (3-5 Years) as a Data Engineer with a focus on Microsoft Azure technologies. - Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse Analytics. - Strong proficiency in SQL, experience with data modeling, and ETL processes. - Familiarity with data integration and orchestration tools. - Knowledge of data warehousing concepts and best practices. - Experience with version control systems, preferably Git. - Excellent problem-solving and communication skills. Additional Company Details: The company culture encourages dreaming big and creating a future with more cheers. Role Overview: You will be joining as an Azure Data Engineer in an agile team, where your main responsibility will be designing, developing, and maintaining scalable data pipelines and workflows using Microsoft Azure cloud services. Your role will involve collaborating with data scientists, analysts, and stakeholders to ensure data solutions align with business objectives. You will also be responsible for optimizing data performance, ensuring data quality, integrity, and security, as well as troubleshooting and resolving data pipeline issues. Key Responsibilities: - Design, develop, and maintain scalable data pipelines and workflows using Azure Data Factory, Azure Databricks, and other relevant tools. - Implement and optimize data storage solutions in Azure. - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and implement solutions that align with business objectives. - Ensure data quality, integrity, and security in all data-related processes and implementations. - Work with both structured and unstructured data, implementing data transformation and cleansing processes. - Optimize and fine-tune data solutions' performance to meet real-time and batch processing requirements. - Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and optimal performance. - Stay updated with industry trends and best practices, proactively recommending improvements to existing data infrastructure. - Conduct data analysis, validation, and rectification of master data inconsistencies. Qualifications Required: - Bachelor's degree in Computer Science, Information Technology, or a related field. - Proven experience (3-5 Years) as a Data Engineer with a focus on Microsoft Azure technologies. - Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse Analytics. - Strong proficiency in SQL, experience with data modeling, and ETL processes. - Familiarity with data integration and orchestration tools. - Knowledge of data warehousing concepts and best practices. - Experience with version control systems, preferably Git. - Excellent problem-solving and communication skills. Additional Company Details: The company culture encourages dreaming big and creating a future with more cheers.