Source description
About the role
Roles and Responsibilities : Design, develop, test, deploy and maintain large-scale data pipelines using Airflow to extract, transform, load (ETL) data from various sources into a centralized repository. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs through ETL processes. Develop complex SQL queries to optimize database performance and troubleshoot issues related to data quality, integrity, and availability. Ensure compliance with security standards by implementing access controls on sensitive data assets. Job Requirements : 6-10 years of experience in Data Engineering with expertise in Airflow for building scalable ETL workflows. Strong understanding of Python programming language for developing custom scripts and integrations. Experience working with relational databases such as MySQL or PostgreSQL for querying large datasets. Roles and Responsibilities : Design, develop, test, deploy and maintain large-scale data pipelines using Airflow to extract, transform, load (ETL) data from various sources into a centralized repository. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs through ETL processes. Develop complex SQL queries to optimize database performance and troubleshoot issues related to data quality, integrity, and availability. Ensure compliance with security standards by implementing access controls on sensitive data assets. Job Requirements : 6-10 years of experience in Data Engineering with expertise in Airflow for building scalable ETL workflows. Strong understanding of Python programming language for developing custom scripts and integrations. Experience working with relational databases such as MySQL or PostgreSQL for querying large datasets.
More at Virtusa