Source description
About the role
Job Description: We are looking for a skilled Data Engineer with hands-on experience in Google Cloud Platform (GCP) , PySpark , SQL , and ETL processes . The ideal candidate will be responsible for building, optimizing, and maintaining scalable data pipelines and workflows using technologies such as Apache Airflow , PySpark , and cloud-native tools . Key Responsibilities : Design, develop, and maintain efficient and scalable data pipelines using PySpark and SQL . Build and manage workflows/orchestration using Apache Airflow . Work with GCP services such as BigQuery, Cloud Storage, Dataflow, and Composer. Implement and optimize ETL processes to ensure data quality, consistency, and reliability. Collaborate with data analysts, data scientists, and other engineering teams to support data needs. Monitor pipeline performance and troubleshoot issues as they arise. Write clean, maintainable, and well-documented code. Required Skills & Qualifications : Strong programming skills in PySpark and SQL . Proven experience with Google Cloud Platform (GCP) . Solid understanding of ETL concepts and data pipeline design . Hands-on experience with Apache Airflow (or similar orchestration tools). Familiarity with cloud-based data warehousing and storage systems (e.g., BigQuery, Cloud Storage). Experience with performance tuning and optimizing large-scale data pipelines. Good problem-solving skills and the ability to work independently or as part of a team.
More at HARP