Source description
About the role
Responsibilities: Design, develop, and maintain scalable data pipelines using Python and PySpark
Build and optimize ETL/ELT workflows using Apache Airflow Develop data transformation models and workflows using dbt Work with BigQuery for data warehousing, querying, and optimization Write efficient and optimized SQL queries for large-scale data processing Ensure data quality, integrity, and reliability across pipelines Collaborate with Data Analysts, Data Scientists, and business stakeholders for data requirements Monitor and troubleshoot production data workflows and resolve issues proactively Implement best practices for logging, monitoring, testing, and CI/CD Optimize cloud resource usage and cost efficiency on GCP Participate in code reviews and engineering best practices
Required Skills: Must have 2–4 years of experience in Data Engineering or related roles Strong hands-on experience with Python and SQL Experience with PySpark and distributed data processing Good knowledge of BigQuery and GCP services Experience with Apache Airflow for orchestration Hands-on experience with dbt for data transformation and modeling Understanding of ETL/ELT frameworks and data warehousing concept Familiarity with GCP services such as: BigQuery Cloud Storage Dataproc Composer Pub/Sub
Experience with Git and CI/CD practices Strong analytical and problem-solving skills
More at Relanto