Source description
About the role
Job DescriptionWe are looking for a GCP Data Engineer with strong expertise in Python and PySpark to join our team. The ideal candidate will have hands-on experience in building scalable ETL/ELT pipelines, processing large datasets, and working with Google Cloud Platform (GCP) services. The candidate should possess excellent communication skills and be available to join immediately. Mandatory SkillsGoogle Cloud Platform (GCP)PythonPySparkETL/ELT Pipeline DevelopmentBigQueryGoogle Cloud Storage (GCS)SQLData Engineering conceptsKey ResponsibilitiesDesign, develop, and maintain scalable ETL/ELT data pipelines on GCP.Develop data processing solutions using Python and PySpark.Build and optimize data pipelines using BigQuery, GCS, and other GCP services.Write efficient SQL queries for data extraction, transformation, and analysis.Process structured and semi-structured data from multiple sources.Monitor, troubleshoot, and optimize data pipelines for performance and reliability.Collaborate with cross-functional teams to understand business requirements and deliver data solutions.Ensure data quality, security, and governance across data platforms.Support deployment, automation, and operational activities within GCP environments.Required Experience4+ years of experience in Data Engineering.Hands-on experience with Google Cloud Platform (GCP).Strong programming experience in Python and PySpark.Experience working with BigQuery, Google Cloud Storage (GCS), and SQL.Good understanding of data warehousing and ETL concepts.Experience in performance tuning and optimization of data pipelines.Preferred SkillsExperience with Dataproc, Airflow, Pub/Sub, or Cloud Composer.Knowledge of Spark SQL and DataFrames.Familiarity with CI/CD and version control tools.Experience working in Agile environments. .
More at Diverse Lynx India Private Limited