GCP Data Engineer (Whitefield)

Job Description GCP Data Engineer Location: Bangalore Experience: 48 Years About the Role We are looking for a skilled GCP Data Engineer to design, build, and optimize scalable data pipelines and data processing solutions on Google Cloud Platform. The ideal candidate should have strong expertise in GCP data services along with experience in one of the following technology combinations: - GCP + Scala - GCP + Apache Spark - Scala + Apache Kafka Key Responsibilities - Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform. - Build and optimize batch and real-time data processing solutions. - Develop data ingestion frameworks using GCP services and distributed processing technologies. - Implement high-performance data transformations and ensure data quality. - Collaborate with data scientists, analysts, and business stakeholders to deliver reliable data solutions. - Optimize data pipelines for scalability, performance, and cost efficiency. - Troubleshoot production issues and ensure high availability of data platforms. - Follow best practices for coding, testing, deployment, and documentation. Required SkillsOption 1: GCP + Scala - Strong experience with Google Cloud Platform (BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, Composer) - Hands-on experience in Scala - Strong SQL and ETL development experience - Experience with Git and CI/CD pipelines OROption 2: GCP + Apache Spark - Hands-on experience with Google Cloud Platform - Solid expertise in Apache Spark (PySpark/Scala Spark) - Experience with BigQuery, Dataflow, Dataproc, Cloud Storage - Strong SQL and data pipeline development experience OROption 3: Scala + Apache Kafka - Strong development experience in Scala - Hands-on expertise in Apache Kafka - Experience building streaming data pipelines - Knowledge of Spark is an added advantage - Experience working with cloud platforms (preferably GCP) is a plus Preferred Skills - Experience with Airflow/Cloud Composer. - Knowledge of Hive, Hadoop, or distributed data processing. - Familiarity with DevOps tools (Git, Jenkins, CI/CD). - Understanding of data warehousing concepts and dimensional modeling. - Experience with Agile/Scrum methodologies. Qualifications - Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. - 48 & 8-12 years of experience in Data Engineering. - Excellent analytical, problem-solving, and communication skills. Location: Bangalore Notice Period- Serving or Immediate Joiners Compensation: 2,400,000.00 - 4,000,000.00 per year Work Location: Hybrid remote in Whitefield, Bengaluru, Karnataka Job Description GCP Data Engineer Location: Bangalore Experience: 48 Years About the Role We are looking for a skilled GCP Data Engineer to design, build, and optimize scalable data pipelines and data processing solutions on Google Cloud Platform. The ideal candidate should have strong expertise in GCP data services along with experience in one of the following technology combinations: - GCP + Scala - GCP + Apache Spark - Scala + Apache Kafka Key Responsibilities - Design, develop, and maintain scalable ETL/ELT data pipelines on Google Cloud Platform. - Build and optimize batch and real-time data processing solutions. - Develop data ingestion frameworks using GCP services and distributed processing technologies. - Implement high-performance data transformations and ensure data quality. - Collaborate with data scientists, analysts, and business stakeholders to deliver reliable data solutions. - Optimize data pipelines for scalability, performance, and cost efficiency. - Troubleshoot production issues and ensure high availability of data platforms. - Follow best practices for coding, testing, deployment, and documentation. Required SkillsOption 1: GCP + Scala - Strong experience with Google Cloud Platform (BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, Composer) - Hands-on experience in Scala - Strong SQL and ETL development experience - Experience with Git and CI/CD pipelines OROption 2: GCP + Apache Spark - Hands-on experience with Google Cloud Platform - Solid expertise in Apache Spark (PySpark/Scala Spark) - Experience with BigQuery, Dataflow, Dataproc, Cloud Storage - Strong SQL and data pipeline development experience OROption 3: Scala + Apache Kafka - Strong development experience in Scala - Hands-on expertise in Apache Kafka - Experience building streaming data pipelines - Knowledge of Spark is an added advantage - Experience working with cloud platforms (preferably GCP) is a plus Preferred Skills - Experience with Airflow/Cloud Composer. - Knowledge of Hive, Hadoop, or distributed data processing. - Familiarity with DevOps tools (Git, Jenkins, CI/CD). - Understanding of data warehousing concepts and dimensional modeling. - Experience with Agile/Scrum methodologies. Qualifications - Bachelor's or Master's degree in Computer Science, Information Technology, or a relat