Source description
About the role
Big Data Engineer – GCP & Scala Location: Bengaluru and Pune Experience: 5–10 Years Apply here: https://forms.cloud.microsoft/r/Vngu7esJju [HIDDEN TEXT] We are looking for an experienced Data Engineer with strong expertise in Google Cloud Platform (GCP) , Scala , and Apache Spark to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands-on experience building cloud-native data pipelines, processing large-scale datasets, and implementing modern data architectures using GCP services and big data technologies. Key Responsibilities Design, develop, and optimize scalable batch and real-time data pipelines using Scala and Apache Spark . Build and maintain ETL/ELT workflows for large-scale data processing. Develop cloud-native data solutions on Google Cloud Platform (GCP) . Work with Hadoop ecosystem technologies, including Hadoop , Hive , and HDFS . Develop and optimize SQL queries and database solutions using PostgreSQL and other relational databases. Implement workflow orchestration using Apache Airflow/Cloud Composer . Build CI/CD pipelines using Jenkins and follow DevOps best practices. Apply cloud design patterns and architectural best practices to deliver secure, scalable, and high-performing solutions. Collaborate with cross-functional teams including Data Architects, Business Analysts, Product Owners, and DevOps engineers. Monitor, troubleshoot, and optimize production data pipelines and cloud resources. Ensure data quality, governance, security, and performance standards are met. Required Skills Strong experience with Scala programming. Hands-on expertise in Apache Spark . Experience with Google Cloud Platform (GCP) . Strong knowledge of Hadoop , Hive , and HDFS . Experience with Apache Airflow . Hands-on experience with Jenkins . Strong SQL development and query optimization skills. Experience with PostgreSQL . Understanding of cloud architecture and cloud design patterns. Experience designing and implementing scalable ETL/data pipeline solutions.
More at Capco