Source description
About the role
You will be responsible for designing, developing, and optimizing scalable ETL/ELT data pipelines. Your role will involve building and maintaining cloud-native data solutions using Google Cloud Platform (BigQuery, Dataflow) and developing large-scale data processing applications using Apache Spark. Additionally, you will create and maintain data models, data warehouses, and data lakes to support business requirements. Your expertise in writing and optimizing complex SQL queries for data transformation, reporting, and analytics will be crucial. It will also be essential to implement data quality, governance, and monitoring processes and collaborate with cross-functional teams including Data Analysts, Data Scientists, and Business Stakeholders. Supporting batch and real-time data processing solutions using Kafka, Spark Streaming, or similar technologies, participating in code reviews, performance tuning, and best practice implementation, as well as contributing to CI/CD processes and automation of data workflows are key aspects of this role. Qualifications Required: - Hands-on experience with Google Cloud Platforms (GCP) specifically BigQuery and Dataflow - Strong experience in designing and implementing ETL/ELT pipelines - Extensive experience with Apache Spark for large-scale data processing - Advanced SQL skills including joins, window functions, CTEs, and query optimization About Senzcraft: Founded by IIM Bangalore and IEST Shibpur Alumni, Senzcraft is a hyper-automation company with a vision to Radically Simplify Today's Work and Design Business Process For The Future using intelligent process automation technologies. Senzcraft offers a suite of SaaS products and services, partnering with automation product companies. Awarded by Analytics India Magazine as a "Niche AI startup" and recognized by NY based SSON as a top hyper-automation solutions provider, Senzcraft is at the forefront of innovation in the AI industry. Please visit our website Senzcraft for more details. About the Opportunity: Senzcraft is seeking a highly skilled Senior Data Engineer with 5-8 years of experience to join their team in Bangalore (Hybrid). The ideal candidate will have expertise in Google Cloud Platform (GCP), Data Engineering, and Apache Spark to develop scalable data platforms and pipelines that support analytics, reporting, and business intelligence initiatives. Preferred Technical Skills: - Programming: Python, Java, or Scala - SQL: Advanced querying and performance tuning - Data Warehousing Concepts - Data Lakes & Lakehouse Architecture - Git and Version Control - CI/CD Pipelines Additional Details of the Company: Senzcraft's AI Operations SaaS platform MagicOps.ai has been making strides in the industry. You can also connect with Senzcraft on LinkedIn for more updates and insights. Note: The section on "Preferred Skills" has been omitted as it is not directly related to the Senior Data Engineer role. You will be responsible for designing, developing, and optimizing scalable ETL/ELT data pipelines. Your role will involve building and maintaining cloud-native data solutions using Google Cloud Platform (BigQuery, Dataflow) and developing large-scale data processing applications using Apache Spark. Additionally, you will create and maintain data models, data warehouses, and data lakes to support business requirements. Your expertise in writing and optimizing complex SQL queries for data transformation, reporting, and analytics will be crucial. It will also be essential to implement data quality, governance, and monitoring processes and collaborate with cross-functional teams including Data Analysts, Data Scientists, and Business Stakeholders. Supporting batch and real-time data processing solutions using Kafka, Spark Streaming, or similar technologies, participating in code reviews, performance tuning, and best practice implementation, as well as contributing to CI/CD processes and automation of data workflows are key aspects of this role. Qualifications Required: - Hands-on experience with Google Cloud Platforms (GCP) specifically BigQuery and Dataflow - Strong experience in designing and implementing ETL/ELT pipelines - Extensive experience with Apache Spark for large-scale data processing - Advanced SQL skills including joins, window functions, CTEs, and query optimization About Senzcraft: Founded by IIM Bangalore and IEST Shibpur Alumni, Senzcraft is a hyper-automation company with a vision to Radically Simplify Today's Work and Design Business Process For The Future using intelligent process automation technologies. Senzcraft offers a suite of SaaS products and services, partnering with automation product companies. Awarded by Analytics India Magazine as a "Niche AI startup" and recognized by NY based SSON as a top hyper-automation solutions provider, Senzcraft is at the forefront o
More at Senzcraft