Source description
About the role
Job Summary We are seeking a Data Engineer with 3+ years of hands-on experience in Spark and SparkSQL , strong SQLexpertise , and solid batch processing experience . The role involves building and optimizing data pipelines to support analytics and business needs, ensuring data reliability and performance, and collaborating effectively with cross-functional teams. Must Have skills – Spark, SparkSQL, at least 3+ years working experience with spark. Very Strong SQL, Batch Exp is a must have. Good Project Experience. Python is a plus not a must have. Participate in code reviews, design discussions, and agile delivery processes. Required Skills & Experience: 3 + years of hands-on experience with Apache Spark (especially Spark SQL and batch processing). Strong SQL expertise — ability to write efficient, optimized, and complex SQL queries for large datasets. Solid understanding of data pipeline fundamentals, including ingestion, transformation, and loading (ETL/ELT) processes. Must have experience with batch data processing (scheduling, dependencies, failure recovery, performance tuning). Knowledge of data warehousing concepts and familiarity with relational databases. Experience working with structured and semi-structured data (CSV, Parquet, JSON, etc.). Strong problem-solving and debugging skills, particularly around pipeline performance and data quality.
More at FLEXTON INC