Source description
About the role
Job Title Data Engineer PySpark | Apache Iceberg | AWS EMR | Airflow | Kafka Job Description We are hiring experienced Data Engineers to join our team in an L3 Support role. The ideal candidate should have strong expertise in building and supporting scalable data pipelines using modern Big Data and Cloud technologies. Roles & Responsibilities Develop, maintain, and optimize ETL/data processing pipelines using PySpark and Python . Build and manage data workflows using Apache Airflow . Work with Apache Iceberg for table management, schema evolution, partitioning, bucketing, and time travel. Execute and monitor Spark jobs on AWS EMR . Support Kafka-based CDC (Change Data Capture) data ingestion pipelines. Write optimized SQL queries and implement data transformations using the Medallion Architecture (Bronze, Silver, Gold) . Troubleshoot production issues and provide L3 application support. Collaborate with cross-functional teams to ensure data quality, performance, and reliability. Follow Git-based version control and CI/CD deployment best practices. Mandatory Skills 6-10 years of experience as a Data Engineer Strong hands-on experience with PySpark and Python Expertise in Apache Iceberg Experience with Apache Airflow Hands-on experience with AWS EMR Strong SQL skills Experience with Kafka and CDC concepts Knowledge of Git and CI/CD Good to Have Experience with Data Lake/Lakehouse architecture Knowledge of performance tuning and Spark optimization Exposure to Microservices Experience supporting production environments Job Details Experience: 6-10 Years Employment Type: Permanent Work Location: Bangalore, Pune, Chennai, Gurgram & Kolkata (Hyderabad not available) Notice Period: Immediate Joiners Preferred
More at Technogen