Data Engineer

We are seeking a highly skilled Data Engineer with strong hands-on experience in Python, PySpark, Apache Spark, Kafka, and Databricks to design, build, and maintain scalable and high-performance data pipelines. The role focuses on developing robust batch and streaming data solutions for large-scale distributed data environments. Key Responsibilities Design, develop, and maintain batch and real-time data pipelines using PySpark and Apache Spark Build and manage scalable data workflows and jobs on Databricks Develop and optimize Python and SQL code to process large and complex datasets Implement Kafka-based streaming solutions for real-time data ingestion and processing Perform Spark job tuning, performance optimization, and resource management Ensure data reliability, scalability, and efficiency across pipelines Work closely with analytics, product, and engineering teams to support data requirements Monitor, troubleshoot, and resolve data pipeline and performance issues Maintain documentation for data pipelines, workflows, and processing logic Required Experience & Qualifications 5–10 years of experience in Data Engineering roles Strong hands-on coding experience in Python and PySpark Extensive experience with Apache Spark and Databricks platforms Hands-on experience with Kafka for streaming data processing Strong SQL skills for data transformation and validation Experience working with large-scale, distributed data systems Preferred Skills Experience with Delta Lake for data versioning and reliability Exposure to cloud platforms such as AWS, Azure, or GCP Experience with workflow orchestration tools such as Airflow