Data Engineer

Hiring: AI Data Engineer (PySpark | Spark | SQL | ETL| Databricks | Kafka) Experience: 68 Years Location: Hyderabad / Bangalore Notice Period: Immediate Joiners Preferred Job Description We are looking for an experienced AI Data Engineer to join our Data & AI Engineering team. The ideal candidate should have strong expertise in PySpark, Apache Spark, ETL, SQL, Kafka, Python, and Databricks to build scalable data pipelines and support AI/ML and analytics solutions. Key Responsibilities Design, develop, and optimize scalable data pipelines for AI and analytics platforms. Develop and maintain PySpark applications and Spark-based data processing jobs. Build and enhance ETL/ELT pipelines for structured and unstructured data. Develop data ingestion, transformation, validation, and reconciliation workflows. Work extensively with Databricks for large-scale data engineering and AI workloads. Integrate batch and streaming data using Apache Kafka. Optimize Spark jobs for performance, scalability, and reliability. Collaborate with AI/ML teams to prepare and process data for machine learning models. Troubleshoot production issues and support enterprise data platforms. Follow coding standards, version control, and deployment best practices. Mandatory Skills Python PySpark Apache Spark ETL Development Advanced SQL Apache Kafka Databricks Good to Have AI/ML Data Pipelines Delta Lake Apache Airflow Snowflake Azure Data Factory (ADF) Docker Kubernetes Git Cloud Platforms (Azure, AWS, GCP, or OCI) Preferred Experience 6–8 years of experience in Data Engineering. Hands-on experience with Databricks, PySpark, and Apache Spark. Experience building AI/ML data pipelines and data engineering solutions. Experience with batch and real-time data processing. Knowledge of Data Lake and Data Warehouse architectures. Experience in Spark performance tuning and optimization. Experience working in Agile development environments. Shift: Candidates should be flexible to work in rotational shifts, including night shifts , based on business requirements. Location: Hyderabad / Bangalore Notice Period: Immediate Joiners Preferred Naukri Keywords: AI Data Engineer, Data Engineer, PySpark, Apache Spark, Python, Databricks, Apache Kafka, ETL, SQL, AI, Machine Learning, Data Engineering, Big Data Engineer, Spark Developer, Delta Lake, Airflow, Immediate Joiners. Email:dmadana@innominds.com