Senior/Lead Data Engineer

As a Senior Data Engineer or Lead Data Engineer at a leading MNC in Gurgaon, you will be responsible for designing and building scalable enterprise data platforms using Databricks, Spark, Delta Lake, and PySpark. Your key responsibilities will include developing and optimizing large-scale distributed data pipelines and processing applications, as well as performing Spark tuning, query optimization, and troubleshooting for performance improvements. Additionally, you will build and maintain enterprise-grade CI/CD pipelines using GitLab and automation tools, work on Databricks automation, REST APIs, and deployment workflows, and support advanced AI/ML initiatives including LLMOps, RAG pipelines, vector search, and model evaluation. You will also be implementing data governance, security, metadata management, and quality standards while mentoring junior engineers and driving engineering best practices across teams. Key Responsibilities: - Design and build scalable enterprise data platforms using Databricks, Spark, Delta Lake, and PySpark - Develop and optimize large-scale distributed data pipelines and processing applications - Perform Spark tuning, query optimization, and troubleshooting for performance improvements - Build and maintain enterprise-grade CI/CD pipelines using GitLab and automation tools - Work on Databricks automation, REST APIs, and deployment workflows - Support advanced AI/ML initiatives including LLMOps, RAG pipelines, vector search, and model evaluation - Implement data governance, security, metadata management, and quality standards - Mentor junior engineers and drive engineering best practices across teams Qualifications Required: - 5+ years of experience in Data Engineering - Proficiency in Python & PySpark - Strong knowledge of Apache Spark & Spark Performance Optimization - Experience with Databricks, Delta Lake & Distributed Data Processing - Familiarity with GitLab CI/CD Pipelines & Automation - Expertise in Databricks REST APIs & SQL Optimization - Understanding of Scalable Data Platform Architecture - Knowledge of Vector Search & Vector-Space Architectures - Exposure to AI/ML Workflows, LLMOps & RAG Pipelines - Ability to work on Data Governance, Metadata & Data Quality Please share the following details along with your resume: - Current CTC: - Expected CTC: - Experience: - Notice Period: As a Senior Data Engineer or Lead Data Engineer at a leading MNC in Gurgaon, you will be responsible for designing and building scalable enterprise data platforms using Databricks, Spark, Delta Lake, and PySpark. Your key responsibilities will include developing and optimizing large-scale distributed data pipelines and processing applications, as well as performing Spark tuning, query optimization, and troubleshooting for performance improvements. Additionally, you will build and maintain enterprise-grade CI/CD pipelines using GitLab and automation tools, work on Databricks automation, REST APIs, and deployment workflows, and support advanced AI/ML initiatives including LLMOps, RAG pipelines, vector search, and model evaluation. You will also be implementing data governance, security, metadata management, and quality standards while mentoring junior engineers and driving engineering best practices across teams. Key Responsibilities: - Design and build scalable enterprise data platforms using Databricks, Spark, Delta Lake, and PySpark - Develop and optimize large-scale distributed data pipelines and processing applications - Perform Spark tuning, query optimization, and troubleshooting for performance improvements - Build and maintain enterprise-grade CI/CD pipelines using GitLab and automation tools - Work on Databricks automation, REST APIs, and deployment workflows - Support advanced AI/ML initiatives including LLMOps, RAG pipelines, vector search, and model evaluation - Implement data governance, security, metadata management, and quality standards - Mentor junior engineers and drive engineering best practices across teams Qualifications Required: - 5+ years of experience in Data Engineering - Proficiency in Python & PySpark - Strong knowledge of Apache Spark & Spark Performance Optimization - Experience with Databricks, Delta Lake & Distributed Data Processing - Familiarity with GitLab CI/CD Pipelines & Automation - Expertise in Databricks REST APIs & SQL Optimization - Understanding of Scalable Data Platform Architecture - Knowledge of Vector Search & Vector-Space Architectures - Exposure to AI/ML Workflows, LLMOps & RAG Pipelines - Ability to work on Data Governance, Metadata & Data Quality Please share the following details along with your resume: - Current CTC: - Expected CTC: - Experience: - Notice Period: