Lead Data Engineer- GenAI/LLM (Coimbatore)

10+ years of hands-on data engineering experience, including technical leadership of data platform initiatives - Strong, hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Delta Live Tables, Databricks Workflows, and Unity Catalog - Proven experience designing and implementing the medallion (bronze/silver/gold) architecture for data ingestion, curation, and consumption - Solid expertise in distributed data processing using Apache Spark (PySpark and Spark SQL) or equivalent big-data frameworks - Proven experience designing and building ETL/ELT pipelines for both batch and streaming data at scale - Expert-level proficiency in Python and SQL for data transformation, validation, and pipeline development - Strong experience building and managing data lakes and data warehouses, including dimensional and lakehouse data modeling - Working knowledge of Large Language Models (LLMs) and GenAI concepts prompts, embeddings, vector databases, and Retrieval-Augmented Generation (RAG) and the data pipelines required to support them - Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data services - Proven experience leading large-scale data migrations from on-premise (e.g. Hadoop) and cloud platforms to modern data architectures - Strong understanding of distributed data processing, partitioning, and performance optimization techniques - Experience implementing data security, governance, lineage, and access control (e.g. IAM, encryption, cataloging) - Strong understanding of object-oriented programming, software design patterns, and CI/CD practices - Familiarity with Agile/Scrum delivery methodologies and experience mentoring engineers - Excellent problem-solving, analytical, and stakeholder-management skills - Strong verbal and written communication skills Skills: pipelines,data,aws .