Lead Data Platform Engineer (OpenSearch)

Position:Lead Data Platform Engineer (OpenSearch) Job Description:Role: Lead Data Platform Engineer (AWS | Kafka | OpenSearch) Preferred Location: Ahmedabad, Pune, Indore Years of Experience: 08 12 years Notice Period 0-30 Days (Preferred) Introduction Our team builds a data platform that reliably collects, processes, and retrieves large-scale IoT data based on AWS. We are seeking senior engineer who can design unobstructed flow even among billions of data records, push data storage performance to the limit, and technically lead service growth. Key Responsibilities High-availability data pipeline design: AWS MSK (Kafka), end-to-end data streaming architecture design and operation using LogstashStorage optimization and sharding strategies: Optimizing large-scale search performance through OpenSearch's index design, sharding, and lifecycle (ISM) managementData Modeling and Partitioning: Table partitioning and data modeling in PostgreSQL/MySQL, architecture guide for efficient queryingPerformance Tuning and Guidance: Leading SQL optimization across services and spreading technical best practices to backend development teamsRelease Management: Be the gatekeeper during key database / data pipeline release activities, create review checklist, review deployment details and validates details before deployment to ensure zero-error deployments without roll back needs.Infrastructure automation: DB schema versioning (such as Flyway) and deployment process automation systems Requirements Over 7 years of experience in data engineering or related backend developmentExtensive experience in operating infrastructure handling large-scale traffic in AWS cloud environmentsStrong capabilities in building real-time data pipelines and troubleshooting using KafkaExperience in OpenSearch (Elasticsearch) cluster design and sharding optimization is essential.Deep understanding of the internal structure of RDBMS (PostgreSQL/MySQL) and ability to tune complex queries Preferred Qualifications Seamless technical collaboration capabilities: Communication skills to discuss infrastructure optimization with DevOps engineers and provide efficient data modeling and query guides to backend developersGlobal collaboration skills: Able to share technical documents with engineers across the globe and conduct work via video conferences during collaborative work sessionsIoT Domain Experience: Experience in handling large-scale device messages and building data pipelinesEnsuring data stability: Experience in designing logic to prevent data loss during failures and ensure consistency verificationNice to have IaC Proficiency: Nice to have Experience coding and automated operations of data infrastructure using Terraform, Ansible, and others4 years of bachelors degree or higher in Computer Science or equivalent discipline What We Offer: Competitive SalaryWork Life BalanceProfessional DevelopmentInnovative Work Environment.Recognition and Rewards Location:IN-GJ-Ahmedabad, India-Ognaj (eInfochips) Time Type:Full time Job Category:Engineering Services .