Data Platform Engineer (IC Role - SDE 2 to Staff) (Chennai)

This role is for one of the Weekday's clients Min Experience: 3 years Location: Bengaluru, Chennai, Hyderabad, Pune JobType; full time This role goes beyond simply maintaining pipelines. You will be responsible for designing and managing the foundational infrastructure upon which everything else is built. What Youll Do - Take full ownership of the data lakehouse, including its architecture, ingestion from CDC sources (Postgres, DynamoDB), scalability, and reliability - Develop and manage real-time stream processing frameworks for applications such as anomaly detection, customer 360 views, and live supply chain signalsensuring high throughput and low latency - Design and scale OLAP stores to support both real-time and batch processing for internal analytics and AI/ML pipelines - Create self-service ETL and query frameworks that enable data consumers to operate quickly without creating bottlenecks for the platform team - Implement cost observability measures that provide detailed insights into compute, storage, and query expenses by job, user, and sourceand then take action to reduce these costs - Build data movement APIs and reverse-ETL pipelines to efficiently deliver data to downstream consumers at scale - Establish robust job orchestration layers that remain stable under scale (experience with YARN, Airflow, EMR is a plus) Who You Are - Have 312 years of experience in data engineering, with at least 17 years focused on building or managing a data platform (beyond just pipelines) - Possess deep hands-on expertise with tools like Spark, Hudi/Delta Lake, Kafka, Airflow, Debezium, Presto/Trino, DBT, Airbyte - Are comfortable working with the AWS data ecosystem, including EMR, S3, Athena, Glue, and CloudWatch - Have managed daily processing of terabytes and billions of eventsscale is part of your daily experience - Have demonstrably reduced infrastructure costs and can provide metrics showing your impact - Are proficient in Java, Python, or Scalaideally experienced in all three - Preferably have experience as a pod lead or tech lead; youre the person others rely on when things break at 2 a.m. Bonus - Experience with OLAP engines such as Pinot, Druid, or ClickHouse - Have built or contributed to data movement or reverse-ETL APIs - Familiarity with feature stores (Feast, Feathr) or data catalog tools like Datahub What Makes This Different Our data platform powers AI that drives supply chain decisions for Fortune 500 companies. Youll directly witness the real business impact of your worknot just through dashboards. Join a small team with high ownership and the challenge of working at true scale. .