Data Engineer/Lead - Big Data Ecosystem

Data Engineer Lead to design, build, and manage scalable data infrastructure and pipelines for enterprise-grade data platforms across cloud and big data ecosystems. The Ideal Candidate Will Lead a Team Of Data Engineers And Drive End-to-end Data Architecture, Ensuring High-performance, Secure, And Reliable Data Solutions That Support Analytics, AI/ML, And Business Intelligence Lead design and development of scalable data pipelines and data engineering solutions. Architect and implement end-to-end data platforms (batch and real-time processing). Build and optimize ETL/ELT workflows using modern data engineering tools. Develop robust data models for analytics, reporting, and machine learning use cases. Work with big data frameworks like Hadoop, Spark, and distributed computing systems. Design cloud-native data solutions on AWS / Azure / GCP. Implement data governance, data quality, and data security standards. Collaborate with Data Scientists, Analysts, and Business stakeholders. Manage streaming data pipelines using Kafka / real-time ingestion tools. Monitor, troubleshoot, and optimize data systems for performance and scalability. Lead and mentor a team of data engineers. Ensure documentation and adherence to best engineering practices. Exp (Years) 10 to 18 Years Required Skills Strong experience in Python, SQL, and distributed data systems. Expertise in Apache Spark, Hadoop, Kafka, Airflow, Databricks. Experience with Machine Learning data pipelines. Knowledge of DevOps tools (Docker, Kubernetes, CI/CD). Familiarity with BI tools (Power BI / Tableau). Education B.E / B.Tech / M.Tech / MCA / M.Sc (Computer Science / IT / Data Science / Statistics) (ref:hirist.tech)