Data Engineering Lead

Job Title: Data Engineering Lead Location: Mumbai Experience: 7+ Years Shift: General IST About the Role We are looking for a highly skilled Data Engineering Lead to design, build, and operate a scalable, cloud-native enterprise data platform on AWS. Data Engineering Lead This is a hands-on leadership role combining architecture design, data pipeline development, and team mentorship in an Agile environment. The role focuses on enabling secure and reliable ingestion of structured and unstructured data for analytics and downstream consumption on Databricks. Key Responsibilities A. Enterprise Data Platform Architecture Design and implement AWS-based enterprise data lake architecture Build scalable frameworks for all data types (structured, semi-structured, unstructured) Define standards for ingestion, transformation, storage, and access Ensure seamless integration with analytics workloads B. Data Ingestion Integration Develop real-time and batch ingestion pipelines Integrate internal systems (CRM, ERP, KYC, OMS, PMS) Integrate external data sources (market data, research feeds, documents) Utilize AWS services like AppFlow, Lambda, Glue, S3, and Athena C. Real-Time Data Processing Build event-driven pipelines for near real-time analytics Support use cases such as: Trading operational analytics Compliance and surveillance monitoring D. Security, Governance Compliance Implement security using AWS KMS, Secrets Manager, and Security Hub Use AWS Config and CloudTrail for governance and audit Ensure strong data protection, encryption, and access control E. Monitoring Observability Implement monitoring using CloudWatch and Grafana Track pipeline performance, failures, data freshness, and system health F. DevOps Automation Build and manage CI/CD pipelines using GitLab Automate testing and deployment processes Maintain code quality and version control standards G. Data Quality Metadata Implement validation, reconciliation, and monitoring frameworks Manage metadata and lineage using AWS Glue Data Catalog H. Technical Leadership Collaboration Lead architecture discussions and troubleshooting Mentor team members through code reviews and guidance Collaborate with analytics teams and business stakeholders Skills Competencies Technical Skills Programming: Python, PySpark, SQL Cloud: AWS (S3, Glue, Athena, Lambda, AppFlow) Analytics Platform: Databricks Monitoring: CloudWatch, Grafana DevOps: GitLab Behavioral Competencies Strong analytical and problem-solving skills Ownership mindset with delivery focus Ability to lead in fast-paced Agile environments Experience Required 7 10 years in Data Engineering / Platform Engineering Proven experience in building cloud-native data platforms Hands-on experience with batch and real-time data processing Experience in financial services / wealth management preferred Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.