Senior Manager Data Engineering - Big Data

As a part of the People Data Platform team at Walmart, you will be responsible for owning and managing the centralized People Data Lake that enables trusted, cross-domain people insights across the enterprise. Your role will involve building and operating a scalable data platform on GCP, ingesting data from multiple source systems, and transforming it through a robust medallion architecture to deliver high-quality, analytics-ready data. Key Responsibilities: - Lead and grow a high-performing team of 8-10 data engineers building and operating the People Data Lake. - Own the architecture, design, and delivery of scalable batch and future streaming data pipelines using Spark, Big Query, Airflow, and related cloud-native technologies. - Drive best practices across data modeling, pipeline reliability, observability, and cost optimization to ensure high performance and availability. - Define and implement the data security framework for the lake, including encryption (at rest and in transit), masking, and fine-grained access controls. - Establish secure data sharing and governance mechanisms to protect sensitive data and ensure regulatory and enterprise compliance. - Partner with Product, HR, Analytics, Security, and Architecture teams to define the roadmap, prioritize initiatives, and deliver scalable, resilient, and secure data solutions. - Build a strong engineering culture focused on operational excellence, technical rigor, and continuous improvement while shaping the mid- to long-term platform vision. - Establish best engineering and operational excellence practices based on product, engineering, and scrum metrics. - Participate in hiring, mentoring, and building high-performing agile teams. - Establish clear expectations with individuals based on their level and role, meeting regularly to discuss performance, provide feedback and coaching. Qualifications Required: - 12+ years of experience in software or data engineering with 3+ years leading high-performing engineering teams. - Strong hands-on expertise in Spark, BigQuery, Airflow (or equivalent), and distributed data systems on GCP. - Experience with big data and distributed systems like Spark, Flink, etc. - Proven experience designing, building, and operating large-scale data lakes or lakehouse platforms with batch and streaming ingestion. - Deep understanding of data security principles including encryption, PII protection, masking, IAM, and row/column-level security. - Strong SQL and data modeling skills with experience implementing CI/CD, monitoring, and reliability best practices for data platforms. - Knowledge of cloud platforms like Azure, GCP etc. - Lead the integration of AI/ML toolkits and frameworks into day-to-day (D2D) data engineering workflows, enabling teams to accelerate development, automate data processes, and enhance decision-making capabilities. At Walmart Global Tech, you will be part of a team that values innovation and collaboration. You will have the opportunity to work in a dynamic environment where you can enhance your skills and make a significant impact in the retail industry. Join us in reimagining the future of retail and be a part of a culture that fosters inclusion and growth. Please note that this role is based in our [Bangalore/Chennai] office with flexibility for associates to manage their personal lives. Walmart, Inc. is an Equal Opportunity Employer By Choice, committed to creating a welcoming and inclusive workplace for all individuals. Minimum Qualifications: - Bachelor's degree in Computer Science and 5 years' experience in software engineering or related field. OR - 7 years' experience in software engineering or related field. OR - Master's degree in Computer Science and 3 years' experience in software engineering or related field. - 4 years' experience in data engineering, database engineering, business intelligence, or business analytics. - 1 year's supervisory experience. As a part of the People Data Platform team at Walmart, you will be responsible for owning and managing the centralized People Data Lake that enables trusted, cross-domain people insights across the enterprise. Your role will involve building and operating a scalable data platform on GCP, ingesting data from multiple source systems, and transforming it through a robust medallion architecture to deliver high-quality, analytics-ready data. Key Responsibilities: - Lead and grow a high-performing team of 8-10 data engineers building and operating the People Data Lake. - Own the architecture, design, and delivery of scalable batch and future streaming data pipelines using Spark, Big Query, Airflow, and related cloud-native technologies. - Drive best practices across data modeling, pipeline reliability, observability, and cost optimization to ensure high performance and availability. - Define and implement the data security framework for the lake, including encryption (at rest and in transit), masking, and fine-grained access controls. - Establ