Source description
About the role
Role & responsibilities Lead the design and implementation of enterprise-scale data platforms on Azure. Drive architecture, development, and optimization of data pipelines using Azure Databricks. Develop scalable ETL/ELT solutions using PySpark and Spark frameworks. Lead data ingestion, transformation, and processing for structured and unstructured data. Design Lakehouse architectures for enterprise reporting and analytics. Collaborate with business stakeholders, architects, and analytics teams to define data solutions. Ensure platform scalability, performance, security, and cost optimization. Establish Data Engineering best practices, coding standards, and CI/CD processes. Mentor and guide a team of data engineers and provide technical leadership. Support customer-facing discussions, solution design workshops, and architecture reviews.
More at Tata Consultancy Services