Source description
About the role
The Data Engineer will architect and manage end-to-end ELT/ETL pipelines using Snowflake as the central cloud data warehouse. This individual will leverage advanced SQL for internal transformations, Python for scripting and workflow orchestration, and robust dimensional modeling techniques to build analytics-ready structures. Key Responsibilities Data Pipeline Development : Design and deploy scalable ETL/ELT pipelines to ingest data from structured, semi-structured (JSON, Parquet), and file sources into Snowflake. Snowflake Optimization : Manage Virtual Warehouses, configure multi-cluster environments, and apply optimization techniques like clustering keys, partitioning, and search optimization services. Data Modeling : Design logical and physical data schemas using Star, Snowflake, or Data Vault architectures to deliver efficient fact and dimension tables. Advanced Automation : Write robust Python scripts for automated workflow orchestration, API data extraction, complex object transformations, and automated error handling. Query Transformation : Develop, tune, and maintain highly optimized SQL queries, stored procedures, dynamic tables, and views
More at Tata Consultancy Services