Source description
About the role
Pune, Maharashtra Job Summary Job Description: Snowflake Engineer (8+ Years Experience) Role Overview We are seeking a highly skilled Snowflake Engineer with 8+ years of overall IT experience, including 3+ years of strong hands-on experience in Snowflake-based data engineering. The ideal candidate brings deep expertise in modern data pipeline engineering, metadata-driven ingestion, and AI-enabled data processing, along with strong consultative and stakeholder engagement skills. Key Responsibilities Key Responsibilities 1. Data Pipeline Engineering Design and develop scalable, high-performance data pipelines using SnowSQL, Snowpark, and Snowflake native capabilities Build and optimize ELT/ETL frameworks supporting both batch and real-time workloads Implement robust ingestion pipelines from diverse enterprise systems 2. Metadata-Driven Ingestion Design and implement metadata-driven ingestion frameworks for scalable onboarding of datasets Handle ingestion across multiple formats including XML, CSV, JSON, and Parquet Automate ingestion, validation, and transformation processes 3. Streaming & Kafka Integration Develop real-time data ingestion pipelines using Kafka Integrate streaming data into Snowflake ensuring reliability and scalability Implement monitoring and failure handling for streaming workflows 4. AI/GenAI Integration (Cortex AI) Leverage Snowflake Cortex AI capabilities to enable AI-driven data transformations, enrichment, and insights generation Collaborate with business and analytics teams to enable AI-powered data consumption use cases Contribute to building intelligent, automated data pipelines and semantic layers 5. Data Modeling & Transformation Develop scalable data models supporting analytics and reporting layers Build transformations using Snowpark, SQL, and Snowflake native features Optimize query performance and data access patterns 6. Data Delivery & Reporting Enablement Deliver analytics-ready datasets in predefined formats for reporting tools (Power BI, Tableau, etc.) Work closely with business stakeholders to ensure data usability and alignment with reporting requirements 7. Performance Optimization & Governance Implement best practices for performance tuning, clustering, and cost optimization Ensure data quality, governance, security, and compliance standards Monitor, troubleshoot, and continuously improve pipelines 8. Stakeholder Engagement & Technical Consulting Engage with stakeholders to understand business needs and translate them into data solutions Provide technical consultation on data architecture, ingestion strategies, and Snowflake capabilities Act as a trusted advisor in data engineering and platform decisions Skill Requirements Required Skills & Experience 8+ years of overall IT experience with 3+ years in Snowflake engineering Strong expertise in: Snowflake (SnowSQL, Snowpark, Snowpipe, Streams/Tasks) Pipeline design and data engineering frameworks Metadata-driven ingestion architectures Hands-on experience with: Kafka-based ingestion (mandatory) File formats: Parquet (must), CSV, XML, JSON Experience delivering data in predefined formats for reporting and analytics Strong proficiency in SQL and data modeling Hands-on experience or exposure to Snowflake Cortex AI (required) Excellent communication, presentation, and stakeholder management skills Strong technical consulting and problem-solving mindset Good-to-Have Skills Experience with DBT (Data Build Tool) Knowledge of PL/SQL or procedural SQL programming Exposure to cloud platforms (AWS/Azure/GCP) Familiarity with CI/CD pipelines and version control systems Skill Matrix Must-Have Skills Skill Area Expected Competency Relevance to Role Snowflake Engineering Strong hands-on experience with Snowflake, SnowSQL, Snowpark, Snowpipe, Streams, and Tasks. Core capability required to design, build, and optimize Snowflake-based data engineering solutions. Data Pipeline Development Ability to design scalable batch and real-time ELT/ETL pipelines using Snowflake native capabilities. Required for building enterprise-grade ingestion and transformation frameworks. Metadata-Driven Ingestion Experience designing metadata-driven ingestion frameworks for onboarding multiple datasets and source systems. Critical for scalable, reusable, and automated data onboarding. Kafka Integration Hands-on experience with Kafka-based data ingestion and streaming pipeline integration. Mandatory for real-time data ingestion and event-driven data processing. File Format Handling Strong experience working with Parquet, CSV, JSON, and XML formats. Required to support structured, semi-structured, and enterprise file-based ingestion scenarios. SQL and Data Modeling Strong SQL proficiency with experience in data modeling, transformations, query optimization, and .
More at HCLTech