Senior Data Engineer

As a Senior Data Engineer, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure. Your expertise in AWS, Snowflake, Terraform, SQL, Python, and PySpark will be crucial in collaborating with the Data tech lead/lead data engineer to manage data workflows, ensure data reliability, and implement best practices for data governance and observability. This role directly impacts data-driven products, personalization, reporting, data science, machine learning, and overall business success. Key Responsibilities: - Develop reusable custom frameworks using cloud technologies like AWS, Snowflake, and Managed Airflow. - Design and develop scalable ETL/ELT pipelines using Python, SQL, and PySpark. - Implement infrastructure-as-code (IaC) using Terraform for cloud-based data environments. - Develop and maintain data models, transformations, and orchestration workflows. - Ensure data quality, observability, and lineage tracking across the ecosystem. - Optimize query performance, storage costs, and compute resources in Snowflake and AWS. - Implement CI/CD pipelines for data infrastructure automation. - Monitor and troubleshoot data pipelines, jobs, and cloud infrastructure to maintain SLAs. Required Skills & Qualifications: - Strong collaboration and communication skills - Strong proficiency in SQL and Python for data processing and transformation. - Hands-on experience with AWS (S3, Glue, Lambda, Redshift, etc.). - Expertise in Snowflake (performance tuning, Snowflake SQL, schema design). - Expertise with Terraform for infrastructure automation. - Proficiency in Airflow or other orchestration tools. - Understanding of data observability, monitoring, and governance best practices. - Experience with version control (Git) and CI/CD for data pipelines. - Strong problem-solving skills and ability to work independently in a fast-paced environment. - Experience with any code base ETL/ELT tools. Good to Have: - Experience in implementing Datamesh and distributed data ownership. - Exposure to Docker, Kafka, and Kinesis. - Knowledge of data security and compliance frameworks (GDPR, SOC2, etc.). - Experience in cost optimization and performance tuning in cloud-based data architectures. - Experience in PySpark. As a Senior Data Engineer, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure. Your expertise in AWS, Snowflake, Terraform, SQL, Python, and PySpark will be crucial in collaborating with the Data tech lead/lead data engineer to manage data workflows, ensure data reliability, and implement best practices for data governance and observability. This role directly impacts data-driven products, personalization, reporting, data science, machine learning, and overall business success. Key Responsibilities: - Develop reusable custom frameworks using cloud technologies like AWS, Snowflake, and Managed Airflow. - Design and develop scalable ETL/ELT pipelines using Python, SQL, and PySpark. - Implement infrastructure-as-code (IaC) using Terraform for cloud-based data environments. - Develop and maintain data models, transformations, and orchestration workflows. - Ensure data quality, observability, and lineage tracking across the ecosystem. - Optimize query performance, storage costs, and compute resources in Snowflake and AWS. - Implement CI/CD pipelines for data infrastructure automation. - Monitor and troubleshoot data pipelines, jobs, and cloud infrastructure to maintain SLAs. Required Skills & Qualifications: - Strong collaboration and communication skills - Strong proficiency in SQL and Python for data processing and transformation. - Hands-on experience with AWS (S3, Glue, Lambda, Redshift, etc.). - Expertise in Snowflake (performance tuning, Snowflake SQL, schema design). - Expertise with Terraform for infrastructure automation. - Proficiency in Airflow or other orchestration tools. - Understanding of data observability, monitoring, and governance best practices. - Experience with version control (Git) and CI/CD for data pipelines. - Strong problem-solving skills and ability to work independently in a fast-paced environment. - Experience with any code base ETL/ELT tools. Good to Have: - Experience in implementing Datamesh and distributed data ownership. - Exposure to Docker, Kafka, and Kinesis. - Knowledge of data security and compliance frameworks (GDPR, SOC2, etc.). - Experience in cost optimization and performance tuning in cloud-based data architectures. - Experience in PySpark.