Senior Data Engineer

Key Responsibilities The Senior Data Engineer will be the technical lead for major initiatives, focusing on architecture, optimization, and setting best practices. Data Architecture & Design: Design and architect scalable, high-performance ETL/ELT pipelines in GCP using Cloud Composer and PySpark on Dataproc. Query & Cost Optimization: Lead the optimization of BigQuery performance—utilizing Partitioning, Clustering, Materialized Views, and slot management to drastically reduce latency and GCP costs. Lakehouse Governance: Implement and manage BigLake to bridge the gap between data lakes and warehouses, ensuring unified governance and security for structured and unstructured data. Advanced Automation & Infrastructure: Drive automation efforts from data ingestion to infrastructure deployment using Terraform (IaC) and CircleCI, ensuring modular, reproducible, and self-healing systems. Data Quality & Operations Leadership: Define and implement robust "Self-Healing" pipelines with sophisticated logging, alerting, and data quality checks to ensure the warehouse remains the ultimate "Source of Truth." Business Problem Solving: Collaborate closely with cross-functional teams and business stakeholders to translate complex requirements into high-performance data architectures and actionable solutions. Mentorship & Best Practices: Mentor mid-level engineers, review their designs, and champion best-in-class coding and architectural standards. Continuous Evolution: Stay ahead of the curve by proactively championing and implementing new GCP methods and tools to solve emerging data challenges. Skills & Competencies Experience: Senior Level - 4 - 6+ years of hands-on Data Engineering experience. GCP Proficiency: Deep knowledge of GCP services with a focus on BigQuery, DataForm, Cloud Composer, Cloud Storage, Cloud functions and Dataproc. Transformation Tools: Proficiency in Dataform for managing version-controlled SQL workflows and modeling within BigQuery. Strong Coding: Excellent Python and SQL skills. You write modular code that is easy to test and maintain. Architectural Knowledge: Solid understanding of Data Modeling (Star Schema, Snowflake) and Data Operations. Automation Mindset: Expert experience with CI/CD tools and Infrastructure-as-Code is essential. Domain Knowledge: Experience in the News or Media industry is a strong plus.