Python Developer - Data Engineer

As a Senior Data Engineer, you will be responsible for designing, building, and operating reliable, scalable, and cost-efficient data pipelines with a focus on production-grade data systems using Python and modern orchestration tools. Your collaboration with Data Science and analytics teams will be crucial in enabling high-quality, well-governed, and discoverable data across the organization. Key Responsibilities: - Design, build, and maintain robust, scalable data pipelines using Python. - Develop and manage orchestration workflows using Apache Airflow / Google Cloud Composer. - Ensure high availability, fault tolerance, and performance of data workflows. - Implement monitoring, alerting, and observability best practices. - Own scheduling, monitoring, and incident management for data pipelines. - Ensure SLAs and data freshness requirements are met. - Implement proactive quality checks and validation frameworks. - Optimize pipelines for performance and cost-efficiency in cloud environments. - Define and enforce data contracts across producers and consumers. - Contribute to and maintain a data catalog. - Partner closely with Data Science and Analytics teams for schema alignment, data discovery, and pipeline optimization. - Support reproducibility and transparency of datasets used in modeling. - Work within Google Cloud Platform (GCP) environments. - Design cost-aware architectures and storage strategies. - Improve CI/CD, version control, and deployment workflows for data systems. Qualifications Required: - Strong proficiency in Python with ML Deployment. - Hands-on experience with Apache Airflow and/or Google Cloud Composer. - Experience building and maintaining production data pipelines. - Strong understanding of Workflow orchestration (Airflow). - Monitoring & alerting. - Data quality frameworks. - Scheduling & dependency management. - Experience with cloud platforms (preferably GCP). - Solid understanding of data modeling and pipeline design principles. - Experience implementing or working with data contracts. - Exposure to data catalogs and metadata management. - Strong operational mindset with a focus on reliability and maintainability. As a Senior Data Engineer, you will be responsible for designing, building, and operating reliable, scalable, and cost-efficient data pipelines with a focus on production-grade data systems using Python and modern orchestration tools. Your collaboration with Data Science and analytics teams will be crucial in enabling high-quality, well-governed, and discoverable data across the organization. Key Responsibilities: - Design, build, and maintain robust, scalable data pipelines using Python. - Develop and manage orchestration workflows using Apache Airflow / Google Cloud Composer. - Ensure high availability, fault tolerance, and performance of data workflows. - Implement monitoring, alerting, and observability best practices. - Own scheduling, monitoring, and incident management for data pipelines. - Ensure SLAs and data freshness requirements are met. - Implement proactive quality checks and validation frameworks. - Optimize pipelines for performance and cost-efficiency in cloud environments. - Define and enforce data contracts across producers and consumers. - Contribute to and maintain a data catalog. - Partner closely with Data Science and Analytics teams for schema alignment, data discovery, and pipeline optimization. - Support reproducibility and transparency of datasets used in modeling. - Work within Google Cloud Platform (GCP) environments. - Design cost-aware architectures and storage strategies. - Improve CI/CD, version control, and deployment workflows for data systems. Qualifications Required: - Strong proficiency in Python with ML Deployment. - Hands-on experience with Apache Airflow and/or Google Cloud Composer. - Experience building and maintaining production data pipelines. - Strong understanding of Workflow orchestration (Airflow). - Monitoring & alerting. - Data quality frameworks. - Scheduling & dependency management. - Experience with cloud platforms (preferably GCP). - Solid understanding of data modeling and pipeline design principles. - Experience implementing or working with data contracts. - Exposure to data catalogs and metadata management. - Strong operational mindset with a focus on reliability and maintainability.