GCP Kafka Integration Engineer

About the Role We are replacing our Legacy platform with a modern, event-driven architecture on Google Cloud, built around Google Cloud Managed Service for Apache Kafka. You will be the hands-on engineer who designs and builds this platform from the ground up: event ingestion from LEGACY SYSTEM ECC and external carrier systems, a Kafka-based streaming backbone processing ~10 million events per day, a rules/processing engine, and a compliant 7-year event archive on BigQuery and Cloud Storage. This is a greenfield build with high ownership. You will work directly with a senior Legacy system functional expert who owns all Legacy knowledge. You bring the cloud, Kafka, and engineering depth; we bring the business and legacy system depth. What You Will Build A managed Kafka cluster (topics, partitions, security, retention) on Google Cloud An HTTPS event-ingestion bridge (Cloud Run, Python) receiving events from legacy system and external carriers A Kafka consumer "rule engine" (Python) that processes events, maintains state in Firestore, and triggers actions back to Legacy System Kafka Connect pipelines archiving every event to BigQuery (queryable) and Cloud Storage (7-year locked/WORM archive) Monitoring, alerting, dead-letter handling, and reconciliation for a zero-data-loss pipeline Must-Have Skills & Experience 5+ years in software / data engineering (backend or data platform) 2+ years production experience with Apache Kafka (any distribution: open-source, Confluent, AWS MSK, Aiven, etc.) topics, partitions, keys, consumer groups, offsets, delivery semantics 2+ years hands-on Google Cloud Platform: Cloud Run (or GKE), IAM & service accounts, VPC basics, Cloud Monitoring Strong Python (production code, not just scripts): REST APIs, error handling, testing BigQuery: SQL, partitioned/clustered tables Experience building reliable integrations: idempotency, retries, dead-letter queues, at-least-once processing Comfortable owning a system end-to-end: design, build, deploy, monitor, document Nice-to-Have (not required) Google Cloud Managed Service for Apache Kafka, or Kafka Connect sink connectors Firestore / Bigtable or another NoSQL store Terraform / infrastructure-as-code Experience with regulated/compliance data retention (WORM, audit trails) Work Location & Hours (please read carefully before applying) 100% IN-OFFICE Fixed shift: 6:30 PM – 3:30 AM IST (aligned to US business hours), Monday–Friday A night-shift allowance is provided