Data Engineer

Role Expectations: Architect and lead the development of complex, scalable data pipelines using technologies like Scala, Spark, Python, and Airflow. Set technical standards and best practices across data engineering teams. Oversee AWS-based data infrastructure (S3, Redshift, Glue, EMR), ensuring reliability and performance at scale. Lead real-time streaming solutions (Kafka, Kinesis, Snowplow), optimizing for low-latency and high throughput. Mentor and guide teams on advanced system design, data observability, and best practices. Drive DataOps initiatives, focusing on data quality, lineage, and governance. Collaborate with cross-functional teams to align technical architecture with business goals. Lead the integration of ML platforms like SageMaker for advanced analytics. Champion operational excellence through incident resolution and proactive monitoring. Qualifications: 8+ years of hands-on experience in data engineering, with expertise in Scala, Spark, Python, and Airflow. Mastery in cloud data infrastructure (AWS Redshift, Glue, Athena, S3). Expertise in real-time streaming technologies (Kafka, Kinesis, Snowplow). Proven ability in DataOps, focusing on data observability, quality, and governance. Strong leadership and mentoring skills with a track record of guiding technical teams. Deep experience with large-scale distributed systems and Java, Vertx, Spring Boot is a plus. Familiarity with machine learning platforms (e.g., SageMaker) is preferred.