Source description
About the role
We are looking for a highly skilled Senior Data Streaming Engineer with strong expertise in Apache Flink and Apache Spark to design, build, and optimize enterprise-grade real-time and batch data processing solutions. The ideal candidate will have extensive experience developing streaming applications, event-driven architectures, and high-performance ETL pipelines while ensuring scalability, reliability, and low-latency processing. Responsibilities Design, develop, and maintain scalable real-time data processing pipelines. Build streaming applications using Apache Flink DataStream API, Table API, and Spark Structured Streaming. Develop batch and near real-time ETL workflows. Implement stateful stream processing, event-time processing, watermarking, windowing, checkpointing, and fault-tolerant streaming solutions. Optimize Spark and Flink jobs for performance, scalability, and resource utilization. Process large-scale datasets with high throughput and low latency. Collaborate with architects, data engineers, and business teams to deliver enterprise data solutions. Implement monitoring, logging, alerting, and troubleshooting mechanisms. Ensure data quality, governance, and security best practices. Required Skills 8–10 years of Data Engineering experience. Strong hands-on expertise in Apache Flink: DataStream API Table API SQL API Stateful Stream Processing Event-Time Processing Windowing Watermarks Checkpointing Fault Tolerance Strong expertise in Apache Spark: Spark Core Spark SQL Structured Streaming Experience developing high-volume ETL pipelines. Strong understanding of distributed computing concepts. Experience with performance tuning and optimization. Good programming skills in Java, Scala, or Python. Preferred Skills Kafka or Azure Event Hubs Azure, AWS, or GCP Docker & Kubernetes CI/CD implementation Data Lakes Microservices Architecture Agile development Skills: etl,apache spark,flink,pipelines
More at zorba ai