Source description
About the role
We are looking for a highly skilled Senior Data Streaming Engineer with strong expertise in Apache Flink and Apache Spark to design, build, and optimize enterprise-grade real-time and batch data processing solutions. The ideal candidate will have extensive experience developing streaming applications, event-driven architectures, and high-performance ETL pipelines while ensuring scalability, reliability, and low-latency processing. Responsibilities Design, develop, and maintain scalable real-time data processing pipelines.Build streaming applications using Apache Flink DataStream API, Table API, and Spark Structured Streaming.Develop batch and near real-time ETL workflows.Implement stateful stream processing, event-time processing, watermarking, windowing, checkpointing, and fault-tolerant streaming solutions.Optimize Spark and Flink jobs for performance, scalability, and resource utilization.Process large-scale datasets with high throughput and low latency.Collaborate with architects, data engineers, and business teams to deliver enterprise data solutions.Implement monitoring, logging, alerting, and troubleshooting mechanisms.Ensure data quality, governance, and security best practices. Required Skills 810 years of Data Engineering experience.Strong hands-on expertise in Apache Flink:DataStream APITable APISQL APIStateful Stream ProcessingEvent-Time ProcessingWindowingWatermarksCheckpointingFault ToleranceStrong expertise in Apache Spark:Spark CoreSpark SQLStructured StreamingExperience developing high-volume ETL pipelines.Strong understanding of distributed computing concepts.Experience with performance tuning and optimization.Good programming skills in Java, Scala, or Python. Preferred Skills Kafka or Azure Event HubsAzure, AWS, or GCPDocker & KubernetesCI/CD implementationData LakesMicroservices ArchitectureAgile development Skills: etl,apache spark,flink,pipelines .
More at Zorba AI