Source description
About the role
Streaming Data Engineer (Kafka & Spark)
Location: Alpharetta, GA (willing to travel to client locations)
Employment Type: Full-Time (W2)
Role Overview
We are seeking a skilled Streaming Data Engineer to build real-time data pipelines using Kafka and Spark. This role focuses on designing low-latency event streaming solutions to enable rapid data processing and analytics.
Key Responsibilities
Develop real-time data pipelines using Kafka for event streaming and Spark Streaming or Structured Streaming for processing.
Design and manage Kafka topics, producers, and consumers to ensure low-latency data flows.
Implement scalable streaming architectures to handle high-velocity data with minimal latency.
Collaborate with data teams to integrate streaming data into analytics and machine learning workflows.
Optimize Spark jobs for performance and reliability in real-time processing environments.
Monitor and troubleshoot streaming pipelines to ensure data integrity and system availability.
Required Qualifications
Bachelor’s degree in Computer Science, Software Engineering, or a related field (or equivalent experience).
3 years of experience as a Data Engineer with a focus on Kafka and Spark Streaming or Structured Streaming.
Proficiency in building real-time and low-latency data pipelines using event streaming technologies.
Strong understanding of distributed systems and streaming data architectures.
Experience with Spark for processing and transforming streaming data in production.
Preferred Qualifications
-
Familiarity with cloud-based streaming services like AWS Kinesis or Azure Event Hubs.
-
Exposure to advanced Kafka configurations for fault tolerance and scalability.
-
Knowledge of monitoring tools like Confluent Control Center or Prometheus for streaming pipelines.
Compensation & Benefits
Competitive salary and comprehensive benefits package (healthcare, PTO, 401k).
Opportunities for professional growth and upskilling in AI and cloud technologies.
R2 Technologies Corporation is an equal opportunity employer and values diversity in the workplace.
More at R2 Technologies