Source description
About the role
Pyspark Developer
Hybrid | Charlotte, NC
Our client, a leader in their industry, has an excellent opportunity for a PySpark Developer to work on a 6-month contract position in Charlotte, NC, working hybrid onsite two days a week.
Responsibilities
-
• Design, develop, and maintain PySpark-based data transformation pipelines for diverse formats (JSON, CSV, RDBMS, stream) on Kubernetes/on-prem platforms.
-
• Optimize and tune PySpark jobs to efficiently handle medium to large-scale data volumes.
-
• Develop and implement data workflows leveraging Apache Airflow.
-
• Support and maintain PySpark applications, ensuring reliability and performance.
-
• Collaborate with cross-functional teams to ensure high-quality data processing and compliance.
Requirements
-
• 2+ years of experience in Python and PySpark development.
-
• 2+ years of experience with PySpark data transformation pipeline design and deployment on Kubernetes/on-prem platforms.
-
• 2+ years of experience in performance optimization for large-scale data processing.
-
• 2+ years of experience in designing and implementing Apache Airflow workflows.
-
• 2+ years of experience with Spark SQL and PostgreSQL for data storage and querying.
-
• Experience working in the Retail industry
More at 3B Staffing