Source description
About the role
Role: Data Engineer
Duration: Contract (4-6 months) Location: Minneapolis, MN /100% Remote
W2 Only 100% REMOTE
Project Scope / Business Need: moving data on prem into GCP based platform. Big data is currently stored on prem and is pushed over to GCP Duties: Building data platform for cloud migration to GCP. Will be handling building spark service for other teams to handle pushing data in batches, will eventually move to a kafka streaming based platform after migration is happening.
Top Requirements: 5+ years • Spark - Batch layer for Kafka streaming, will be focused, will push batch jobs to airflow • Scala - Will be chief • Python - will be building utility programs • Kubernetes - creating/supporting containers, won’t be deploying them. • ETL - receiving/building pipelines, needs to have foundational knowledge.
Nice to Have: • Cloud - GCP/AWS • Financial/Bank experience • Presto - Will be pushing SQL across multiple DB's, being access to push batch data across those will be a plus • Kafka - will eventually push over to streaming, right now everything is in batch
Thanks & Regards,
Mohan Kumar B | Technical Recruiter | Wave Solutions Inc. | 1444 N. Farnsworth Ave, Suite 302, Aurora, IL - 60505 | Email: kumar@wavsols.com Direct #: 630-566-9398| Fax: 630-566-9398 Main #:6305669398, EXT: 108 Website: www.wavsols.com LinkedIn: https://www.linkedin.com/in/mohan-kumar-bangaru-b424467b/
More at Thoughtwave Software and Solutions