Source description
About the role
Role: Data Engineer
Location: San Francisco Bay Area, CA Duration: 6+ months
Project Scope: Data engineer Mid to sr level Min 5 years Multi product offering project, confidential bank products Build data pipelines to get info from systems Mix of real time and batch
Team Size & Breakdown: Already built team, 16-17 engineers Development Methodology: agile High-Level Individual Duties: Data pipeline work Python and spark pipelines Automating testing of pipelines Using kafka for real time can train, but a sr resource should have this Both batch and real time streaming depending on need, ok to train up a mid person on real time
Must-Haves: Python SQL Airflow Batch jobs Spark – very nice to have Kafka – very nice to have
Nice to Have: AWS – can train up here
--
Thanks & Regards, Ujwala Pinninti Thoughtwave Software and Solutions 314 N. Lake St, Suite 6, Aurora IL 60506 Desk: 6302168656, EXTN: 156 Email: ujwala@thoughtwavesoft.com Website: www.thoughtwavesoft.com Linked in: https://www.linkedin.com/in/ujwala-pinninti-675285213/
More at Thoughtwave Software and Solutions