Source description
About the role
Job Location: 100% REMOTE Contract type: Only for W2 Duration: 12 Months ROLE: Big Data Platform Engineer Top 3 requirements: • Python (Object Oriented Programming experience) • Pyspark • Scala • SQL • Airflow • Experience with big data. • Are any of them flexible? Required: • Strong verbal and written communication skills to effectively articulate messages to internal and external teams. • Hands on experience with Object Oriented programming with Python • Experience with Python, Pyspark, Scala and SQL • Experience with designing, building, optimizing, troubleshooting end-to-end big data pipelines using structured (relational and files) and semi-structured data • Experience with building metadata driven data processing frameworks • Strong experience in SQL, python, Pyspark, Scala and shell scripting. • Experience working with Airflow • Experience working with big data. • Ability to take ownership of a request from initial requirements, design, development and production deployment Nice to have: • Azure • Azure EventHubs • Apace Kafka • Streaming data • CosmosDB/NoSQL Database common data model Workflow - talk to business partners, gather requirements Use azure stack to complete this work current common data model is a dimensional data warehouse Data factory and functions - not necessarily needed Azure devops - can have something similar like github All other azure requirements needed Dimensional modeling SQL - coding Spark - questions on optimization Spark - coding Python – coding Shell scripting - if on resume, need to be ready to speak to Thanks & Regards, PremTarun Palnati|Technical Recruiter Thoughtwave Software and Solutions 314 N. Lake St, Suite 6, Aurora IL 60506 Desk: 6306182581 Email: www.tarun@wavsols.com Website: https://www.thoughtwavessoft.com/
More at Thoughtwave Software and Solutions