Source description
About the role
Role: Sr. Data Engineer w/ Pyspark Location: 100% Remote Length: 12+ months Required: • Strong verbal and written communication skills to effectively articulate messages to internal and external teams. • Hands on experience with Azure data platform stack (Azure Databricks, Azure Data Factory, Azure Data Lake storage, Azure DevOps, Azure Functions) • Experience with dimensional modeling and data warehousing concepts • Experience with designing, building, optimizing, troubleshooting end-to-end big data pipelines using structured (relational and files) and semi-structured data • Experience with building metadata driven data processing frameworks • Strong experience in Spark, SQL, python, Pyspark, shell scripting. • Ability to work in an agile team. • Ability to take ownership of a request from initial requirements, design, development and production deployment Nice to have: • Powershell, scala, java • Apache Airflow • Azure EventHubs • Apace Kafka • Streaming data Thanks & Regards, Krishnasree | US IT Recruiter Thought wave software and solutions 314 N, Lake St, Suite 6, Aurora IL 60506 Mobile:6305668710 Email: krishnasree@thougtwavesoft.com Website: www.thoughtwavesoft.com LinkedIn: linkedin.com/in/krishnasree-kolla-7582a01a3
More at Thoughtwave Software and Solutions