DATABRICKS ENGINEER

ROLE:DATABRICKS ENGINEER

DURATION: 12 MONTHS CONTRACT LOCATION: 100% REMOTE

Requirements

Azure Databricks Azure Data Factory Azure DevOps is a strong plus Strong Python Data lake/data warehouse background Strong SQL (someone with a strong SQL background who switched over to Python for database engineering) Snowflake is a nice to have, not a requirement Excellent communication skills Someone that’s inquisitive, can ask questions, and can focus on the task at hand

Job Description: Databricks / Python Engineer We are looking for a savvy Databricks / Python Engineer to join our growing team of analytics experts. The person will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will develop and support a broad range of software capabilities including building data pipelines, managing the ETL/ELT processes, receiving, and delivering data through various interfaces, and processing significant amounts of data related to railcar movements, railcar liability, and financial calculations. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

Responsibilities

To perform this job successfully an individual must be able to perform the following essential duties satisfactorily. Other duties may be assigned to address business needs and changing business practices. Participate as a member of an Agile team developing Data Engineering solutions. Engage in requirements gathering and technical design discussions to meet business needs. Design and develop generic, scalable data pipelines in Azure Data factory and Databricks with python for on-prem and cloud data sources. Assemble large, complex sets of data that meet non-functional and functional business requirements. Leverage your curiosity for solving unstructured data problems and ability to manipulate and optimize large data sets to advance business problem-solving. Contribute to documentation, testing and cross-training of other team members. Work closely with others to assist and resolve production issues. -- Thanks & Regards, Pallavi Reddy| Technical Recruiter Thoughtwave Software and Solutions Desk: 630-228-9745 , EXTN:167 Email: pallavi@thoughtwavesoft.com Linked in: https://www.linkedin.com/in/pallavi-batchu-2756a8296/