DATA ENGINEER

Role: Data Engineer

Duration: 12 months Location: Hybrid 2-3 days in Dallas,TX or Bentonville, AK Visa: w2/1099 Rate: $68/hr

Tech stack requirements: • High level Databricks experience • Python, bash, or shell scripting • Experience architecting data pipelines and providing strategy/guidance on how to build from scratch • SQL queries and building dashboards and reports (Databricks visualization preferred but ok with Tableau or PowerBI)

Responsibilities for Data Engineer

• • Create and maintain optimal data pipeline architecture for data intensive applications.

• • Assemble large, complex data sets that meet functional / non-functional business requirements.

• • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure SQL, Cosmo DB, Databricks and other legacy databases.

• • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

• • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

• • Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.

• • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

• • Work with data and analytics experts to strive for greater functionality in our data systems.

Qualifications for Data Engineer

• • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

• • Extensive Experience on Databricks on Azure Cloud platform, deep understanding on Delta lake, Lake House Architecture.

• • Programming experience on Python, Shell scripting, PySpark, and other data programming language.

• • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

• • Strong analytic skills related to working with Data Visualization Dashboard, Metrics and etc.

• • Build processes supporting data transformation, data structures, metadata, dependency and workload management.

• • A successful history of manipulating, processing and extracting value from large disconnected datasets.

• • Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.

• • Familiar with Deployment tool like Docker and building CI/CD pipelines.

• • Experience supporting and working with cross-functional teams in a dynamic environment.

• • 8+ years’ experience in software development, Data engineering, and

• • Bachelor’s degree in computer science, Statistics, Informatics, Information Systems or another quantitative field. Postgraduate/master’s degree is preferred.

• • Experience in Machine Learning and Data Modeling is a plus.

• -- Thanks & Regards, Kalyan Sadanala| Talent acquisition specialist Thoughtwave Software and Solutions 314 N. Lake St, Suite 6, Aurora IL 60506 Desk: 6303132177, EXTN: 148 Text Message: 8722346381 Email: Kalyan@thoughtwavesoft.com Website: www.thoughtwavesoft.com Linked in: https://www.linkedin.com/in/kalyan-sadanala-21b678214/