Source description
About the role
Role: Data Engineer (jr-mid)
Location: Hybrid 3/days on site (NYC, Hartford CT, Wellesley MA) Duration: 6+ month contract to hire
Must-Haves (Concepts & Tools): Python SQL Creating & tuning queries Hadoop / Hive / Spark – migrating these legacy environments to GCP ETL experience Any cloud experience (GCP preferred) DevOps / CI/CD Testing skills – there is no QA team that is borrowed
Nice-to-Haves: Building ingestion pipelines for ML Apps GCP OS Scripting
Job Details Develop large scale data structures and pipelines to organize, collect and standardize data that helps generate insights and addresses reporting needs. Write ETL (Extract / Transform / Load) processes, design database systems and develop tools for real-time and offline analytic processing. Collaborate with data science and IT teams to transform data and integrate algorithms and models into automated processes. Use knowledge and experience in various architectures including on-prem (such as Hadoop, Hive, Spark, HDFS commands) and cloud (GCP, Azure, AWS) to design and optimize data pipelines. Use strong programming skills such as Python, Java or any of the major programming languages to build robust data pipelines and dynamic systems. Build data marts and data models to support Data Science and other internal customers. Integrate data from a variety of sources, ensuring that they adhere to CVS Health data quality and accessibility standards. Analyze current information technology environments to identify and assess critical capabilities and recommend solutions. Experiment with available tools and advise on new tools in order to determine optimal solution given the requirements dictated by the model/use case. Leverage practices such as Agile, DevOps, DevSecOps, MLOps, etc.
Required Qualifications • 3+ years of relevant work experience in Data Engineering • 3+ years experience with SQL • 3+ years experience with Python • Experience with Hadoop, Hive or PySpark • Experience with Cloud-based platforms (GCP preferred, AWS or Azure) • Strong Communication, Critical Thinking & Problem-Solving skills
Preferred Qualifications
-
• Experience with Data Warehousing Concepts • Experience with Shell Scription (Bash) • Health Care industry knowledge is a plus
-
Bhargavi Ankam IT Recruiter Staffing | Thoughtwave Software and Solutions Limited 6306891366 bhargavi@thoughtwavesoft.com www.thoughtwavesoft.com 314 N. Lake St, Suite 6, Aurora IL 60506
More at Thoughtwave Software and Solutions