Data Engineer

Role: Data Engineer Location: Remote Duration: Contract to Hire (Preferred) but fine with contract for a someone with solid experience It is an exclusive requirement with interview times this week. Feedback on previous candidate whom we have to backfill – Missing on Scala / wasn’t able to learn it in 45 days We would still be OK to look for someone with Java because that transition from Java à Scala is much easier than Python à Scala. That said, it is HIGHLY PREFERRED that they have Scala out of the gate. If they have Databricks and Java, they will have Python though… Also lacked in engagement – wasn’t speaking up on calls, wasn’t being proactive Wasn’t getting tasks done as quickly as they wanted Project Scope / Components: This is part of the Optum Insight SOW where we are working to develop ingestion pipelines for EIP à Extract data and put it into the raw space – they are using Databricks as the ETL tool. Rather than doing drag and drop dev, they want to use Java & Python to create custom ingestion frameworks. Full tool set is Databricks w/Pyspark or Scala & Java. They will also exposing the analytical interface to EIP w/Scala. Development Methodology: Agile – 2 week sprints High-Level Individual Duties: Must-Haves (Concepts & Tools): • Databricks (NOT looking for Databricks & SQL [must be w/Scala or Java], or SQL Server / SSIS ETL experience) • Scala • Java would be fine • Azure is HIGHLY preferred but AWS would be ok • SQL Nice to have • Python / PySpark • ADF • Predictive Analytics is nice, but it’s a learning opportunity that they would work on in phase 3 • DeltaLake -- -- -- Thanks & Regards, Krishnasree | US IT Recruiter Thought wave software and solutions 314 N, Lake St, Suite 6, Aurora IL 60506 Mobile:6305668710 Email: krishnasree@thougtwavesoft.com Website: www.thoughtwavesoft.com LinkedIn: linkedin.com/in/krishnasree-kolla-7582a01a3