Data Engineer

Role- Data Engineer Duration- 6 Months Location- hybrid on site 2-3 days/week in Sunnyvale, CA. Project Scope: Gather all the backend logs, aggregate data, and run batch processing. These aggregated logs provide more insights to backend systems – shows any weakness within the services to allow for improvement which is a huge business value-add for revenue loss. Development Methodology: Agile – 2 week sprints High-Level Individual Duties:  There are some new builds of pipelines o New pipeline is only a dew weeks old o Generate Tableau dashboards from the data sets o New Features o Add Data o Add backend logs Must-Haves (Concepts & Tools):  Python – Prefers a Software Engineering mindset (CS Fundamentals) vs. Big Data / Hadoop though  Airflow / Spark / PySpark  GCP – Big Query (open to other clouds, but let’s start with this!) o Agro Files are in GCS Buckets (this is the Google version or AWS-S3)  Basics of DevOps CI/CD  Testing – Unit testing experience Nice-to-Haves:  Tableau or PowerBI Development – REALLY nice to have -- ---- Thanks & Regards, Tejash - Technical Recruiter Thought wave Software and Solutions 314 N. Lake St, Suite 6, Aurora IL 60506 Desk: 6304918494 EXTN: 157 Email :tejash@thoughtwavesoft.com Website:www.thoughtwavesoft.com https://www.linkedin.com/in/tejash-chettipalli-342aa2246