Sr Engineer(AI/ML)

Role:Sr Engineer(AI/ML)

Location:Seattle,WA Duration:12 Months

Technology requirements: Proficiency in Apache Spark, including Spark Core, Spark SQL and Spark Streaming. Proficiency in languages such as Python for data processing and scripting. Degree or certifications required: Azure Data Engineering Databricks Spark/DevOps practices

Required background? Skills?: Experience in designing and implementing ETL processes using Databricks notebooks for efficient data extraction, transformation, and loading. In-depth knowledge of the Databricks Unified Analytics Platform and its features for collaborative big data analytics. Understanding of data modeling concepts for designing database structures. Proficiency in working with both relational databases and NoSQL databases. Integration of data from diverse sources, including APIs, databases, streaming data, and external data feeds. Implementation of processes to ensure data quality, including data validation, cleansing, and error handling. Knowledge of cluster management, optimization, and scaling for efficient data processing. Optimization of Spark jobs and Databricks clusters for better performance. Proficiency in cloud platforms such as Azure for building scalable and flexible data architectures. Use of tools like Apache Airflow, ADF, or Databricks to orchestrate and schedule data workflows.

Nice-to-Haves: Knowledge of data security best practices and the implementation of measures to ensure data privacy and compliance. Implementation of monitoring and logging solutions to track the health and performance of pipelines. Familiarity with monitoring platforms like DataDog and New Relic

Bhargavi Ankam IT Recruiter Staffing | Thoughtwave Software and Solutions Limited 6306891366 bhargavi@thoughtwavesoft.com www.thoughtwavesoft.com 314 N. Lake St, Suite 6, Aurora IL 60506 A Certified Minority Business Enterprise, Disadvantaged Business Enterprise, SAM.gov, SOC2 & ISO2005, Vendors for: · STATES: IL, PA, TN, AK, OR, CT, GA, VA, ID, IA, UT, FL, MN & CO, · NATIONAL LABS: ARGONNE & FERMI, · COUNTIES: HENNEPIN, MN, FULTON & GA, · PUBLIC SCHOOLS: ATLANTA.