Source description
About the role
Role:Data Engineer
Location:100% Remote Duration:12+ Months
Must Have: Databricks Azure cloud experience (ADF, Databricks, ADLS) Airflow or Kafka Experience building data pipelines and automating ETL processes for machine learning models
Full Details: • Create data pipelines that bring in data for ML Model training, operationalization, and evaluation • ETL work to automate transfer of data required to support ML Models. • Support data modelling and schema design. • Development within an Azure environment using tools such as Azure Storage, Data Lake, Data Factory, Databricks, CosmosDB, and open source tools such as Apache Airflow and Kafka • Create CI/CD pipelines and perform unit and integration testing. • Technology and tools being used include, but are not limited to Azure ML, AKS, Data Lake, Data Factory, Databricks, Python, Shell Scripting, CRON Jobs and Terraforms
Bhargavi Ankam IT Recruiter Staffing | Thought wave Software and Solutions Limited 6306891366 bhargavi@wavsols.com www.thoughtwavesoft.com 314 N. Lake St, Suite 6, Aurora IL 60506
More at Thoughtwave Software and Solutions