Source description
About the role
Role: Data Engineer with Azure, Spark and Python
Location: Redmond, WA, Remote but locals required.
Primary Responsibilities • Design and implement data ingestion, data transformation, and data storage pipelines that are reliable and performant. • Develop data foundational models for Customer Development and Supply Chain datasets. • Design and implement semantic objects in Azure Analysis Services to incorporate business logic, incremental loading, and performance into data models. • Introduce design elements to handle data volume and complexity including partitioning, incremental loading, and data aggregation/filtering for reduction. • Perform unit testing, data validation testing, and performance testing on all components of data engineering pipelines. • Support data engineering needs of data science exercises to provide data in the right format and structure. • Collaborate with stakeholders to understand and define analyses that utilize advanced analytic techniques (descriptive analytics, statistical techniques, Machine Learning, etc.). • Attend daily stand-up calls to discuss progress and blockers and be responsible for reasonable tracking of activities within Kanban framework. • Perform exploratory data analysis and experimentation in an effort to answer business questions with data science applications and identify opportunities for productionization. • Support ongoing data engineering & business intelligence activities and projects
More at Thoughtwave Software and Solutions