ETL Support Engineer

ETL Support engineer Hybrid; CHARLOTTE 2-3 days a week onsite • Daily Schedule & OT estimate: Monday - Friday, 40 hours a week. They occasionally may have to work weekends but if so, they'd still only work 5 days a week. Candidate Requirements • Years of Experience Required: 5-10 overall years of experience in the field. • Disqualifiers: poor communication skills in the HSA. • Best vs. Average: The ideal resume would contain deep technical skills in a variety of technologies • Performance Indicators: Performance will be assessed based on quality of work and meeting deadlines. • Strong knowledge of Data Management principles • Experience in building ETL / data warehouse transformation processes • Direct experience of building data pipelines using HDInsights and Apache Spark (preferably Databricks). • Experience using geospatial frameworks on Apache Spark and associated design and development patterns • big plus: Microsoft Azure Big Data Architecture certification. • Hands on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics • Experience with Apache Kafka / Nifi for use with streaming data / event-based data

Top 3 Hard Skills Required + Years of Experience

1. Minimum 5+ years experience with HD Insights; Spark
2. Minimum 5+ years experience with Azure Fundamentals and Azure PAAS (desirable)
3. Minimum 5+ years experience with Excellent Verbal and written Comms.

Typical Day in the Role • Purpose of the Team: This team is the Azure database support for MSFT. • Typical task breakdown and operating rhythm: The role will consist of 10% meetings, 90% working with MS customers on support tickets

• What makes this role interesting? - This role provides the opportunity to gain great technical depth in understanding the technology. • Unique Selling Points: Opportunity for FTE conversion

Hard Skills Assessments • Expected Dates that Hard Skills Assessments will be scheduled: asap • Hard Skills Assessment Process: The assessment process will include 2 rounds. one technical HSA with an FTE and the second round with the sponsor.

Job responsibilities: • Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks and/or HDInsights • Delivering and presenting proofs of concept to of key technology components to project stakeholders. • Developing scalable and re-usable frameworks for ingesting of geospatial data sets • Integrating the end to end data piple-line to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times • Working with event based / streaming technologies to ingest and process data • Working with other members of the project team to support delivery of additional project components (API interfaces, Search) • Evaluating the performance and applicability of multiple tools against customer requirements • Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.