Data Engineer

Required Skills : Data Engineer using PySpark to produce Data Bricks in an Azure Data Lake. Need to have BI reporting understanding – either PowerBI or Microstrategy. Interview Process: 1 and Done Video (Microsoft Teams Interview) – 30 minutes **Will complete Apex technical Screening prior to client interview. Duration: SOW is through end of 2021 with extension possible Key: • Using PySpark to produce Data Bricks in an Azure Data Lake. Need to have BI reporting understanding – either PowerBI or Microstrategy. • This team is building the data pipelines for the data to come into the Data Lake for PowerBI reporting. Technical Competencies: • Develop Business Intelligence applications/dashboards/Reports in Microstrategy and Power BI. • Develop complex SQL/USQL/pySpark code for Data Engineering pipelines in Azure Data Lake analytics and azure data factory. • Develop complex data science algorithms using R programming language and industrialize them. • Develop and Implement big data applications using Hadoop components (HDInsight - Microsoft Azure). • Create requirement documents, review the functional design documents created by vendor partners as a part of software development life cycle. • Work closely with business teams in every stage from gathering requirements to closure of project. • Work with the key stakeholders to create requirements for each business solution, ensure the requirements are developed in concert with and agreed upon by the business partner, and that the solution delivers the agreed upon business needs.