Source description
About the role
We do have a job opening for DataOps/Cloud Data Engineer - Senior for our direct client Province of Ontario. The position details are given below: Job Details: Job ID: RQ11099 - DataOps/Cloud Data Engineer - Senior Client : Health Services Cluster Location : 5700 Yonge St. Toronto , Ontario Duration: 1 year + Extension Type: Onsite Job Description: Responsibilities: Design, develop, test, implement, and troubleshoot: data pipelines using various tools such as Databricks, Azure Data Factory, AWS Gluecomplex data transformation procedures, data models for data warehouses and lakehouses Experience and Skill Set Requirements: Programming & scripting: Python, SQL, Linux shell, PowerShell. Data manipulation/analysis using pandas and pyspark. Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data AWS and/or Azure services. Cloud data warehouse solutions such as AWS Redshift. Data lakehouse solutions such as Databricks (Delta Lake). Data processing orchestration/automation ETL tools (such as Azure Data Factory, AWS Glue) as well as cloud agnostic tools such as Informatica IDMC. Data modeling & architecture: relational and dimensional modeling. Data reporting/visualization. Full SDLC from requirements gathering, design, implementation, testing to deployment and production support Project management (agile/scrum and waterfall) Change and Incident management Communication, presentation and negotiation skills Consulting, problem-solving and decision-making skills Experience working in OPS and/or public sector in general MUST HAVES: Experience using AWS Services such as Glue, StepFunctions, Lambda, S3 Extract/Transform/Load data using tools such as Informatica IDMC
More at ITProfound