Padmi

Qentelli - Microsoft Fabric Data Engineer - Python/PySpark/SQL

HyderabadPosted 1 month ago
Infrastructure And DatabasesSeniorFull Time; Regular
Apply at Qentelli

Opens the source posting on shine.com

Source description

About the role

View original

Job Summary : We are seeking a Senior Fabric Data Engineer with strong hands-on Microsoft Fabric experience to support migration and modernization of learning-platform data from Saba and Cura into the Cornerstone Galaxy ecosystem. This role will focus on building scalable ingestion, transformation, business-rule, and curated data layers to enable analytics, compliance reporting, skills insights, and downstream integrations. Work Youll Do : - Design and implement robust data solutions using Microsoft Fabric, including OneLake, Fabric Pipelines, Lakehouse, Warehouse, and Notebooks. - Build and optimize scalable ingestion and transformation pipelines using PySpark, Python, and SQL across structured and unstructured datasets. - Migrate and harmonize data from legacy source systems including Saba, Cura, and related enterprise platforms. - Develop API-driven and scheduled workflows to ingest data from internal and external systems and support downstream processing. - Implement business rules for learning history, completion status, certifications, compliance, skills, and learner profile data. - Build curated and consumption-ready datasets/models for analytics, dashboards, and operational reporting. - Perform SQL development and tuning for reporting, analytics, and downstream consumption. - Implement automated data quality checks, reconciliations, monitoring, exception handling, and audit logging. - Enforce secure access controls using Azure AD, RBAC, managed identities, and secure connectivity patterns. - Support CI/CD, code promotion, release management, and operational monitoring in Agile delivery. Required Skills & Qualifications : - 7+ years of experience in data engineering and enterprise data integration. - Strong hands-on experience with Microsoft Fabric and related Azure data services. - Strong experience with PySpark, SparkSQL, Python, and SQL/T-SQL. - Experience building ingestion and transformation pipelines from APIs, files, databases, and third-party platforms. - Experience with curated data layers, data modeling, and analytics-ready datasets. - Experience tuning data pipelines for scale, resiliency, and cost optimization. - Working knowledge of security, access controls, and data governance practices. - Experience with Azure DevOps and Git-based deployment automation. - Strong analytical, troubleshooting, and communication skills. Preferred Skills : - Experience with learning, HR, or talent data domains. - Familiarity with ADF, Synapse, Databricks, Azure Functions, or Logic Apps. - Exposure to AI tools such as GitHub Copilot and Claude to accelerate engineering productivity and solution design. - Experience with data governance tooling such as Purview. - Exposure to AI/ML-ready data preparation and feature engineering practices. Education : - Bachelors or Masters degree in Computer Science, Information Systems, Engineering, or a related field. Work Environment : - Ability to work independently and proactively in a fast-paced delivery environment. Job Summary : We are seeking a Senior Fabric Data Engineer with strong hands-on Microsoft Fabric experience to support migration and modernization of learning-platform data from Saba and Cura into the Cornerstone Galaxy ecosystem. This role will focus on building scalable ingestion, transformation, business-rule, and curated data layers to enable analytics, compliance reporting, skills insights, and downstream integrations. Work Youll Do : - Design and implement robust data solutions using Microsoft Fabric, including OneLake, Fabric Pipelines, Lakehouse, Warehouse, and Notebooks. - Build and optimize scalable ingestion and transformation pipelines using PySpark, Python, and SQL across structured and unstructured datasets. - Migrate and harmonize data from legacy source systems including Saba, Cura, and related enterprise platforms. - Develop API-driven and scheduled workflows to ingest data from internal and external systems and support downstream processing. - Implement business rules for learning history, completion status, certifications, compliance, skills, and learner profile data. - Build curated and consumption-ready datasets/models for analytics, dashboards, and operational reporting. - Perform SQL development and tuning for reporting, analytics, and downstream consumption. - Implement automated data quality checks, reconciliations, monitoring, exception handling, and audit logging. - Enforce secure access controls using Azure AD, RBAC, managed identities, and secure connectivity patterns. - Support CI/CD, code promotion, release management, and operational monitoring in Agile delivery. Required Skills & Qualifications : - 7+ years of experience in data engineering and enterprise data integration. - Strong hands-on experience with Microsoft Fabric and related Azure data services. - Strong experience with PySpark, SparkSQL, Python, and SQL/T-SQL. - Experience building ingestion and transformat

One address, no account. We’ll tell you when matching roles go live.

More at Qentelli

Related open roles

View all roles