Source description
About the role
Job Description – Senior Data Engineer (Azure Databricks | Fabric | PySpark) Position Senior Data Engineer Experience 5+ Years Primary (Must-Have) Skills
Azure Databricks PySpark Python Microsoft Fabric Azure Data Factory (ADF) Azure Synapse Analytics Informatica ETL/Data Engineering SQL Data Quality Frameworks DataOps CI/CD Pipeline implementation Solution Design & Architecture Strong communication and stakeholder management skills
Secondary Skills
Apache Spark Azure Cloud Services Data Modeling Performance Optimization Azure DevOps Git Agile/Scrum methodology JavaScript (basic knowledge is acceptable)
Roles & Responsibilities
Design, develop, and optimize scalable data pipelines using Azure Databricks, PySpark, and Python. Build and maintain ETL/ELT workflows using Azure Data Factory, Synapse, Informatica, and Microsoft Fabric. Develop robust data engineering solutions to support business intelligence and analytics initiatives. Ensure data quality, integrity, governance, and monitoring across data platforms. Implement DataOps best practices and automate deployments using CI/CD pipelines. Translate functional and business requirements into technical solutions and development tasks. Collaborate with business stakeholders, product owners, architects, and cross-functional teams. Troubleshoot production issues, optimize performance, and ensure high availability of data solutions. Participate in solution design discussions, technical reviews, and estimation activities. Follow coding standards, documentation practices, and industry best practices.
Good-to-Have Skills
Domain experience in Manufacturing or Retail Experience with Azure DevOps Knowledge of Delta Lake and Lakehouse architecture Experience with Data Warehousing concepts Exposure to Power BI Experience with cloud migration projects Certification in Azure Data Engineering or Databricks Knowledge of AI/ML data preparation pipelines
Required Soft Skills
Excellent communication and interpersonal skills Strong analytical and problem-solving abilities Client-facing experience Ability to work independently as well as in a team Strong requirement analysis and solutioning skills
Apache Spark, Azure, Databricks, JavaScript, Python
More at UST