Lead II - Data Engineering

We are looking for a skilled Azure Data Engineer to design, develop, and optimize scalable data solutions on the Azure cloud platform. The ideal candidate should have strong expertise in Azure data services, big data technologies, and data transformation frameworks. Primary Skills (Must Have)

Azure Databricks - Development of scalable data pipelines, Spark-based transformations, performance tuning

Azure Data Factory (ADF) - Designing and orchestrating ETL/ELT workflows

PySpark - Data processing, transformation, and optimization

SQL - Advanced querying, performance tuning, data modeling

Secondary Skills (Good to Have)

Big Data Technologies - Hadoop ecosystem exposure, distributed data processing

PL/SQL - Stored procedures, performance optimization

Python - Scripting, automation, and data manipulation

Oracle Apps Technical - Interfaces, conversions, reports, integrations

Key Responsibilities

Design, build, and maintain robust ETL/ELT pipelines using Azure Data Factory and Azure Databricks.

Develop scalable data transformation logic using PySpark.

Optimize data workflows for performance, scalability, and reliability.

Perform data modeling and implement SQL-based transformations.

Ensure data quality, governance, and security compliance.

Collaborate with business stakeholders, analysts, and architects to translate requirements into technical solutions.

Support production deployments, monitoring, and issue resolution.

Preferred Qualifications

• Experience working in Azure cloud environments.

• Exposure to CI/CD for data pipelines.

• Strong analytical and problem-solving skills.

• Good communication and stakeholder management abilities.

• azure data factory,etl,sql,azure databricks,pyspark,python,azure data engineer