Senior Azure Data Engineer

As a skilled Senior Azure Data Engineer at HTC Global Services, you will be responsible for designing, developing, and maintaining scalable data pipelines using Azure services such as Azure Data Factory (ADF) and Azure Databricks. Your expertise in PySpark, SQL, and modern data engineering practices will be crucial in implementing data transformation workflows, optimizing data pipelines, and managing Spark-based data processing frameworks. Collaborating with stakeholders to understand data requirements, ensuring data quality, governance, and security standards, as well as troubleshooting and resolving production issues in data workflows will be part of your key responsibilities. Key Responsibilities: - Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks - Implement data transformation workflows using PySpark - Work with structured and unstructured data in Azure data platforms - Optimize data pipelines for performance, reliability, and cost efficiency - Develop and manage Spark-based data processing frameworks - Collaborate with stakeholders to understand data requirements and deliver solutions - Ensure data quality, governance, and security standards are maintained - Troubleshoot and resolve production issues in data workflows Required Skills: - Solid experience in Azure Data Engineering stack Azure Data Factory (ADF) - Azure Databricks - Hands-on experience with PySpark and Spark - Strong SQL skills - Experience with data warehousing concepts and ETL/ELT processes - Good understanding of distributed data processing Good to Have: - Exposure to Spark Declarative Pipelines / Lakeflow (Delta Live Tables concept) - Experience with Delta Lake - Knowledge of CI/CD pipelines in Azure - Experience with data modeling and optimization techniques Join HTC Global Services for this exciting opportunity to showcase your expertise as a Senior Azure Data Engineer and contribute to premium projects. As a skilled Senior Azure Data Engineer at HTC Global Services, you will be responsible for designing, developing, and maintaining scalable data pipelines using Azure services such as Azure Data Factory (ADF) and Azure Databricks. Your expertise in PySpark, SQL, and modern data engineering practices will be crucial in implementing data transformation workflows, optimizing data pipelines, and managing Spark-based data processing frameworks. Collaborating with stakeholders to understand data requirements, ensuring data quality, governance, and security standards, as well as troubleshooting and resolving production issues in data workflows will be part of your key responsibilities. Key Responsibilities: - Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks - Implement data transformation workflows using PySpark - Work with structured and unstructured data in Azure data platforms - Optimize data pipelines for performance, reliability, and cost efficiency - Develop and manage Spark-based data processing frameworks - Collaborate with stakeholders to understand data requirements and deliver solutions - Ensure data quality, governance, and security standards are maintained - Troubleshoot and resolve production issues in data workflows Required Skills: - Solid experience in Azure Data Engineering stack Azure Data Factory (ADF) - Azure Databricks - Hands-on experience with PySpark and Spark - Strong SQL skills - Experience with data warehousing concepts and ETL/ELT processes - Good understanding of distributed data processing Good to Have: - Exposure to Spark Declarative Pipelines / Lakeflow (Delta Live Tables concept) - Experience with Delta Lake - Knowledge of CI/CD pipelines in Azure - Experience with data modeling and optimization techniques Join HTC Global Services for this exciting opportunity to showcase your expertise as a Senior Azure Data Engineer and contribute to premium projects.