Senior Data Engineer

Senior Data Engineer Key Responsibilities Design, build, and maintain scalable data pipelines using Python & PySpark Develop and optimize ETL/ELT workflows for structured and semi-structured data Work with Azure data services to ingest, process, and store large datasets Write efficient SQL queries for data analysis, transformation, and reporting Collaborate with data analysts, data scientists, and business teams to deliver reliable datasets Implement data quality checks, monitoring, and performance tuning Handle batch and (if applicable) streaming data workloads Follow best practices for data security, governance, and documentation Mandatory Skills Python PySpark Azure Required Skills Programming & Data Processing Strong hands-on experience with Python Solid expertise in PySpark / Apache Spark Experience with data transformations, joins, aggregations, window functions Databases & SQL Advanced SQL (CTEs, indexing, query optimization) Experience with relational and analytical databases Cloud & Azure Hands-on experience with Microsoft Azure , including: Azure Data Factory (ADF) Azure Data Lake Storage (ADLS) Azure Synapse Analytics Azure Databricks Understanding of cloud-based data architecture Data Engineering Concepts ETL vs ELT Data modeling (Star / Snowflake schemas) Partitioning, caching, performance tuning Handling large-scale datasets Good to Have Experience with streaming tools (Kafka, Event Hubs) Knowledge of CI/CD pipelines for data workflows Exposure to Airflow / Azure Data Factory scheduling Understanding of DevOps / Git / version control Basic knowledge of data governance & compliance Education Bachelor s degree in Computer Science, Engineering, or related field (Equivalent practical experience is welcome) Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.