Source description
About the role
8+ years in data engineering/ETL roles, with at least 4+ years in Azure cloud ETL leadership. Azure certifications (e.g., Azure Analytics Specialty, Solutions Required Qualifications: Azure Data Sources: Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics. External Sources: APIs, on-prem databases, flat files (CSV, Parquet, JSON). Tools : Azure Data Factory (ADF) for orchestration, Databricks connectors. Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing. Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution. Performance Optimization: Partitioning, caching, broadcast joins. Delta Lake: Implementing ACID transactions, time travel, and schema enforcement. Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks. Azure Key Vault: Secure credential management. Azure Monitor & Logging: For ETL job monitoring and alerting. Networking & Security: VNET integration, private endpoints.
More at EXL