Source description
About the role
Design and implement ETL/ELT pipelines using Azure Databricks, Scala, and Spark Develop and maintain data models, schemas, and data lake structures Perform data ingestion from various sources (structured/unstructured) Optimize data transformation workflows for performance and scalability Collaborate with data analysts, and business stakeholders Ensure data quality, integrity, and security across the platform Implement CI/CD pipelines for data engineering workflows Monitor and troubleshoot data pipelines and jobs Mandatory skills Strong experience with Azure Databricks, Apache Spark, and Scala Proficiency in Azure Data Lake, Azure Synapse, Azure Data Factory Solid understanding of data modeling (star/snowflake schemas, normalization) Experience with SQL, Python, and Delta Lake Knowledge of Unified Data Architecture and data governance Familiarity with DevOps practices and version control (Git) Experience with data visualization tools (Power BI, Tableau) is a plus Desired/ Secondary skills Good experience in CI/CD pipelines for data engineering workflow
More at CLIFYX INC