Data Engineer

Data Engineer Contract 3 months- extendable Exp- 10 years overall, 7 years relevant Mandatory skills Corporate Banking Knowledge. Strong hands-on experience in developing, optimizing, and managing ETL pipelines, particularly for large-scale, real-time data. Advanced proficiency in SQL. Expertise in Python programming languages. In-depth experience with Azure services (e.g., Azure Data Factory, Azure Databricks) Strong experience with Cloudera/Hadoop ecosystem, including HDFS, Hive, Spark, Knowledge of data governance, data quality, and data security best practices. Any Data engineer Professional Certification. Strong analytical and problem-solving skills Desired Skills Real-time Data Processing Basic experience with machine learning models and tools such as Azure ML Roles Responsibilities Data Pipeline Management Design and optimize ETL pipelines to integrate corporate banking data into the data lake (Cloudera/Hadoop, Azure + Databricks) Data Integration Transformation Integrate data from multiple sources, transforming it for analytics using Apache Spark and Databricks. Automation Optimization Automate data processes and optimize pipeline performance. Data Quality Validation Implement data quality checks to ensure accurate and reliable data. Collaboration Work with business teams and IT to design and document data solutions. Overall years of experience 10+ years of experience in data engineering, with a focus on large-scale data pipelines. Relevant years of experience 7+ years of hands-on experience with Cloudera/Hadoop, Azure, Databricks, and other relevant technologies Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.