Specialist - Data Engineering

Role description The Senior Data Engineer will be responsible for designing developing and maintaining largescale data ingestion and processing pipelines The role focuses on Oracle to Big Data ingestion Sparkbased transformations and production support for critical data pipelines Roles Responsibilities Data Engineering Development Design and build endtoend data pipelines from OracleRDBMS to Big Data platforms Develop scalable transformations using Apache Spark PySpark Scala Spark SQL Implement staging curated and consumption layers Create and manage Hive external tables ORC Iceberg Handle snapshot incremental and CDCbased ingestion Scheduling Operations Develop and maintain Autosys jobs JILs box jobs dependencies Monitor batch pipelines and ensure SLA adherence Provide production support issue analysis and RCA DevOps Collaboration Use Git Bitbucket for version control and PR reviews Support CICD deployments RLM Jenkins Lightspeed Prepare technical documentation runbooks and KT documents Mentor junior engineers Required Skills Apache Spark PySpark Scala Spark SQL Strong SQL Oracle SQL Iceberg Unix Shell scripting Autosys scheduling Git Bitbucket Production support experience Good to Have Iceberg Starburst Trino CDP Public Cloud Databricks Data quality frameworks Mandatory Certification Cloudera CCA Spark and Hadoop Developer CCA175