Source description
About the role
We are looking for a Lead Data Engineer / Technical Anchor with strong expertise in Databricks on AWS to lead the architecture, engineering, and delivery of enterprise-scale Data Lakehouse solutions. This is a hands-on technical leadership role involving architecture, development, mentoring, and stakeholder collaboration. Required Skills DatabricksAWS (S3, EMR, Athena, Glue, CloudWatch)PySpark / Apache SparkPythonAdvanced SQLDelta Lake & Delta Live Tables (DLT)Unity CatalogMedallion Architecture (Bronze, Silver, Gold)Data EngineeringPerformance Tuning & Spark OptimizationKey Responsibilities Design and implement scalable Data Lakehouse architectures using Databricks on AWS.Develop robust batch and streaming data pipelines using PySpark, SQL, Delta Lake, and Delta Live Tables.Configure and manage Unity Catalog across multiple environments.Implement secure AWS data platform architectures using IAM, KMS, VPC, and PrivateLink.Optimize Spark performance, cluster utilization, and compute costs.Lead architecture discussions, conduct code reviews, and mentor Data Engineers.Collaborate with product owners, architects, and client stakeholders to deliver enterprise data solutions.Preferred Qualifications 8+ years of Data Engineering experience.Minimum 4+ years of hands-on experience with Databricks on AWS.Strong expertise in PySpark, Python, SQL, and AWS.Experience with Data Lakehouse architecture and enterprise-scale data platforms.Excellent communication and leadership skills. .
More at Obrimo Technologies