Reveille Technologies - Databricks Architect/Lead

We are looking for an experienced Databricks Architect Lead to design, build, and modernize enterprise-scale data platforms using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in cloud-based data engineering, distributed data processing, and modern data architecture while leading technical teams and driving large-scale data transformation initiatives. The role requires deep hands-on experience with Databricks, Apache Spark, PySpark, Delta Lake, cloud services, and enterprise data architecture. Key Responsibilities: - Design and implement enterprise-scale Databricks Lakehouse architectures. - Lead the migration of legacy data warehouses and ETL platforms to Databricks. - Build scalable and high-performance batch and real-time data pipelines. - Develop data ingestion, transformation, and orchestration frameworks using PySpark and Spark SQL. - Design Bronze, Silver, and Gold layer architectures following Medallion Architecture principles. - Optimize Spark workloads for performance, scalability, and cost efficiency. - Implement Delta Lake features including ACID transactions, time travel, schema evolution, and optimization. - Architect secure and governed data platforms using Unity Catalog. - Design enterprise data models for analytics, reporting, AI, and Machine Learning workloads. - Build reusable data engineering frameworks and coding standards. - Collaborate with business stakeholders to translate business requirements into technical solutions. - Mentor data engineers, conduct architecture reviews, and establish engineering best practices. - Lead proof-of-concepts and technology evaluations for modern data platforms. - Work closely with DevOps teams to implement CI/CD pipelines for Databricks deployments. - Ensure platform reliability, monitoring, observability, security, and governance across data workloads. Required Technical Skills: - Strong experience with Databricks Lakehouse Platform - Apache Spark & PySpark - Spark SQL - Delta Lake - Python - SQL - Data Engineering - Data Architecture - ETL & ELT Design - Medallion Architecture - Unity Catalog - Databricks Workflows - Databricks Repos - MLflow (preferred) Qualifications & Experience: - Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. - 10+ years of experience in Data Engineering, Big Data, or Data Architecture. - 4+ years of hands-on experience with Databricks platform development and architecture. - Strong expertise in distributed computing using Apache Spark. - Experience designing enterprise data lakes and Lakehouse architectures. - Strong understanding of cloud-native architecture patterns. - Experience implementing secure and governed enterprise data platforms. - Excellent analytical, troubleshooting, and stakeholder management skills. - Experience leading technical teams and mentoring engineers. - Databricks Certified Professional or Associate Certification. - Experience with Data Mesh or Data Fabric architectures. - Knowledge of Kafka, Event Hubs, or real-time streaming platforms. - Experience with AI/ML pipelines and MLOps. - Terraform or Infrastructure as Code. - Kubernetes exposure. - Experience with Snowflake migration or legacy data warehouse modernization. - Strong architecture and solution design expertise. - Leadership experience managing enterprise data engineering initiatives. - Excellent communication and client-facing skills. - Ability to work with cross-functional global teams. - Passion for building scalable, cloud-native data platforms and driving innovation. We are looking for an experienced Databricks Architect Lead to design, build, and modernize enterprise-scale data platforms using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in cloud-based data engineering, distributed data processing, and modern data architecture while leading technical teams and driving large-scale data transformation initiatives. The role requires deep hands-on experience with Databricks, Apache Spark, PySpark, Delta Lake, cloud services, and enterprise data architecture. Key Responsibilities: - Design and implement enterprise-scale Databricks Lakehouse architectures. - Lead the migration of legacy data warehouses and ETL platforms to Databricks. - Build scalable and high-performance batch and real-time data pipelines. - Develop data ingestion, transformation, and orchestration frameworks using PySpark and Spark SQL. - Design Bronze, Silver, and Gold layer architectures following Medallion Architecture principles. - Optimize Spark workloads for performance, scalability, and cost efficiency. - Implement Delta Lake features including ACID transactions, time travel, schema evolution, and optimization. - Architect secure and governed data platforms using Unity Catalog. - Design enterprise data models for analytics, reporting, AI, and Machine Learning workloads. - B