Databricks Solution Architect (RSA)

Company Description Celebal Technologies is a software services company specializing in Data Science, Big Data, and Enterprise Cloud solutions. The company focuses on turning Data to Intelligence by leveraging advanced analytics, AI, ML, and robotics to help clients make faster, smarter business decisions. Celebal Technologies partners with global product ISVs and delivery organizations, supporting both product development and complex enterprise implementations. Its teams work across diverse industries, modernizing legacy systems with cloud-native architectures, microservices, and API management. The organization emphasizes high service quality and operational excellence to help clients become truly data-driven enterprises. Position: Databricks Solution Architect Experience Level: 8+ years Location: Onsite (All Celebal Office Locations - Jaipur, Noida, Gurugram, Pune, Hyderabad, Bengaluru) Position Overview: We are seeking a highly skilled and experienced Databricks Solution Architect to join our dynamic team. As a senior technical leader, you will design, deploy, and optimize next-generation Data Lakehouse solutions. The ideal candidate is a hands-on expert who can seamlessly transition from high-level architectural design to deep production code optimization and infrastructure troubleshooting. Key Responsibilities: Next-Gen Architecture: Design and deploy modern Data Lakehouse solutions using Serverless Compute (SQL, Jobs, Notebooks) to eliminate infrastructure overhead.Production Engineering: Write, optimize, and deploy production-grade code. Troubleshoot and fix Spark shuffling errors, CI/CD failures, and complex vector search pipelines.Performance Tuning: Diagnose and resolve performance bottlenecks (Skew, OOM, Spill) using modern techniques like Liquid Clustering and legacy tuning (Z-Order/Partitioning).Governance & Federation: Implement strict data governance using Unity Catalog. Configure Lakehouse Federation for cross-platform querying and enforce Attribute-Based Access Control (ABAC).Stakeholder Management: Act as the technical expert to explain complex cloud, data, and AI concepts to non-technical business stakeholders. Must-Have Technical Skills: Core Spark & Architecture: Expert knowledge of Catalyst Optimizer, AQE, and the transition from static partitioning to Liquid Clustering.Coding Fluency: Advanced, hands-on proficiency in Python (PySpark) and SQL for complex transformation logic and UDFs.Delta Lakehouse: Deep expertise in ACID transactions, Time Travel, and Delta Live Tables (DLT).Unity Catalog & Security: Practical experience with System Tables for observability and Volume-based access controls.Operational Excellence: Hands-on experience with Infrastructure-as-Code (Terraform), Databricks Asset Bundles (DABs) for CI/CD, and Databricks Connect v2. Nice-to-Have Skills (High Value): SAP Data Integration: Experience extracting data from SAP systems (ECC, S/4HANA, Datasphere, SLT) into Databricks Delta Lake using partner connectors (Fivetran/Qlik).Advanced AI & Machine Learning: Hands-on experience with MLflow, Feature Stores, Mosaic AI Gateway, LLM fine-tuning, and AI/BI Genie.Serverless Migration: Proven experience migrating workloads from Classic Compute to Serverless, including cost-benefit analysis.Certification: Databricks Certified Data Engineer Professional is strongly preferred. Company Description Celebal Technologies is a software services company specializing in Data Science, Big Data, and Enterprise Cloud solutions. The company focuses on turning Data to Intelligence by leveraging advanced analytics, AI, ML, and robotics to help clients make faster, smarter business decisions. Celebal Technologies partners with global product ISVs and delivery organizations, supporting both product development and complex enterprise implementations. Its teams work across diverse industries, modernizing legacy systems with cloud-native architectures, microservices, and API management. The organization emphasizes high service quality and operational excellence to help clients become truly data-driven enterprises. Position: Databricks Solution Architect Experience Level: 8+ years Location: Onsite (All Celebal Office Locations - Jaipur, Noida, Gurugram, Pune, Hyderabad, Bengaluru) Position Overview: We are seeking a highly skilled and experienced Databricks Solution Architect to join our dynamic team. As a senior technical leader, you will design, deploy, and optimize next-generation Data Lakehouse solutions. The ideal candidate is a hands-on expert who can seamlessly transition from high-level architectural design to deep production code optimization and infrastructure troubleshooting. Key Responsibilities: Next-Gen Architecture: Design and deploy modern Data Lakehouse solutions using Serverless Compute (SQL, Jobs, Notebooks) to eliminate infrastructure overhead.Production Engineering: Write, optimize, and deploy production-grade code. Troubleshoot and fix