Databricks solution architect (Jaipur)

Databricks Solution Architect Position: Databricks Solution Architect Experience Level: 8+ years Key Responsibilities Next-Gen Architecture Delivery: Design and deploy contemporary Data Lakehouse solutions using Serverless Compute (for SQL, Jobs, and Notebooks) to eliminate infrastructure overhead and optimize TCO. Production Engineering: Write, optimize, and deploy production-grade code. You are expected to fix what breakswhether its a shuffling error in Spark, a CI/CD failure in Databricks Asset Bundles (DABs), or a complex Mosaic AI vector search pipeline. Deep Performance Tuning: Diagnose and resolve bottlenecks (Skew, OOM, Spill). You must know when to apply legacy tuning (Z-Order/Partitioning) vs. modern Liquid Clustering to handle changing data patterns automatically. Governance & Federation: Implement strict data governance using Unity Catalog. Configure Lakehouse Federation to query external systems without data movement, and enforce Attribute-Based Access Control (ABAC). Customer Obsession: Act as the "Expert in the Room." Explain complex concepts (like why Serverless reduces cold starts or how AI/BI Genie handles hallucination) to non-technical stakeholders. Must-Have Skills Core Spark & Liquid Clustering: Expert knowledge of the Catalyst Optimizer and AQE. You must understand the architectural shift from static partitioning to Liquid Clustering and how it impacts file layout and query skipping. Coding Fluency: Advanced proficiency in Python (PySpark) and SQL. You must be comfortable livecoding complex UDFs and transformation logic without reliance on Google. Delta Lakehouse Mastery: Deep knowledge of ACID transactions, Time Travel, and Delta Live Tables (DLT) for declarative pipeline management. Unity Catalog & Security: Hands-on experience with System Tables for observability, Lakehouse Federation for cross-platform querying, and setting up Volume based access controls. Operational Excellence: Experience with Infrastructure-as-Code (Terraform), Databricks Asset Bundles (DABs) for CI/CD, and Databricks Connect v2 for local development. Nice-to-Have Skills SAP Data Integration (High Value): Experience extracting data from SAP systems (ECC, S/4HANA) into Databricks Delta Lake. Familiarity with SAP Datasphere, SAP SLT, or Partner Connectors (e.g., Fivetran/Qlik for SAP) for operational reporting. Understanding of common SAP data structures (IDOCs, BAPIs) is a major plus. Advanced AI & Machine Learning: ML Engineering: Hands-on experience with MLflow for experiment tracking and model registry. Familiarity with Feature Store implementation. GenAI/Mosaic AI: Experience fine-tuning LLMs (Foundation Models), using Mosaic AI Gateway for governance, or building conversational analytics with AI/BI Genie. Libraries: Proficiency with core ML libraries (Scikit-learn, XGBoost, PyTorch) for non-generative use cases. Serverless Architectures: Proven experience migrating workloads from Classic Compute to Serverless, including cost analysis. Certifications: Databricks Certified Data Engineer Professional (Strongly Preferred). Databricks Solution Architect Position: Databricks Solution Architect Experience Level: 8+ years Key Responsibilities Next-Gen Architecture Delivery: Design and deploy contemporary Data Lakehouse solutions using Serverless Compute (for SQL, Jobs, and Notebooks) to eliminate infrastructure overhead and optimize TCO. Production Engineering: Write, optimize, and deploy production-grade code. You are expected to fix what breakswhether its a shuffling error in Spark, a CI/CD failure in Databricks Asset Bundles (DABs), or a complex Mosaic AI vector search pipeline. Deep Performance Tuning: Diagnose and resolve bottlenecks (Skew, OOM, Spill). You must know when to apply legacy tuning (Z-Order/Partitioning) vs. modern Liquid Clustering to handle changing data patterns automatically. Governance & Federation: Implement strict data governance using Unity Catalog. Configure Lakehouse Federation to query external systems without data movement, and enforce Attribute-Based Access Control (ABAC). Customer Obsession: Act as the "Expert in the Room." Explain complex concepts (like why Serverless reduces cold starts or how AI/BI Genie handles hallucination) to non-technical stakeholders. Must-Have Skills Core Spark & Liquid Clustering: Expert knowledge of the Catalyst Optimizer and AQE. You must understand the architectural shift from static partitioning to Liquid Clustering and how it impacts file layout and query skipping. Coding Fluency: Advanced proficiency in Python (PySpark) and SQL. You must be comfortable livecoding complex UDFs and transformation logic without reliance on Google. Delta Lakehouse Mastery: Deep knowledge of ACID transactions, Time Travel, and Delta Live Tables (DLT) for declarative pipeline management. Unity Catalog & Security: Hands-on experience with System Tables for observability, Lakehouse Federation for cross-platform quer