Data Bricks Production Support

Role & responsibilities Job Description Databricks Production Support Engineer (L2/L3 Production Support) Job Title - Databricks Support Engineer Experience - 4 to 8 Years Location - Hybrid Noida Employment Type - Full-Time About the Role We are looking for a highly motivated Databricks Support Engineer to join our Data Engineering team. The ideal candidate will be responsible for supporting and maintaining enterprise-scale data platforms built on the Databricks Lakehouse Platform . This role requires hands-on experience in production support, Databricks administration, Apache Spark, Delta Lake, Airflow, SQL, and cloud technologies , with a strong focus on troubleshooting, performance optimization, and ensuring high availability of data pipelines. The successful candidate should be comfortable working in a fast-paced production environment, collaborating with cross-functional teams, and resolving complex technical issues while meeting strict SLAs. Key Responsibilities Provide L2/L3 production support for Databricks-based data platforms and analytics solutions. Monitor, troubleshoot, and resolve issues related to Databricks Jobs, Workflows, Spark applications, and ETL pipelines. Investigate production incidents, perform Root Cause Analysis (RCA), and implement preventive solutions. Support ETL/ELT pipelines built using PySpark, Spark SQL, and Delta Lake . Optimize Spark jobs for performance, scalability, and cost efficiency. Manage Delta Lake operations including MERGE, OPTIMIZE, VACUUM, schema evolution, and incremental data loads . Troubleshoot and maintain Apache Airflow DAGs, scheduling, retries, and dependency management. Support CI/CD deployments using Azure DevOps, GitHub Actions, or Jenkins. Work with cloud storage services, REST APIs, and enterprise data sources. Monitor cluster utilization, autoscaling, job execution, and resource consumption. Collaborate with Data Engineering and DevOps teams to improve platform reliability and operational excellence. Ensure compliance with security, governance, and operational best practices using Unity Catalog . Maintain technical documentation, runbooks, and production support procedures. Required Technical Skills Databricks Databricks Lakehouse Platform Databricks Workspaces Databricks Jobs & Workflows Delta Lake Unity Catalog Databricks SQL Cluster Management Performance Tuning Job Monitoring & Troubleshooting Data Engineering Apache Spark PySpark Spark SQL SQL Delta MERGE Operations Incremental Data Loading Change Data Capture (CDC) Data Partitioning Z-Ordering OPTIMIZE & VACUUM Adaptive Query Execution (AQE) Medallion Architecture (Bronze, Silver, Gold) Workflow Orchestration Apache Airflow DAG Scheduling Retry Mechanisms Dependency Management Pipeline Monitoring Cloud Platforms Experience with one or more cloud platforms: Microsoft Azure (Preferred) Amazon Web Services (AWS) Google Cloud Platform (GCP) CI/CD & DevOps Git Azure DevOps GitHub Actions Jenkins Deployment Automation Infrastructure as Code concepts Databases Experience with one or more: Databricks SQL Snowflake SQL Server PostgreSQL Amazon Redshift Google BigQuery Preferred Qualifications Experience supporting enterprise-scale production data platforms. Strong understanding of Spark optimization and distributed computing. Experience with monitoring and alerting tools. Hands-on experience with Unity Catalog and data governance. Knowledge of cloud security and access management. Good to Have Terraform Kubernetes Kafka dbt MLflow REST API Integrations Technical Interview Focus Areas Candidates should have practical experience and be able to explain: Project Experience Explain your end-to-end project architecture. Describe your role and responsibilities. Explain the complete data flow from source to reporting. Discuss production issues you have resolved. Explain the largest volume of data you have processed. Databricks & Lakehouse Databricks Architecture Lakehouse Architecture Delta Lake ACID Transactions Databricks Jobs & Workflows Unity Catalog Databricks SQL Difference between Databricks, Snowflake, Redshift, and BigQuery Unity Catalog & Data Governance What is Unity Catalog? Why is Unity Catalog required? Three-level namespace (Catalog Schema Table) Access Control & Permissions Data Lineage Governance and Security Schema Design Candidates should be able to explain: Which schema was used in their project. Why that schema was selected. Difference between Star Schema and Snowflake Schema. Schema evolution and handling source changes. Partitioning strategy and table design. Apache Spark Lazy Evaluation Narrow vs. Wide Transformations Shuffle Optimization Broadcast Joins Caching vs. Persist Adaptive Query Execution (AQE) Repartition vs. Coalesce Delta Lake Delta MERGE Incremental Loading Time Travel Schema Evolution OPTIMIZE VACUUM Z-Ordering File Compaction Apache Airflow DAG Design Scheduling Retry Logic Dependency Management Triggering Databricks Jobs Production Troubleshooting Performance & Cost Optimization Cluster Sizing Autoscaling Job Clusters vs. All-Purpose Clusters Compute Cost Optimization Spark Performance Tuning\ Scenario-Based Problem Solving Candidates should be able to solve real-world scenarios such as: Optimizing a Delta MERGE for large incremental datasets within a strict SLA. Troubleshooting failed Databricks Jobs or Airflow DAGs. Resolving duplicate data issues in Delta tables. Reducing Spark job execution time through tuning and optimization. Designing scalable data pipelines for high-volume enterprise workloads. Performing Root Cause Analysis (RCA) for production incidents. Education Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Preferred candidate profile We are looking for a Databricks Support Engineer with 48 years of experience in supporting enterprise data platforms built on Databricks . The ideal candidate should have hands-on expertise in Databricks, Apache Spark (PySpark), Delta Lake, Apache Airflow, SQL, and Azure/AWS/GCP , with strong experience in L2/L3 production support , troubleshooting, performance tuning, and ETL/ELT pipeline management. The role involves monitoring Databricks Jobs and Workflows, resolving production incidents, performing root cause analysis (RCA), optimizing Spark jobs, managing Delta Lake operations (MERGE, OPTIMIZE, VACUUM), and ensuring SLA compliance. Experience with Unity Catalog, Medallion Architecture, CI/CD (Azure DevOps/GitHub/Jenkins), and cloud-based data engineering is highly preferred. NOTE - THIS POSITION IS ONLY FOR NOIDA LOCATION AND NO PROFILES WILL BE CONSIDERED OUT OF DELHI NCR Contact - Name - Sachin Sharma - 9958500662 Email - sachin.sharma@moptra.com Name - Siddharth Mathur Call / What's App Resume - 9718978697 Email - siddharth.mathur@moptra.com Manager - Global Talent Acquisition MOPTRA INFOTCH PVT LTD