Databricks Architect

Role Overview: You will be responsible for leading the design and implementation of enterprise-scale Lakehouse data platforms using your expertise in Databricks, Apache Spark, Delta Lake, and cloud-native architectures. Your role will involve delivering end-to-end data engineering solutions across batch and real-time pipelines, requiring strong architectural leadership, hands-on development capability, and collaboration with stakeholders. Key Responsibilities: - Architect and implement enterprise-grade Lakehouse solutions using Databricks - Design scalable cloud-based data platforms integrating multiple data sources - Define data architecture standards, governance, and best practices - Build end-to-end ETL/ELT pipelines using PySpark, Scala, and SQL - Develop batch and real-time streaming pipelines using Spark - Design incremental loading frameworks and metadata-driven ingestion pipelines - Implement and manage Delta Live Tables, Autoloader, Structured Streaming, and Databricks Workflows - Integrate with orchestration tools like Apache Airflow - Design 3NF, dimensional models, and enterprise data warehouse solutions - Implement data quality frameworks and governance standards - Manage Unity Catalog with fine-grained security and access control - Optimize Spark jobs, pipelines, and compute resources for scalability, reliability, and cost efficiency - Implement CI/CD pipelines and deployment strategies for DevOps - Drive DevOps best practices for data engineering - Provide technical leadership and architectural guidance - Collaborate with stakeholders to translate business needs into scalable solutions Qualification Required: - Deep expertise in Databricks and Lakehouse architecture - Strong experience with Apache Spark (batch & streaming) - Hands-on experience with Delta Lake - Strong programming skills in Python, PySpark, Scala, and SQL - Experience with distributed data processing systems - Strong experience in data modeling and data warehousing concepts - Experience with real-time data processing architectures - Hands-on experience with Unity Catalog and data governance Role Overview: You will be responsible for leading the design and implementation of enterprise-scale Lakehouse data platforms using your expertise in Databricks, Apache Spark, Delta Lake, and cloud-native architectures. Your role will involve delivering end-to-end data engineering solutions across batch and real-time pipelines, requiring strong architectural leadership, hands-on development capability, and collaboration with stakeholders. Key Responsibilities: - Architect and implement enterprise-grade Lakehouse solutions using Databricks - Design scalable cloud-based data platforms integrating multiple data sources - Define data architecture standards, governance, and best practices - Build end-to-end ETL/ELT pipelines using PySpark, Scala, and SQL - Develop batch and real-time streaming pipelines using Spark - Design incremental loading frameworks and metadata-driven ingestion pipelines - Implement and manage Delta Live Tables, Autoloader, Structured Streaming, and Databricks Workflows - Integrate with orchestration tools like Apache Airflow - Design 3NF, dimensional models, and enterprise data warehouse solutions - Implement data quality frameworks and governance standards - Manage Unity Catalog with fine-grained security and access control - Optimize Spark jobs, pipelines, and compute resources for scalability, reliability, and cost efficiency - Implement CI/CD pipelines and deployment strategies for DevOps - Drive DevOps best practices for data engineering - Provide technical leadership and architectural guidance - Collaborate with stakeholders to translate business needs into scalable solutions Qualification Required: - Deep expertise in Databricks and Lakehouse architecture - Strong experience with Apache Spark (batch & streaming) - Hands-on experience with Delta Lake - Strong programming skills in Python, PySpark, Scala, and SQL - Experience with distributed data processing systems - Strong experience in data modeling and data warehousing concepts - Experience with real-time data processing architectures - Hands-on experience with Unity Catalog and data governance