Databricks Architect

As a seasoned Databricks Architect, you will play a crucial role in designing, optimizing, and maintaining scalable data pipelines on the Databricks platform. Your responsibilities will include: - Designing end-to-end Lakehouse architecture, encompassing ingestion, transformation, modeling, and consumption. - Developing and maintaining ETL/ELT pipelines using Databricks, leveraging PySpark, SQL, and Notebooks. - Defining and implementing the Medallion Architecture (Bronze/Silver/Gold) with Delta Lake. - Building high-performance ETL/ELT pipelines utilizing PySpark, Spark SQL, and Delta Lake. - Architecting and optimizing batch and streaming data pipelines. - Performing advanced Spark optimization, cluster sizing, and cost management. - Implementing CI/CD, version control, and DevOps best practices within Databricks. Your expertise should cover areas such as: - Databricks Lakehouse Platform - PySpark & Spark SQL - Delta Lake, Structured Streaming - Unity Catalog & governance frameworks - Strong experience with a major cloud platform (Azure/AWS/GCP) - Distributed computing & Spark internals - Data warehousing concepts - Data modeling (dimensional, normalized, Lakehouse modeling) - ETL/ELT architecture - Experience in CI/CD pipelines, Git, orchestration tools - Experience in leading large-scale data platform implementations In addition to the exciting technical challenges, you can look forward to the following benefits: - Competitive salary and benefits package - Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications - Opportunity to work with cutting-edge technologies - Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards - Annual health check-ups - Insurance coverage for self, spouse, two children, and parents At Persistent, we are committed to fostering a values-driven, people-centric, and inclusive work environment. We encourage applications from all qualified individuals, irrespective of disabilities, gender, or gender preference. Our workplace supports hybrid work and flexible hours to accommodate diverse lifestyles. We prioritize accessibility, providing ergonomic setups and assistive technologies for employees with physical disabilities. If you have specific requirements due to disabilities, please let us know during the application process or anytime during your employment. Join us at Persistent (persistent.com/careers) to unleash your full potential in an inclusive and diverse environment. We are an Equal Opportunity Employer that prohibits all forms of discrimination and harassment. As a seasoned Databricks Architect, you will play a crucial role in designing, optimizing, and maintaining scalable data pipelines on the Databricks platform. Your responsibilities will include: - Designing end-to-end Lakehouse architecture, encompassing ingestion, transformation, modeling, and consumption. - Developing and maintaining ETL/ELT pipelines using Databricks, leveraging PySpark, SQL, and Notebooks. - Defining and implementing the Medallion Architecture (Bronze/Silver/Gold) with Delta Lake. - Building high-performance ETL/ELT pipelines utilizing PySpark, Spark SQL, and Delta Lake. - Architecting and optimizing batch and streaming data pipelines. - Performing advanced Spark optimization, cluster sizing, and cost management. - Implementing CI/CD, version control, and DevOps best practices within Databricks. Your expertise should cover areas such as: - Databricks Lakehouse Platform - PySpark & Spark SQL - Delta Lake, Structured Streaming - Unity Catalog & governance frameworks - Strong experience with a major cloud platform (Azure/AWS/GCP) - Distributed computing & Spark internals - Data warehousing concepts - Data modeling (dimensional, normalized, Lakehouse modeling) - ETL/ELT architecture - Experience in CI/CD pipelines, Git, orchestration tools - Experience in leading large-scale data platform implementations In addition to the exciting technical challenges, you can look forward to the following benefits: - Competitive salary and benefits package - Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications - Opportunity to work with cutting-edge technologies - Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards - Annual health check-ups - Insurance coverage for self, spouse, two children, and parents At Persistent, we are committed to fostering a values-driven, people-centric, and inclusive work environment. We encourage applications from all qualified individuals, irrespective of disabilities, gender, or gender preference. Our workplace supports hybrid work and flexible hours to accommodate diverse lifestyles. We prioritize accessibility, providing ergonomic setups and assistive technologies for employees with physical disabilities. If you have specific r