Senior Data Engineer

Enforce adherence to architectural standards, global product-specific guidelines, and usability design standards.Proactively guide engineering methodologies, standards, and leading practices.Identify, communicate, and mitigate risks, assumptions, issues, and decisions throughout the full lifecycle.Provide guidelines for best practices and repeatable methodologies in Cloud Data Engineering, including:Data StorageETL (Extract, Transform, Load)Data Integration MigrationData WarehousingData Governance Required Skills:Experience:9+ years in Data Engineering project development/design, handling large volumes of data.At least 5+ years using cloud data engineering services (preferably on AWS).5+ years as a Big Data architect/solution architect on Big Data platforms.3+ years of proven experience as an architect/solution architect on the Databricks platform. Project Design Implementation:Designed and implemented at least 2-3 end-to-end projects in Databricks.Expertise in E2E architecture of unified data platforms covering data lifecycle aspects: Data Ingestion, Transformation, Serving, and Consumption.Experience in composable architecture to fully leverage Databricks capabilities.Implemented and configured Databricks environments (clusters, notebooks, libraries) for optimal performance and resource utilization.Technical Skills:Designed and developed scalable batch and streaming data pipelines, data lake architectures, and data warehousing solutions on the Databricks platform using Spark and Delta Lake. Knowledgeable in the Databricks Lakehouse concept and its enterprise implementation.Strong understanding of data warehousing, governance, and security standards related to Databricks.Skilled in cluster optimization, integration with various cloud services, and performance optimization to improve efficiency and reduce costs.Proficient in writing unit and integration tests, and setting best practices for Databricks CI/CD.Experience in integrating data from various sources (structured, semi-structured, and unstructured) into Databricks for processing and analysis. Qualifications:Proven experience as an Architect/Solution Architect on Databricks.Hands-on experience with AWS Databricks platform for data processing, warehousing, and analytics solutions.Strong background in cloud data engineering, ETL, data integration, and data governance.Looking forward to receiving applications from candidates who meet these criteria and are excited to join our team.