Source description
About the role
Job Description :Technical Skills :- Deep expertise in Databricks platform (Unity Catalog, Delta Lake, MLflow, Workflows).- Strong proficiency in PySpark, Scala, or SQL for large-scale data processing.- Experience designing and building medallion architecture (Bronze/Silver/Gold).- Hands-on with cloud platforms, preferably Azure.- Familiarity with CI/CD pipelines and DevOps practices for data engineering.Architecture & Design :- Ability to design and implement modern data platforms using Databricks Lakehouse architecture end-to-end.- Define data architecture, data models, and governance frameworks tailored for enterprise analytics platforms.- Experience migrating legacy data warehouses (Teradata, Snowflake, etc.) to Databricks.- Ability to design scalable, cost-optimized data lakehouse solutions.- Can produce architecture diagrams, technical documentation, and lead design reviews.Data Modeling & Legacy-to-Cloud Transition :- Strong data modeling background with hands-on experience in legacy systems such as SQL Server and Oracle.- Proven track record of transitioning from traditional relational/on-prem data models to modern cloud-native architectures.- Ability to re-platform legacy schemas, stored procedures, and ETL pipelines into Databricks lakehouse patterns.- Experience translating dimensional models (star/snowflake schemas) into lakehouse-optimized Delta Lake designs.- Comfortable bridging the gap between legacy DBA/data modeler roles and modern cloud architecture responsibilities.Data Governance, Security & Compliance :- Strong understanding of metadata management, data lineage, data quality, and governance frameworks.- Hands-on with Databricks Unity Catalog for centralized governance, access control, and lineage tracking.- Experience enforcing data security and compliance standards, particularly within Financial Services (SOX, PCI-DSS, GDPR).- Ability to implement role-based access control (RBAC), column/row-level security, and data masking.- Familiarity with data quality frameworks and tooling integrated within the Databricks ecosystem.AI & Machine Learning Features (Databricks-Native) :- Hands-on experience with Databricks Mosaic AI for building, training, and deploying ML models.- Proficiency with MLflow (native to Databricks) for experiment tracking, model registry, and serving.- Experience with Databricks Feature Store for creating and managing reusable ML features.- Ability to build GenAI applications using Databricks Vector Search and Mosaic AI Model Serving.- Familiarity with Databricks AI/BI and the Genie interface for natural language analytics.Soft Skills & Collaboration :- Strong communication skills for working across time zones with onshore teams.- Able to lead technical discussions, produce architecture diagrams, and document decisions.- Experience mentoring junior engineers.Engagement Expectations :- Willingness to participate in design reviews, sprint planning, and stakeholder calls.- Deliverable-driven with ability to work independently.Good to have :- Good understanding of Agent bricks and Databricks Generative AI capabilities.Education :- UG: B.Tech / B.E. in Any Specialization.Key Skills :- Skills highlighted with are preferred keyskills: Architect, Azure, Databricks, Artificial Intelligence, SQL, Unity Catalog. (ref:hirist.tech) .