Data Engineer - Azure Databricks

Experience:- 5 to 10 years of experience in Data Engineering.Key Skills Required:- Strong hands-on experience with Databricks + Streaming.- PySpark.- SQL.- Delta Lake.- Unity Catalog.- Databricks Workflows / Jobs orchestration.- Experience with Azure Data Factory (ADF).- Experience building Lakehouse architectures using Bronze, Silver & Gold layers.- Good knowledge of Databricks Lakeflow:1. Declarative Pipelines.2. Lakeflow Connect.3. CDC (Change Data Capture).4. Data quality validations.- Experience with AI/BI and Genie:1. Semantic layers.2. Certified datasets.3. Self-service analytics solutions.- Experience building AI Agents/Copilots using:1. LLMs.2. RAG (Retrieval-Augmented Generation).3. Vector Search.4. Prompt Engineering.- Good understanding of Unity Catalog governance:1. Access controls.2. Data masking.3. Audit logging.4. Lineage.- Experience with CI/CD, Git, Databricks Repos & Asset Bundles.Expected Responsibilities:- Design, develop, and support end-to-end data pipelines in Databricks.- Build and maintain Delta tables under Unity Catalog.- Develop Lakeflow pipelines with incremental processing and CDC.- Enable AI/BI use cases through semantic models and Genie spaces.- Integrate SharePoint and external data sources into the Lakehouse.- Ensure data quality, governance, and operational reliability.- Drive delivery independently and proactively resolve issues. (ref:hirist.tech) Experience:- 5 to 10 years of experience in Data Engineering.Key Skills Required:- Strong hands-on experience with Databricks + Streaming.- PySpark.- SQL.- Delta Lake.- Unity Catalog.- Databricks Workflows / Jobs orchestration.- Experience with Azure Data Factory (ADF).- Experience building Lakehouse architectures using Bronze, Silver & Gold layers.- Good knowledge of Databricks Lakeflow:1. Declarative Pipelines.2. Lakeflow Connect.3. CDC (Change Data Capture).4. Data quality validations.- Experience with AI/BI and Genie:1. Semantic layers.2. Certified datasets.3. Self-service analytics solutions.- Experience building AI Agents/Copilots using:1. LLMs.2. RAG (Retrieval-Augmented Generation).3. Vector Search.4. Prompt Engineering.- Good understanding of Unity Catalog governance:1. Access controls.2. Data masking.3. Audit logging.4. Lineage.- Experience with CI/CD, Git, Databricks Repos & Asset Bundles.Expected Responsibilities:- Design, develop, and support end-to-end data pipelines in Databricks.- Build and maintain Delta tables under Unity Catalog.- Develop Lakeflow pipelines with incremental processing and CDC.- Enable AI/BI use cases through semantic models and Genie spaces.- Integrate SharePoint and external data sources into the Lakehouse.- Ensure data quality, governance, and operational reliability.- Drive delivery independently and proactively resolve issues. (ref:hirist.tech)