Azure Databricks Developer

Role Overview: As a Senior Data Engineer at Birlasoft, you will utilize your expertise in Databricks, PySpark, and Data Modeling to lead the design and development of scalable data solutions on Azure. Your role will involve designing, developing, and deploying robust data engineering solutions, building and maintaining efficient data pipelines, leading data modeling efforts, and collaborating with various teams to deliver high-quality solutions. Key Responsibilities: - Design, develop, and deploy robust data engineering solutions using Azure Databricks, PySpark, SQL, and Azure Data Factory (ADF). - Build and maintain scalable and efficient data pipelines to support growing data volume and complexity. - Lead data modeling efforts including conceptual, logical, and physical models for structured and semi-structured data. - Develop Modern Data Warehouse solutions leveraging Azure Data Lake, Delta Lake, and Databricks. - Implement batch and streaming data processing solutions for structured and unstructured data sources. - Create and maintain technical design documents, including application interface designs and data flow diagrams. - Collaborate with BI, analytics, and business teams to understand data requirements and deliver high-quality solutions. - Optimize data workflows and ensure data quality, consistency, and governance across platforms. Qualifications Required: - 23 years of hands-on experience with Azure Databricks in enterprise-scale projects. - 2+ years of experience in PySpark for data transformation and processing. - 23 years of experience in Azure Data Factory (ADF) for orchestration and integration. - Strong proficiency in SQL for data querying and manipulation. - Proven experience in data modeling (dimensional modeling, normalization, star/snowflake schema). - Solid understanding of Delta Lake architecture, data partitioning, and performance tuning. - Experience working with structured and semi-structured data (e.g., JSON, Parquet, Avro). - Familiarity with CI/CD pipelines, version control (Git), and agile methodologies is a plus. Role Overview: As a Senior Data Engineer at Birlasoft, you will utilize your expertise in Databricks, PySpark, and Data Modeling to lead the design and development of scalable data solutions on Azure. Your role will involve designing, developing, and deploying robust data engineering solutions, building and maintaining efficient data pipelines, leading data modeling efforts, and collaborating with various teams to deliver high-quality solutions. Key Responsibilities: - Design, develop, and deploy robust data engineering solutions using Azure Databricks, PySpark, SQL, and Azure Data Factory (ADF). - Build and maintain scalable and efficient data pipelines to support growing data volume and complexity. - Lead data modeling efforts including conceptual, logical, and physical models for structured and semi-structured data. - Develop Modern Data Warehouse solutions leveraging Azure Data Lake, Delta Lake, and Databricks. - Implement batch and streaming data processing solutions for structured and unstructured data sources. - Create and maintain technical design documents, including application interface designs and data flow diagrams. - Collaborate with BI, analytics, and business teams to understand data requirements and deliver high-quality solutions. - Optimize data workflows and ensure data quality, consistency, and governance across platforms. Qualifications Required: - 23 years of hands-on experience with Azure Databricks in enterprise-scale projects. - 2+ years of experience in PySpark for data transformation and processing. - 23 years of experience in Azure Data Factory (ADF) for orchestration and integration. - Strong proficiency in SQL for data querying and manipulation. - Proven experience in data modeling (dimensional modeling, normalization, star/snowflake schema). - Solid understanding of Delta Lake architecture, data partitioning, and performance tuning. - Experience working with structured and semi-structured data (e.g., JSON, Parquet, Avro). - Familiarity with CI/CD pipelines, version control (Git), and agile methodologies is a plus.