Databricks Developer-(SJ-BY)

Job Description Design and develop scalable ETL/ELT pipelines using Databricks and Apache Spark. Build and maintain Data Lake and Lakehouse architectures. Develop data ingestion pipelines from multiple sources (RDBMS, APIs, Kafka, Files, Cloud Storage). Design dimensional models (Star Schema/Snowflake Schema) for reporting and analytics. Create Delta Lake tables and optimise them for performance. Develop SQL-based transformations and reporting datasets. Work with BI tools such as Power BI, Tableau, or Looker. Optimise Spark jobs for performance and scalability. Implement data quality checks and validation. Collaborate with Data Architects, Data Scientists, and Business Analysts. Manage production deployments and monitoring.