Azure Data Engineer - Databricks/PySpark

As an Azure Databricks Developer, you will be responsible for designing and developing scalable data pipelines using Azure Databricks. Your role will involve working with large datasets using PySpark for data transformation and processing. You will also be required to implement data storage and retrieval solutions using Azure Data Lake Storage (ADLS) and optimize data processing jobs for performance and efficiency. Collaboration with data architects and stakeholders to define data solutions, integration of data pipelines with Azure services and external systems, troubleshooting and resolving data-related issues in production environments, and following best practices for coding, testing, and deployment are also key responsibilities. Key Responsibilities: - Design and develop scalable data pipelines using Azure Databricks - Work with large datasets using PySpark for data transformation and processing - Implement data storage and retrieval solutions using Azure Data Lake Storage (ADLS) - Optimize and tune data processing jobs for performance and efficiency - Collaborate with data architects and stakeholders to define data solutions - Integrate data pipelines with Azure services and external systems - Troubleshoot and resolve data-related issues in production environments - Follow best practices for coding, testing, and deployment Qualifications Required: - Strong hands-on experience with Azure Databricks (Python-based development) - Proficiency in PySpark - Experience with Azure Data Lake Storage (ADLS) - Previous experience in building and optimizing big data pipelines - Strong analytical and problem-solving skills - Knowledge of data processing frameworks and distributed systems (Note: Company details are not provided in the job description.) As an Azure Databricks Developer, you will be responsible for designing and developing scalable data pipelines using Azure Databricks. Your role will involve working with large datasets using PySpark for data transformation and processing. You will also be required to implement data storage and retrieval solutions using Azure Data Lake Storage (ADLS) and optimize data processing jobs for performance and efficiency. Collaboration with data architects and stakeholders to define data solutions, integration of data pipelines with Azure services and external systems, troubleshooting and resolving data-related issues in production environments, and following best practices for coding, testing, and deployment are also key responsibilities. Key Responsibilities: - Design and develop scalable data pipelines using Azure Databricks - Work with large datasets using PySpark for data transformation and processing - Implement data storage and retrieval solutions using Azure Data Lake Storage (ADLS) - Optimize and tune data processing jobs for performance and efficiency - Collaborate with data architects and stakeholders to define data solutions - Integrate data pipelines with Azure services and external systems - Troubleshoot and resolve data-related issues in production environments - Follow best practices for coding, testing, and deployment Qualifications Required: - Strong hands-on experience with Azure Databricks (Python-based development) - Proficiency in PySpark - Experience with Azure Data Lake Storage (ADLS) - Previous experience in building and optimizing big data pipelines - Strong analytical and problem-solving skills - Knowledge of data processing frameworks and distributed systems (Note: Company details are not provided in the job description.)