Source description
About the role
Data Engineer (Databricks) Experience Level-5 - 8 Years Gurugram, Bengaluru, Hyderabad Valuable To Have Skills Change Management tool, DevOps Must Have Skills Pyspark, Azure, ADF, Databricks, ETL, SQL Job Description - Design and build data pipelines using Spark-SQL and PySpark in Azure Databricks - Design and build ETL pipelines using ADF - Build and maintain a Lakehouse architecture in ADLS / Databricks. - Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc. - Work with DevOps team to deploy solutions in production environments. - Control data processes and take corrective action when errors are identified. - Corrective action may include executing a work around process and then identifying the cause and solution for data errors. - Participate as a full member of the global Analytics team, providing solutions for and insights into data related items. - Collaborate with your Data Science and Business Intelligence colleagues across the world to share key learnings, leverage ideas and solutions and to propagate best practices. - You will lead projects that include other team members and participate in projects led by other team members. - Apply change management tools including training, communication and documentation to manage upgrades, changes and data migrations. Must-Have Skills: Azure Databricks, Azure Data Factory, PySpark. Spark - SQL ADLS .
More at PradeepIT Consulting Services