Source description
About the role
Project Role Description: Design, develop, and support scalable data platform solutions using Databricks technologies. Work closely with Data Architects and Integration teams to ensure smooth data integration, processing, and platform performance across systems. Must Have Skills: - Databricks Unified Data Analytics Platform - PySpark - Python - ETL/ELT Pipeline Development - Cloud Platforms (AWS/Azure/GCP) Good to Have: - Experience with AI/ML or Generative AI technologies - Knowledge of Big Data tools like Spark, Hadoop, Hive, or Kafka - Experience with Airflow, dbt, CI/CD, or DevOps practices Experience Required: - Minimum 5 years of experience - Pan India Location Educational Qualification: - 15 years full time education required Roles & Responsibilities: - Build scalable ETL/ELT pipelines using Databricks, Delta Lake, Auto Loader, and DLT - Develop data transformation and validation logic using PySpark and Python - Work with AWS, Azure, or GCP cloud platforms - Implement secure and optimized data pipelines - Configure Unity Catalog, data access controls, audit logging, and PII masking - Create reusable functions and data processing modules - Integrate Power BI, Tableau, or Looker with Databricks - Build dashboards and monitor data pipeline performance - Optimize jobs for performance, scalability, and cost efficiency - Support CI/CD deployment using GitHub or GitLab Project Role Description: Design, develop, and support scalable data platform solutions using Databricks technologies. Work closely with Data Architects and Integration teams to ensure smooth data integration, processing, and platform performance across systems. Must Have Skills: - Databricks Unified Data Analytics Platform - PySpark - Python - ETL/ELT Pipeline Development - Cloud Platforms (AWS/Azure/GCP) Good to Have: - Experience with AI/ML or Generative AI technologies - Knowledge of Big Data tools like Spark, Hadoop, Hive, or Kafka - Experience with Airflow, dbt, CI/CD, or DevOps practices Experience Required: - Minimum 5 years of experience - Pan India Location Educational Qualification: - 15 years full time education required Roles & Responsibilities: - Build scalable ETL/ELT pipelines using Databricks, Delta Lake, Auto Loader, and DLT - Develop data transformation and validation logic using PySpark and Python - Work with AWS, Azure, or GCP cloud platforms - Implement secure and optimized data pipelines - Configure Unity Catalog, data access controls, audit logging, and PII masking - Create reusable functions and data processing modules - Integrate Power BI, Tableau, or Looker with Databricks - Build dashboards and monitor data pipeline performance - Optimize jobs for performance, scalability, and cost efficiency - Support CI/CD deployment using GitHub or GitLab
More at Lecan Solutions