Data Engineer

Role Overview: You will be responsible for developing and maintaining ETL/ELT data pipelines and data workflows. This includes writing optimized SQL queries, performing data cleansing and validation, and working with Python, PySpark, and Databricks for data processing and automation. Additionally, you will support AI/ML data preparation, collaborate with Data Science and Analytics teams, and handle structured and unstructured data from multiple sources. Monitoring and troubleshooting data pipeline issues, supporting dashboarding and reporting, and developing processes for data scraping using RPA will also be part of your responsibilities. Key Responsibilities: - Develop and maintain ETL/ELT data pipelines and data workflows. - Write optimized SQL queries for data extraction, transformation, and analysis. - Perform data cleansing, validation, and quality checks on large datasets. - Work with Python, PySpark, and Databricks for data processing and automation. - Support AI/ML data preparation and model input pipelines. - Handle structured and unstructured data from multiple cloud and database sources. - Collaborate with Data Science and Analytics teams for AI/ML initiatives. - Monitor and troubleshoot data pipeline, workflow, and processing issues. - Support dashboarding, reporting, data preparation, data mapping, and automation requirements. - Develop processes to scrape data using RPA (UI-PATH). Qualifications Required: - Bachelor's degree in Computer Science, IT, Engineering, Artificial Intelligence, Data Science, etc. - Strong hands-on experience in SQL and Python. - Knowledge of PySpark/Spark and Databricks. - Understanding of Artificial Intelligence (AI) and Machine Learning (ML) concepts. - Experience with GCP BigQuery, Elastic Search, etc. - Knowledge of ETL processes, data warehousing, and data modeling. - Familiarity with APIs, CSV/JSON handling, and automation techniques. - Good analytical, problem-solving, and debugging skills. - Knowledge of UI-PATH. Note: Preference will be given to candidates who can join immediately or within 15 days. Role Overview: You will be responsible for developing and maintaining ETL/ELT data pipelines and data workflows. This includes writing optimized SQL queries, performing data cleansing and validation, and working with Python, PySpark, and Databricks for data processing and automation. Additionally, you will support AI/ML data preparation, collaborate with Data Science and Analytics teams, and handle structured and unstructured data from multiple sources. Monitoring and troubleshooting data pipeline issues, supporting dashboarding and reporting, and developing processes for data scraping using RPA will also be part of your responsibilities. Key Responsibilities: - Develop and maintain ETL/ELT data pipelines and data workflows. - Write optimized SQL queries for data extraction, transformation, and analysis. - Perform data cleansing, validation, and quality checks on large datasets. - Work with Python, PySpark, and Databricks for data processing and automation. - Support AI/ML data preparation and model input pipelines. - Handle structured and unstructured data from multiple cloud and database sources. - Collaborate with Data Science and Analytics teams for AI/ML initiatives. - Monitor and troubleshoot data pipeline, workflow, and processing issues. - Support dashboarding, reporting, data preparation, data mapping, and automation requirements. - Develop processes to scrape data using RPA (UI-PATH). Qualifications Required: - Bachelor's degree in Computer Science, IT, Engineering, Artificial Intelligence, Data Science, etc. - Strong hands-on experience in SQL and Python. - Knowledge of PySpark/Spark and Databricks. - Understanding of Artificial Intelligence (AI) and Machine Learning (ML) concepts. - Experience with GCP BigQuery, Elastic Search, etc. - Knowledge of ETL processes, data warehousing, and data modeling. - Familiarity with APIs, CSV/JSON handling, and automation techniques. - Good analytical, problem-solving, and debugging skills. - Knowledge of UI-PATH. Note: Preference will be given to candidates who can join immediately or within 15 days.