Data Engineer - PySpark

Job Description: Job Title: Data Engineer Location: Mumbai (Onsite) Experience: 48 Years About the Role: We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines that support analytics, reporting, and AI-driven initiatives. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, cloud platforms, and performance optimization, with a focus on building reliable and cost-efficient data solutions. Key Responsibilities: - Design, develop, and maintain robust data pipelines and data integration workflows. - Build scalable ETL/ELT processes for structured and unstructured data sources. - Optimize data pipelines for performance, reliability, scalability, and cost efficiency. - Develop and maintain data models, data warehouses, and lakehouse architectures. - Ensure data quality, governance, security, and compliance across platforms. - Collaborate with Data Scientists, AI Engineers, and Business teams to operationalize data for AI/ML use cases. - Monitor, troubleshoot, and enhance data platform performance and availability. - Implement automation, orchestration, and CI/CD practices for data engineering workflows. - Support real-time and batch data processing requirements. Required Skills: - Strong experience in Data Engineering and large-scale data processing. - Proficiency in Python, SQL, and ETL/ELT development. - Hands-on experience with Apache Spark / PySpark. - Experience with cloud platforms such as AWS, Azure, or GCP. - Knowledge of Data Warehousing, Data Lakes, and Lakehouse architectures. - Experience with data orchestration tools such as Airflow, ADF, or similar. - Strong understanding of data modeling, performance tuning, and query optimization. - Experience with version control and CI/CD practices. Job Description: Job Title: Data Engineer Location: Mumbai (Onsite) Experience: 48 Years About the Role: We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines that support analytics, reporting, and AI-driven initiatives. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, cloud platforms, and performance optimization, with a focus on building reliable and cost-efficient data solutions. Key Responsibilities: - Design, develop, and maintain robust data pipelines and data integration workflows. - Build scalable ETL/ELT processes for structured and unstructured data sources. - Optimize data pipelines for performance, reliability, scalability, and cost efficiency. - Develop and maintain data models, data warehouses, and lakehouse architectures. - Ensure data quality, governance, security, and compliance across platforms. - Collaborate with Data Scientists, AI Engineers, and Business teams to operationalize data for AI/ML use cases. - Monitor, troubleshoot, and enhance data platform performance and availability. - Implement automation, orchestration, and CI/CD practices for data engineering workflows. - Support real-time and batch data processing requirements. Required Skills: - Strong experience in Data Engineering and large-scale data processing. - Proficiency in Python, SQL, and ETL/ELT development. - Hands-on experience with Apache Spark / PySpark. - Experience with cloud platforms such as AWS, Azure, or GCP. - Knowledge of Data Warehousing, Data Lakes, and Lakehouse architectures. - Experience with data orchestration tools such as Airflow, ADF, or similar. - Strong understanding of data modeling, performance tuning, and query optimization. - Experience with version control and CI/CD practices.