Data Engineer - PySpark

Role Overview: As a Data Engineer at our company in Mumbai, you will be responsible for designing, building, and maintaining scalable data pipelines to support analytics, reporting, and AI-driven initiatives. Your expertise in data engineering, ETL/ELT processes, cloud platforms, and performance optimization will be crucial in developing reliable and cost-efficient data solutions. Key Responsibilities: - Design, develop, and maintain robust data pipelines and data integration workflows. - Build scalable ETL/ELT processes for structured and unstructured data sources. - Optimize data pipelines for performance, reliability, scalability, and cost efficiency. - Develop and maintain data models, data warehouses, and lakehouse architectures. - Ensure data quality, governance, security, and compliance across platforms. - Collaborate with Data Scientists, AI Engineers, and Business teams to operationalize data for AI/ML use cases. - Monitor, troubleshoot, and enhance data platform performance and availability. - Implement automation, orchestration, and CI/CD practices for data engineering workflows. - Support real-time and batch data processing requirements. Qualification Required: - Strong experience in Data Engineering and large-scale data processing. - Proficiency in Python, SQL, and ETL/ELT development. - Hands-on experience with Apache Spark / PySpark. - Experience with cloud platforms such as AWS, Azure, or GCP. - Knowledge of Data Warehousing, Data Lakes, and Lakehouse architectures. - Experience with data orchestration tools such as Airflow, ADF, or similar. - Strong understanding of data modeling, performance tuning, and query optimization. - Experience with version control and CI/CD practices. Additional Details: The ideal candidate for this role should have at least 4-8 years of experience in data engineering and a proven track record of successful project implementations in a similar capacity. The company values innovation, collaboration, and continuous learning, providing an environment where you can grow both professionally and personally. Join our team to be a part of exciting projects that leverage cutting-edge technologies to drive business growth. Role Overview: As a Data Engineer at our company in Mumbai, you will be responsible for designing, building, and maintaining scalable data pipelines to support analytics, reporting, and AI-driven initiatives. Your expertise in data engineering, ETL/ELT processes, cloud platforms, and performance optimization will be crucial in developing reliable and cost-efficient data solutions. Key Responsibilities: - Design, develop, and maintain robust data pipelines and data integration workflows. - Build scalable ETL/ELT processes for structured and unstructured data sources. - Optimize data pipelines for performance, reliability, scalability, and cost efficiency. - Develop and maintain data models, data warehouses, and lakehouse architectures. - Ensure data quality, governance, security, and compliance across platforms. - Collaborate with Data Scientists, AI Engineers, and Business teams to operationalize data for AI/ML use cases. - Monitor, troubleshoot, and enhance data platform performance and availability. - Implement automation, orchestration, and CI/CD practices for data engineering workflows. - Support real-time and batch data processing requirements. Qualification Required: - Strong experience in Data Engineering and large-scale data processing. - Proficiency in Python, SQL, and ETL/ELT development. - Hands-on experience with Apache Spark / PySpark. - Experience with cloud platforms such as AWS, Azure, or GCP. - Knowledge of Data Warehousing, Data Lakes, and Lakehouse architectures. - Experience with data orchestration tools such as Airflow, ADF, or similar. - Strong understanding of data modeling, performance tuning, and query optimization. - Experience with version control and CI/CD practices. Additional Details: The ideal candidate for this role should have at least 4-8 years of experience in data engineering and a proven track record of successful project implementations in a similar capacity. The company values innovation, collaboration, and continuous learning, providing an environment where you can grow both professionally and personally. Join our team to be a part of exciting projects that leverage cutting-edge technologies to drive business growth.