Source description
About the role
Job Description Position Senior Associate Projects (Big Data Engineer) Experience 6 10 Years Location Chennai (Hybrid) Shift Day Shift Job Summary We are seeking a highly skilled Big Data Engineer with expertise in Hadoop, Spark, Python, PL/SQL, and ETL Tools to design, develop, and optimize large-scale data processing solutions. The ideal candidate will be responsible for building scalable data pipelines, ensuring data quality, and enabling data-driven decision-making across the organization. The role requires hands-on experience in big data technologies, data integration, data transformation, and performance optimization within enterprise environments. Mandatory Skills - Hadoop - Apache Spark - Python - PL/SQL - ETL Tools - Data Gap Analysis - Data Integration - SQL Key Responsibilities - Design, develop, and maintain scalable data pipelines using Hadoop and Spark. - Build and optimize ETL workflows to extract, transform, and load data from multiple sources. - Perform data gap analysis and implement solutions to improve data quality and consistency. - Develop complex PL/SQL procedures, functions, packages, and performance-tuned queries. - Use Python to automate data processing, workflow orchestration, and reporting activities. - Monitor, troubleshoot, and optimize big data applications and ETL jobs. - Collaborate with business stakeholders and technical teams to gather requirements and deliver data solutions. - Ensure data integrity, security, and compliance with organizational standards. - Participate in code reviews and implement best practices for scalable data engineering solutions. - Improve system performance through tuning, optimization, and process enhancements. - Document technical designs, data flows, and operational procedures. Required Qualifications - 610 years of experience in Data Engineering or Big Data Development. - Robust hands-on experience with Hadoop ecosystem components. - Expertise in Apache Spark for large-scale data proce .
More at Cognizant