Hadoop, Spark Data Engineer

As a Hadoop, Spark Data Engineer at HSBC, you will be responsible for designing, developing, and maintaining data infrastructure solutions using cutting-edge technologies such as Hadoop and Spark. Your role will be crucial in impacting the business globally by working on complex data integration projects. If you are passionate about data and thrive in a fast-paced and collaborative environment, this opportunity is for you. Key Responsibilities: - Design and develop data infrastructure solutions using Hadoop and Spark technologies. - Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. - Create and maintain data pipelines and ETL processes for efficient data ingestion and processing. - Monitor and troubleshoot data issues and performance bottlenecks in the big data infrastructure. - Optimize and tune Hadoop and Spark clusters for maximum efficiency and performance. - Develop and maintain data governance processes to ensure data quality and integrity. - Stay up-to-date with industry trends in big data technologies for continuous improvement. - Work with data scientists and analysts to provide data sets and support for their analytical needs. - Document data processes and procedures for future reference and knowledge sharing. - Collaborate with other data engineers, participate in code reviews, and ensure best practices are followed. - Communicate effectively with stakeholders and provide regular updates on project progress. - Proactively identify and recommend solutions to improve data architecture and infrastructure. - Adhere to data security and privacy policies, ensuring compliance with regulatory requirements. - Mentor and train junior team members on Hadoop and Spark technologies and best practices. At HSBC, we are an Equal Opportunity Employer committed to creating an inclusive environment for all employees. We celebrate diversity and do not discriminate based on various legally protected characteristics. As a Hadoop, Spark Data Engineer at HSBC, you will be responsible for designing, developing, and maintaining data infrastructure solutions using cutting-edge technologies such as Hadoop and Spark. Your role will be crucial in impacting the business globally by working on complex data integration projects. If you are passionate about data and thrive in a fast-paced and collaborative environment, this opportunity is for you. Key Responsibilities: - Design and develop data infrastructure solutions using Hadoop and Spark technologies. - Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. - Create and maintain data pipelines and ETL processes for efficient data ingestion and processing. - Monitor and troubleshoot data issues and performance bottlenecks in the big data infrastructure. - Optimize and tune Hadoop and Spark clusters for maximum efficiency and performance. - Develop and maintain data governance processes to ensure data quality and integrity. - Stay up-to-date with industry trends in big data technologies for continuous improvement. - Work with data scientists and analysts to provide data sets and support for their analytical needs. - Document data processes and procedures for future reference and knowledge sharing. - Collaborate with other data engineers, participate in code reviews, and ensure best practices are followed. - Communicate effectively with stakeholders and provide regular updates on project progress. - Proactively identify and recommend solutions to improve data architecture and infrastructure. - Adhere to data security and privacy policies, ensuring compliance with regulatory requirements. - Mentor and train junior team members on Hadoop and Spark technologies and best practices. At HSBC, we are an Equal Opportunity Employer committed to creating an inclusive environment for all employees. We celebrate diversity and do not discriminate based on various legally protected characteristics.