Sr. Data Engineer

The senior data engineer will build and maintain data lake and data warehousing solutions for a B2B multi-tenant SaaS solution. The position requires a demonstrated ability to design processes from end to end and a strong technical understanding of modern data pipelines and strategies. In-depth knowledge of multiple data sources, data design patterns, ETL transformation, SQL and performance tuning are required. Responsibilities: In order of significance, list the essential functions and responsibilities of the position.Participate in the entire data pipeline lifecycle, focusing on building data lakes, data warehouses and related processes.Lead code and design reviews with junior and peer developers.Write clean code to develop data pipelines and necessary infrastructure as code.Collaborate with cross-functional teams to understand requirements and effectively design appropriate solutions.Monitor and troubleshoot performance, scalability, and security issues, making improvements as needed.Participate in the evaluation of new technologies and tools that can improve the performance of our system.Participate in the data governance process: Contribute to the design and documentation of our data lake platform. Proficient in test automation solutions for data and able to optimise performance.Gather technical and design requirements: Communicate effectively with technical and non-technical team members to ensure alignment and successful execution of projects and continuous improvement. Participate in an on-call rotation. Requirements: List applicable knowledge, skills, experience, certificates, licenses, and other criteria required or preferred for this position. (e. g., Management experience required.Experience in the development of modern data architecture, analytics, data governance, AI/ML, or related areas.Experience with AWS big data technologies, including EMR, Glue, S3 EKS, Lambda, Athena, RDS, MKS/Kafka/Kinesis.In-depth knowledge of the entire data engineering process (design, development, deployment).Exposure to relational and NoSQL databases such as PostgreSQL, Cassandra, DynamoDB, MongoDB, etc., and data modelling principles.Experience with DevOps, including CI/CD pipelines, containerised deployment/Kubernetes, and infrastructure-as-code/AWS CloudFormation/Terraform.Proficient in SQL (SQL Server, Postgres) and Python.Identify opportunities for process improvement as well as efficiency in solutions.Teamwork skills with a problem-solving attitude and a willingness to take a variety of approaches.Strong experience with legacy data warehouses and ETL pipelines.Demonstrated ability to mentor junior team members.Excellent analytical and time management skills, with a proven ability to deliver value independently.Strong written and verbal communication skills, with demonstrated experience providing technical input to technical and non-technical stakeholders.Ability to participate in on-call rotation. Preferred: Experience with multi-tenant, B2B2C SaaS offerings.FinTech industry experience.Understanding of MPP Data Warehouse (Redshift, Greenplum). The senior data engineer will build and maintain data lake and data warehousing solutions for a B2B multi-tenant SaaS solution. The position requires a demonstrated ability to design processes from end to end and a strong technical understanding of modern data pipelines and strategies. In-depth knowledge of multiple data sources, data design patterns, ETL transformation, SQL and performance tuning are required. Responsibilities: In order of significance, list the essential functions and responsibilities of the position.Participate in the entire data pipeline lifecycle, focusing on building data lakes, data warehouses and related processes.Lead code and design reviews with junior and peer developers.Write clean code to develop data pipelines and necessary infrastructure as code.Collaborate with cross-functional teams to understand requirements and effectively design appropriate solutions.Monitor and troubleshoot performance, scalability, and security issues, making improvements as needed.Participate in the evaluation of new technologies and tools that can improve the performance of our system.Participate in the data governance process: Contribute to the design and documentation of our data lake platform. Proficient in test automation solutions for data and able to optimise performance.Gather technical and design requirements: Communicate effectively with technical and non-technical team members to ensure alignment and successful execution of projects and continuous improvement. Participate in an on-call rotation. Requirements: List applicable knowledge, skills, experience, certificates, licenses, and other criteria required or preferred for this position. (e. g., Management experience required.Experience in the development of modern data architecture, analytics, data governance, AI/ML, or related areas.Experience with AWS big data technologies, in