Software Engineer PySpark

Join us as a Software Engineer, PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutionsIts a chance to hone your existing technical skills and advance your career while building a wide network of stakeholdersWe're offering this role at associate levelWhat you'll doIn your new role, youll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. Youll also be: Producing complex and critical software rapidly and of high quality which adds value to the businessWorking in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioningCollaborating to optimise our software engineering capabilityDesigning, producing, testing and implementing our working software solutionsWorking across the life cycle, from requirements analysis and design, through coding to testing, deployment and operationsThe skills you'll needTo take on this role, youll need a background in software engineering, software design, and architecture, and an understanding of how your area of expertise supports our customers. Youll need at least six years of experience in PySpark, data engineering with strong expertise in ETL design, data quality testing, cleansing, monitoring, sourcing and exploration. You'll need experience working with AWS services such as Glue, Lambda, EMR or Spark and S3. You'll also need programming proficiency in Python, Spark, and SQL and demonstrate working knowledge of Large Language Models or LLMs and AI-assisted development tools like GitHub Copilot, GitLab Duo along with practical prompt engineering skills. Youll also need: Experience of working with development and testing tools, bug tracking tools and wikisExperience in multiple programming languages or low code toolsetsExperience of DevOps and Agile methodology and associated toolsetsA background in solving highly complex, analytical and numerical problemsExperience of implementing programming best practice, especially around scalability, automation, virtualisation, optimisation, availability and performance .