Senior Engineer, Commodities Technology

Role This role sits in the commodities data team at Balyasny - at the intersection of the front office and central data services. The team is responsible for commodities specific data ingestion, modelling and distribution across a range of technologies and data frequencies. Data that is managed in the team is used directly by commodities trading teams; as such the role has a direct impact on trading and requires working with traders and researchers to ensure optimum performance of data systems at all times. Experience • 5+ years in data engineering including ownership of production data systems. • Has built low latency or near real-time pipelines. • Has run trade-critical systems on a formal on-call rota, including fixing live incidents under pressure. • Has supported front-office or technical users directly (traders, PMs, quants or similar), resolving issues quickly and explaining them clearly. • Has worked on long-running platform migration on containerised, cloud-native infrastructure. • Has integrated and normalised a wide range of non-standard external data sources. • Commodities or energy market exposure (power, gas, oil, shipping, agriculture) is a strong plus but not essential. Skills • Python (expert): Production services, custom data transformation, bespoke data pipelines • SQL (expert): Complex modelling and performance-aware DML and DDL across Snowflake/Clickhouse • Kubernetes and Docker • Kafka • Strong understanding of Git, code review, CI, TDD • Apache Airflow - desirable, but not essential Responsibilities • Design, build and own bespoke low latency pipelines for our most demanding data sources. • On-call rota, owning detection, triage and resolution of production data incidents. • Be a subject matter expert for PMs and quants on data issues - fixing production issues quickly and communicating technical points clearly and reliably. • Hold engineering standards across the team: review, testing, release.