UX Data Scientist

The Opportunity

As a Senior Data Scientist on the Design team, you will play a pivotal role in transforming how some of the world's most critical infrastructure companies operate. The experiences you shape for both humans and agents will directly influence how complex projects are planned, deployed, and managed at scale.

We are rethinking what excellent UX means in an agentic world. When the interface is a model, the quality of the experience is no longer carried primarily by pixels and flows — it is carried by behavior. What the agent retrieves, what it decides, what it says, when it refuses, and how reliably it does all of that across thousands of real situations. Those are measurement problems, experimentation problems, and evaluation problems. They are your problems.

This role is ideal for someone who combines statistical rigor with a strong builder mentality — someone energized by ambiguity, fluent in model behavior, and obsessed with what "good" actually means and how to prove it. You will set the standard for how agent experiences are designed, instrumented, and held to a quality bar at Sitetracker — hands-on, technically deep, and relentlessly connected to customer outcomes.

You will be embedded in a scrum team — designing, measuring, and delivering alongside Engineering and Design on a day-to-day basis. You will own day-to-day decisions for your experience area, mentor others on rigor and method, and ship to production. For someone who wants to build, measure, and elevate the craft around them, this is the role.

The Skills You'll Have

Act with Agency

Thrive in fast-moving environments and deliver high-impact outcomes at pace

Independently identify ambiguous challenges and take ownership of driving them to resolution

Champion new ideas, influence stakeholders, and rally support around strategic initiatives

Identify workflows where AI tools and emerging technologies accelerate delivery, and make building the default way you work

Measurement, Experimentation, and Evaluation

Define what "good" means for an agentic experience and turn it into metrics, evals, and datasets that hold up under scrutiny

Design and analyze experiments — A/B tests, holdouts, shadow runs — with correct sizing, clean reads, and honest conclusions

Build offline and online evaluation systems for LLM behavior across correctness, grounding, safety, tone, and latency

Apply statistical and ML fluency to real, messy production behavior — distinguishing signal from noise and root cause from symptom

Agentic UX and Behavior Design

Turn complex enterprise workflows into agent experiences people can actually use without training

Design context architecture, decision paths, and communication patterns, and reason about how they shape behavior across the full input distribution

Work directly with customers — running pilots, mining usage, and iterating based on how they actually behave, including customers with complex needs or in delicate situations

Make tradeoff decisions across user needs, agent behavior, technical constraints, and business goals, and connect those decisions to commercial outcomes

Competitive & Market Awareness

Maintain running awareness of the competitive landscape across multiple verticals

Identify gaps and opportunities and produce analysis that connects to product and behavior decisions

Track market trends, adjacent products, and emerging tools, connecting what's happening in the market to what Sitetracker should be doing

Regularly contribute to competition and industry Slack channels with substantive observations