How Rox achieved 99% accuracy with Snorkel

Rox, an AI-powered sales platform, needed outbound emails to be accurate and brand-aligned but found off-the-shelf evaluation insufficient, and lacked confidence in its own LLM-as-a-judge prototype. Using the Snorkel Evaluation Suite to benchmark the judge against human experts, Rox found it aligned with humans only about 75% of the time, and iterating with Snorkel uncovered that the outbound model was using the wrong recipient name in roughly 11% of cases. After fixing this, Rox reached 99%+ accuracy with specialized evaluators and a +24 point improvement in the shipped critical outbound-email feature, giving it enough confidence to deploy; a co-founder said "the level of visibility and control Snorkel delivers is a huge advantage as we build trustworthy, agentic AI at scale."

Snorkel AI is a data-development and evaluation company whose programmatic labeling tools build training and benchmark datasets used to stress-test whether a model approach is viable before a full build.

This sits in the Design & Stress-Test stage of the AI use-case lifecycle: Named examples from the tools pharma teams use to scope, prototype and benchmark an AI use case before committing to build it — data-catalog, sandbox and evaluation platforms.