From stalled pilot to $43M annual ROI and 95% accuracy

A Top 5 global telecom tried to turn an internal billing co-pilot into a customer-facing chatbot, but the project stalled at 54% accuracy due to data blind spots and reasoning errors. Snorkel applied its data-development framework to embed subject-matter expertise into the GenAI application, building rigorous evaluation workflows and using human-in-the-loop data acceleration to handle edge cases. Accuracy rose from 54% to 95%, the system was rolled out to 80,000 daily customers globally, and 90% of complex billing cases were resolved in under 24 hours. The company reports $43 million in annual ROI from the deployment.

The customer is anonymised in Snorkel AI’s own case study — not withheld by us.

Snorkel AI is a data-development and evaluation company whose programmatic labeling tools build training and benchmark datasets used to stress-test whether a model approach is viable before a full build.

This sits in the Design & Stress-Test stage of the AI use-case lifecycle: Named examples from the tools pharma teams use to scope, prototype and benchmark an AI use case before committing to build it — data-catalog, sandbox and evaluation platforms.