Atropos Health uses Arize AX to evaluate LLM-generated clinical evidence summaries in production.

Atropos's Auto Summary tool generates clinical evidence summaries across hundreds of studies a month but the team lacked visibility into quality at scale. It built a measurement framework on Arize AX using LLM-as-a-judge evaluation. The page gives no exact figures and says the team plans to narrow to one or two key metrics.

Arize AI is an ML and LLM observability platform, used by healthcare AI companies to evaluate and monitor LLM outputs in production

This sits in the Scale & Govern stage of the AI use-case lifecycle: Named examples from the tools pharma teams use to monitor, govern and keep validating an AI system once it is already live in production.