Query Economics, Not Model Accuracy, Decides Clinical Data Management ROI

A Phase III study generates roughly 97,000 data queries. That number is not a vendor estimate: it comes from a 2021 study in the Journal of the Society for Clinical Data Management in which seven pharmaceutical companies and a vendor pooled 20 Phase III studies covering 20,125 participants and 49.3 million entered data points. Those studies produced nearly two million queries, an average of 96,980 each. Queries were raised against 3.9% of entered data. They resulted in database changes affecting 1.7% of it. The authors' conclusion was blunt: end the generalised use of automated and manual queries against non-critical form types and fields.

Take the same paper's own cost statement — "the cost of queries is not known precisely, with estimates varying from $28 to $225 per query" — and multiply. One Phase III study's query process costs between $2.7m and $21.8m to remove errors from 1.7% of the data. That is the baseline any automation business case is measured against, and it is where the interesting decisions live.

They are not model decisions. On published API prices as of 30 August 2026, the inference cost of having a language model triage one reconciliation discrepancy sits between about one cent and forty cents depending on model and token budget. Against a $28 manual floor, the entire span from the cheapest small model doing a single pass to a frontier model running an agentic loop moves total cost by roughly one percentage point. The decision that moves the number by an order of magnitude more is how much human quality control you keep on top of the machine's output — and, critically, whether ICH E6(R3) lets you move it. It does, and it names the clause.

What the 32% cycle-time number actually says

The statistic everyone cites is real, and it is older than it sounds. The Tufts-eClinical Solutions Data Strategies and Transformation Study found that "since 2017, large pharmaceutical companies are seeing a 32 percent increase in the time from the last patient last visit (LPLV) to database lock cycle time metric". The study was fielded in 2019 and released on 23 April 2020. The comparator is a 2017 study. Overall cycle times rose by six days over two years for companies using five or more data sources, and companies with a formal data strategy locked at 36.1 days against 41.8 days for those without.

Quote it precisely or not at all: a 2019 finding about large pharma, benchmarked to 2017, with no public refresh of the metric from Tufts in the six years since. The mechanism it describes has only intensified. Tufts CSDD and 15 TransCelerate member companies analysed 105 protocols and reported in Therapeutic Innovation and Regulatory Science (Getz et al., 2025) that a Phase III protocol now collects 5.9 million datapoints on average, growing 11% annually since 2020, and that non-core procedures make up 16.2% of Phase III procedures. Nearly a third of all procedures and datapoints per protocol are classified as non-core or non-essential.

More sources, more datapoints, more of them arriving from outside the electronic data capture system. The cycle time did not lengthen because data entry got slower. It lengthened because the number of things that have to agree with each other went up.

Where the money actually sits: EDC versus everything else

Veeva's 2025 clinical data industry research, a survey of 88 data managers and CRAs with follow-up interviews, puts numbers on the split. An EDC query takes 9.6 days to resolve. A non-EDC query takes 16.4 days — 71% longer. Respondents reported spending 34% more active time on a non-EDC query, because the external data arrives late and because resolving it means going back to the vendor.

The structural finding is more damning than the time one. Ninety-seven per cent of data managers said manual data reconciliation happens either outside their core clinical systems or across a mix of systems; only 3% do it inside a single system. Fifty-two per cent named manual reconciliation as their single most inefficient task. Data managers spend 18% of their time on data review and cleaning, and 12 hours per study per week on reconciliation and cleaning combined.

Two cautions on those cost figures. First, the narrower $28–$71 range that circulates widely traces through secondary write-ups to Oracle ClearTrial benchmarking by therapeutic area — $28 endocrine, $32 respiratory, $64–$71 oncology — not to a peer-reviewed source. The JSCDM authors, who had the underlying query data, declined to be that precise. Second, Veeva's own $32.50 lower bound does not reproduce from a 34% uplift on $28, which would give $37.50. Rebuild the arithmetic on your own volumes before you take any of it into a business case.

Why model choice barely moves the business case

Define the unit of work: reconcile one discrepancy between an external vendor record and the EDC record, apply the protocol's reconciliation rules, decide whether it is genuine, and draft the query with its supporting evidence.

Two token budgets, both assumptions rather than measurements. A single-pass call: 8,000 input tokens covering the record pair, the reconciliation rules and prior query history, and 800 output tokens. An agentic run with retrieval, tool calls and a self-check pass: 60,000 input and 4,000 output. Prices are Anthropic's published list rates on 30 August 2026 — Haiku 4.5 at $1/$5 per million tokens, Sonnet 5 at $2/$10, Opus 5 at $5/$25. The Batch API halves both sides.

The whole table spans a factor of 33 in cost and lands entirely between 0.04% and 1.43% of the cheapest manual query. Against the $300 upper bound for a third-party query it is between 0.004% and 0.13%. Be ten times more generous with tokens than these assumptions and the most expensive configuration still sits in low single digits of the manual cost. Model selection is a latency and quality decision. It is not a cost decision, and any business case that presents it as one is measuring the wrong variable.

The decision that does move it

Here is the variable that matters. Assume the model proposes a disposition on every discrepancy and a qualified human reviews it. Assume — and this is the assumption to pressure-test with a time-and-motion study before you commit to it — that reviewing a fully prepared disposition takes 25% of the time a data manager spends resolving the query unaided. Use $50 per query as the manual baseline, the midpoint figure Oracle used in its own illustration of a 6,000-query trial costing $200,000.

The third row assumes 30% of queries touch critical data — safety, primary endpoint, eligibility — which are reviewed in full, and a 20% random sample of the remaining 70%, giving a 44% review fraction.

Moving from row two to row three saves $7.00 per query. Switching from Opus 5 agentic to Haiku 4.5 single-pass saves $0.388 per query. The quality control design decision is worth about 18 times the model decision. On the JSCDM average of 96,980 queries per Phase III study, that is roughly $679,000 against roughly $38,000.

This is why so many clinical data AI pilots report a disappointing number. They ran a careful model bake-off, kept 100% human review because nobody would sign anything else, and landed on row two. Row two is a real 75% saving and worth having. But the pilot that stops there has optimised the variable worth $38,000 and left the one worth $679,000 untouched.

What the regulation permits, and what it does not

Row three is a regulatory decision before it is an operational one, and the relevant text is more permissive than most quality functions assume.

Binding in the EU today. ICH E6(R3) Principles and Annex 1 have been effective in the EU since 23 July 2025 (EMA). Annex 1 section 3.16.1(b) states that the sponsor "should focus their quality assurance and quality control activities, including data review, on data of higher criticality and relevant metadata". Section 4.2.1(a) says that where data are transcribed into a computerised system, "the need for and the extent of data verification should take the criticality of the data into account". Section 4.2.1(c) says automated validation checks that raise queries "should be considered as required based on risk". Section 4.2.3 requires that review of data and metadata be "a planned activity" whose "extent and nature should be risk-based". Section 4.2.6(a) confines rectification to errors and omissions "that have a meaningful impact on the safety of trial participants and/or the reliability of the trial results". Principle 7.4 tells sponsors to avoid "unnecessary complexity, procedures and data collection".

Read together, those clauses do not merely tolerate risk-weighted QC sampling. They direct it. A sponsor running 100% manual review of every non-critical query is arguably further from E6(R3) than one running a documented 20% sample with a defined critical-data carve-out.

Not yet in force. ICH E6(R3) Annex 2, covering decentralised and other non-traditional designs, reached Step 4 on 3 June 2026 and was adopted by CHMP on 25 June 2026 as EMA/CHMP/ICH/162879/2026, with a legal effective date of 15 January 2027. If your reconciliation load comes from decentralised data flows, Annex 2 is a planning input for the next four and a half months, not a compliance obligation.

The system itself. For the AI component, the applicable EU text is the EMA Guideline on computerised systems and electronic data in clinical trials (EMA/INS/GCP/112288/2023), adopted 7 March 2023 and in effect since 9 September 2023, which sets ALCOA++ expectations and explicitly covers cloud solutions. Add GAMP 5 Second Edition and the ISPE GAMP Guide: Artificial Intelligence, published 29 July 2025.

Not applicable, whatever the conference talk said. EU GMP Annex 22 on artificial intelligence is a draft, published for consultation on 7 July 2025 alongside a draft revised Annex 11 and Chapter 4. The consultation closed on 7 October 2025 and, as of 30 August 2026, no final text has been adopted; the EMA inspectors' working group work plan targets Q4 2026 for delivering a final text to the Commission, with no operative date announced. The binding EU computerised-systems text remains the 2011 Annex 11. And both are GMP instruments: a clinical data management system falls under the GCP stack above, not under Annex 11 or Annex 22. Separately, FDA's Computer Software Assurance guidance, finalised 24 September 2025, is scoped to medical-device production and quality-system software under 21 CFR Part 820, now the QMSR — it does not govern a drug sponsor's GCP system.

AI Act. The EU AI Act was amended by Regulation (EU) 2026/1744 of 8 July 2026, the Digital Omnibus on AI, published in the Official Journal on 24 July 2026 and in force since 27 July 2026. It moved Annex III standalone high-risk obligations to 2 December 2027 and Annex I product-embedded ones to 2 August 2028. Live now and unchanged by the omnibus: the Article 5 prohibitions and the Article 4 AI literacy duty since 2 February 2025, GPAI obligations since 2 August 2025, and Article 50 transparency since 2 August 2026. A query-triage agent is very unlikely to be Annex III high-risk. The Article 4 literacy obligation on you as deployer applies today.

eTMF classification at 98% is the same arithmetic

The trial master file version of this argument looks different and is identical. Vendors publish auto-classification accuracy claims clustering in the mid-nineties — Phlexglobal has published a figure around 95% for PhlexTMF auto-classification, a mechanism claim I have not independently verified. Buyers then negotiate over the last few points as if that were the decision.

Run the numbers on a 50,000-document TMF. At 98% accuracy the residual defect pool is 1,000 misclassified documents. At 94% it is 3,000. The four-point accuracy gap is worth 2,000 documents. But nobody sees those documents until QC finds them, and what QC finds is set by sample design, not by model accuracy. A 10% random sample surfaces about 100 of the 1,000 at 98% and about 300 of the 3,000 at 94%. The other 900 and 2,700 stay in the file until an inspector opens one.

The useful statistic here is the rule of three. With a zero-defect sample of n documents, the upper 95% confidence bound on the true defect rate is approximately 3/n. A clean 100-document sample supports a claim of under 3% misfiling. A clean 300-document sample supports under 1%. That sample size does not scale with TMF size — 300 clean documents defends the same claim in a 20,000-document file as in a 200,000-document one. Which means the marginal cost of defending a tighter bound is a few hundred documents of skilled review, and the marginal cost of the accuracy points is a model migration, a revalidation and a change control.

None of which argues for a worse model. It argues that the inspection-facing number is your defensible bound, not your vendor's accuracy claim — and record keeping and essential documents remain the most common finding category in MHRA sponsor inspections, at roughly 18% of all sponsor findings per Ennov's analysis of published MHRA GCP metrics.

What this means in practice

Start by counting queries, not by evaluating models. Pull last year's query log, split it EDC versus non-EDC, and split it again by whether the underlying field is critical to safety, the primary endpoint or eligibility. If your split resembles the published pattern, most of your query volume is non-critical and a meaningful share of your cost is non-EDC. That table is the business case. Nothing about model selection changes it.

Write the sampling plan before the pilot, not after. The deliverable that unlocks row three is a data-management-plan amendment defining the critical-data carve-out, the sample fraction for everything else, the acceptance criteria, and the escalation path when the sample fails. Anchor it on the E6(R3) Annex 1 clauses named above — 3.16.1(b), 4.2.1(a) and (c), 4.2.3, 4.2.6(a) — because a QA director who has read them will recognise that you are implementing the guideline rather than arguing around it. That document is signed by the sponsor's data management lead and countersigned by QA. Budget six to ten weeks for it, and expect the argument to be about the critical-data definition, not about the AI.

Validate the model as a proposer, not a decider. Fix the context of use in writing, performance-qualify against an independent, representative test set with pre-registered acceptance criteria, pin the model version, log every prompt and output as a record, and give the system read-only access to regulated stores so that nothing it produces writes to the database without an attributable human step. That architecture is what makes the sampling plan defensible: the human accountable step still exists on every query, it is simply preceded by a machine that has already done the retrieval and comparison.

What goes wrong. The reconciliation logic is protocol-specific and the vendor data formats are not, so a general-purpose classifier that performs beautifully on lab data will fail on ePRO and imaging feeds; budget separate qualification per source type. Vendor feed schemas change without notice mid-study, and drift monitoring on model performance will show it as a model problem when it is a data problem — instrument the input side too. And the honest limit: if your QA function will not sign a sampling plan, you are capped at 75% of the available saving no matter which model you buy. That conversation, not the bake-off, is the project.