pathledger
The layer under LLM abstraction of pathology reports: it lets the extractor say unknown, checks every answer against the public registry, keeps a ledger of where they disagree, records exactly how each result was produced, and fails the build when quality drops.
Built on the same public reports used in the Rasool Lab consensus framework (Tripathi, Waqas et al., Laboratory Investigation, 2025), to answer what that paper leaves open.
Why this exists
Cancer registries abstract stage, grade, histology, site and laterality from every surgical pathology report, by hand. The Rasool Lab showed that eleven locally deployed language models with three reasoning judges can do this with pathologist validated accuracy in the high eighties to high nineties on TCGA reports. Read as an infrastructure engineer, three questions follow.
A forced consensus value hides the coverage versus accuracy curve that a registry manager needs to set a threshold.
GDC structured fields exist for the same cases. Where the report and the registry disagree, one of them is wrong, and that is the finding.
Every run is content addressed. A rerun is a cache hit, a changed prompt is a new namespace, and a gate fails the build when quality drops.
How it works
Rules first and free, the model only for what rules could not settle, every answer either verbatim evidence or UNKNOWN, every run content addressed, a gate at the end.
Try it in your browser
Pick a report from the published run to see the full result, rules and model, exactly as it was produced, embedded in this page. Or paste any de-identified public report: the rules run right here, with no model calls or external requests.
Report
This browser demo runs local rules and embedded examples only. No API key is required; unresolved fields remain UNKNOWN.
Extracted fields
Evidence in the text
Results
Coverage is the share of reports where the system committed to an answer. Agreement is measured on answered fields against the GDC registry value.
| field | answered | coverage | agreement with GDC | rules share | published, pathologist reviewed | local model, subset |
|---|---|---|---|---|---|---|
| stage | 50 / 60 | 83.3% | 62.0% | 83.3% | 89.0% | 68.8% |
| grade | 49 / 60 | 81.7% | 95.6% | 81.7% | 96.0% | 93.8% |
| histology | 60 / 60 | 100.0% | 51.7% | 0.0% | 99.0% | 45.0% |
| site | 60 / 60 | 100.0% | 100.0% | 0.0% | 95.0% | 100.0% |
| laterality | 57 / 60 | 95.0% | 84.2% | 95.0% | 88.0% | 85.0% |
Disagreements are data quality findings
Where the report and the GDC field disagree, one of three things is true: the extractor is wrong, the registry is wrong, or the report is ambiguous. A person adjudicated a sample.
| field | case | extracted | registry | conf | evidence | adjudication |
|---|
Provenance and cost
Deterministic rules settled 156 field values across 60 reports; a model was needed for 60. Every extraction is content addressed on report hash, prompt hash, rules version, backend and model. A rerun is a cache hit. A changed prompt is a new namespace and never mixes with the old one.
Run ledger
| started | backend | model | prompt | rules | commit | n | cache hits | model calls |
|---|---|---|---|---|---|---|---|---|
| 2026-09-13T17:32:21 | fake | none | 32c0f6c31744eb80 | rules-v1 | nogit | 3 | 0 | 3 |
| 2026-09-13T17:32:21 | fake | none | 32c0f6c31744eb80 | rules-v1 | nogit | 3 | 3 | 0 |
| 2026-09-13T17:32:22 | preview | synthetic-template | 32c0f6c31744eb80 | rules-v1 | nogit | 60 | 0 | 60 |
Reproduce this run
python pathledger.py fetch --projects TCGA-BRCA TCGA-LUAD --per-project 100 python pathledger.py extract --backend anthropic python pathledger.py evaluate && python pathledger.py check
Limitations
- Registry agreement is not pathologist agreement, and GDC fields may have been abstracted from these same reports, so agreement is partly circular. Disagreements are still informative.
- A few hundred reports across two cancer types, not 6,100 across ten organ systems.
- Hosted API for throughput; the local model column is a subset run. Stated confidence is not calibrated.
- Built by one person in a few days on public data as a design argument with numbers. Not a medical device, no clinical claims.
What this would become inside an institution
Organ specific TNM to stage group mapping instead of string comparison. mCODE aligned output so the structured fields land in a registry without a second transform. Per modality governance on the derived artifacts, since an extraction ledger is derived data with its own re-identification profile. And the gate wired to the nightly job, so a prompt or model upgrade cannot change registry numbers without failing a build first.