NEWS · RESEARCH · #153
GAVEL: an LLM-based adjudication protocol to compare and merge clinical timelines from case reports
Authors present GAVEL, an LLM judge protocol that compares two extracted clinical timelines against the source case report and returns a discrepancy type, verdict, and report passage for each difference. The paper evaluates an event matcher and reviews 2,738 findings from GPT5.6sol and DeepSeek V3.2, ranks six LLM extractors and two human annotators, and tests GAVEL-guided merging: across 126 reports merged timelines were preferred in 77.0% of comparisons and discrepancies attributed to the evaluated timeline fell from 7.63 to 0.85 per report; manual review confirmed 89.4% and 88.6% of findings, and reported true match rates of 60% just below and 48% just above a 0.10 cutoff.
KEY POINTS
- Authors present GAVEL, an LLM judge protocol that compares two extracted clinical timelines against the source case report and returns a discrepancy type, verdict, and report passage for each difference.
- The paper evaluates an event matcher and reviews 2,738 findings from GPT5.6sol and DeepSeek V3.2, ranks six LLM extractors and two human annotators, and tests GAVEL-guided merging: across 126 reports merged timelines were preferred in 77.0% of comparisons and discrepancies attributed to the evaluated timeline fell from 7.63 to 0.85 per report; manual review confirmed 89.4% and 88.6% of findings, and reported true match rates of 60% just below and 48% just above a 0.10 cutoff.
- GAVEL provides an LLM-based method to adjudicate and merge clinical timelines without treating any timeline as ground truth, which could improve evaluation and aggregation practices in clinical information extraction research.
WHY IT MATTERS
GAVEL provides an LLM-based method to adjudicate and merge clinical timelines without treating any timeline as ground truth, which could improve evaluation and aggregation practices in clinical information extraction research.