RESEARCH · RESEARCH · #3395
Counterfactual lesion erasure trains MedScope-4B to report only image-visible findings
The paper (arXiv:2611.13006v1) creates counterfactual views by inpainting lesions and matched control edits from public segmentation datasets and uses them to train MedScope-4B (Qwen3.5-4B, LoRA, image tools) on label-derived trajectories. On 507 internal cases across five imaging types MedScope-4B achieved an evidence sensitivity gap (ESG) of 0.758 and claimed erased lesions in 9.6% of cases versus frontier VLMs with ESG 0.27–0.47 and erased-lesion claims of 41–62%; the effect weakened without counterfactual views, replicated on MedGemma 1.5 4B and other inpainting tools, and held in external centres though frontier models found more real lesions and MedScope-4B missed some critical findings (e.g., 0 of 48 pneumothorax films in one external test).
KEY POINTS
- The paper (arXiv:2611.13006v1) creates counterfactual views by inpainting lesions and matched control edits from public segmentation datasets and uses them to train MedScope-4B (Qwen3.5-4B, LoRA, image tools) on label-derived trajectories.
- On 507 internal cases across five imaging types MedScope-4B achieved an evidence sensitivity gap (ESG) of 0.758 and claimed erased lesions in 9.6% of cases versus frontier VLMs with ESG 0.27–0.47 and erased-lesion claims of 41–62%; the effect weakened without counterfactual views, replicated on MedGemma 1.5 4B and other inpainting tools, and held in external centres though frontier models found more real lesions and MedScope-4B missed some critical findings (e.g., 0 of 48 pneumothorax films in one external test).
- This demonstrates a data-driven training method that substantially reduces image-ungrounded claims by a small medical VLM (increasing evidence sensitivity) but does not improve perceptual detection and may miss rare critical findings.
WHY IT MATTERS
This demonstrates a data-driven training method that substantially reduces image-ungrounded claims by a small medical VLM (increasing evidence sensitivity) but does not improve perceptual detection and may miss rare critical findings.