RESEARCH · RESEARCH · #2319
Omni-Woodpecker paper (arXiv v1): training-free representation rebalancing to reduce cross-modal hallucinations in omni LLMs
Omni-Woodpecker (OWP) is a training-free framework that detects cross-modal conflicts, decomposes hidden states to separate grounding evidence from competing priors, and adaptively rebalances fused representations to suppress misleading priors before decoding. On arXiv v1 the authors report that OWP improves Qwen2.5-Omni-7B by 12.9 and 8.6 points on CMM and AVHBench respectively, yields 0.67–2.25 point gains across four general OLLM reasoning benchmarks, and also improves VideoLLaMA2-AV by 4.6 and 3.9 points on the same benchmarks; code is released on GitHub.
KEY POINTS
- Omni-Woodpecker (OWP) is a training-free framework that detects cross-modal conflicts, decomposes hidden states to separate grounding evidence from competing priors, and adaptively rebalances fused representations to suppress misleading priors before decoding.
- On arXiv v1 the authors report that OWP improves Qwen2.5-Omni-7B by 12.9 and 8.6 points on CMM and AVHBench respectively, yields 0.67–2.25 point gains across four general OLLM reasoning benchmarks, and also improves VideoLLaMA2-AV by 4.6 and 3.9 points on the same benchmarks; code is released on GitHub.
- Reduces cross-modal hallucinations without additional training and generalizes across different omni LLM architectures, potentially improving multimodal model reliability.
WHY IT MATTERS
Reduces cross-modal hallucinations without additional training and generalizes across different omni LLM architectures, potentially improving multimodal model reliability.