Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3971

BOS Summary: inference-only method that makes the BOS token carry input context to boost reasoning

The paper proposes BOS Summary, a training-free, inference-time modification that relaxes causal masking for the beginning-of-sequence (BOS) query row at a single transformer layer during prefill so BOS aggregates all visual and textual inputs into an input-specific context; other tokens keep standard causal attention. The change requires no extra parameters or decoding overhead and reportedly improves multi-step reasoning across several LLM and LVLM families and benchmarks, outperforming direct/CoT prompting and recent sink-intervention methods, while analysis shows it also reduces BOS sink dominance and redistributes attention toward content tokens.

KEY POINTS

  1. The paper proposes BOS Summary, a training-free, inference-time modification that relaxes causal masking for the beginning-of-sequence (BOS) query row at a single transformer layer during prefill so BOS aggregates all visual and textual inputs into an input-specific context; other tokens keep standard causal attention.
  2. The change requires no extra parameters or decoding overhead and reportedly improves multi-step reasoning across several LLM and LVLM families and benchmarks, outperforming direct/CoT prompting and recent sink-intervention methods, while analysis shows it also reduces BOS sink dominance and redistributes attention toward content tokens.
  3. A simple, training-free inference intervention that repurposes the persistent BOS pathway can materially improve multi-step reasoning and reveals how changing sink content reshapes attention allocation in transformers.

WHY IT MATTERS

A simple, training-free inference intervention that repurposes the persistent BOS pathway can materially improve multi-step reasoning and reveals how changing sink content reshapes attention allocation in transformers.

SOURCES & TIMELINE

1