Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #3744

attention heads

Related event timeline, sources and context from the news index.

EVENT TIMELINE

3

RESEARCH · 1 SOURCE · arXiv cs.AI

Detecting hallucination in LLMs by tracing attention-graph topology (arXiv:2609.21096v1)

This arXiv preprint proposes a single-pass method for detecting hallucinated outputs from LLMs by analysing information-flow topology in attention graphs, using Forman-Ricci curvature to identify bottlenecks. The paper reports consistent improvements over prior attention-based and multi-response baselines on two hallucination-detection benchmarks and finds that hallucinations correlate with impaired context sharing (e.g., over-reliance on self-attention, diffused retrieval from earlier tokens, or information over-squashing), especially in the final transformer layer, across several LLMs.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

ArXiv paper: fine-tuning alters LLM representations but EAP-identified causal components are decoupled

The new arXiv paper (arXiv:2609.21113v1) studies how fine-tuning changes internal LLM mechanisms and whether those representational changes align with task-relevant components identified by EAP (e.g., attention heads and logit-level activations). The authors report that EAP-identified components are localized to particular layers, yet their layer distribution is largely uncorrelated with layers showing the biggest representational shifts during fine-tuning, and overlap in EAP components across tasks does not reliably produce cross-task transfer and can sometimes cause performance degradation between dissimilar tasks.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

Four-stage decomposition of LLM math reasoning; distractor fragility localized to Operation Planning

The paper proposes that LLMs solve grade‑school math word problems via a four-stage pipeline—Schema Abstraction, Operation Planning, Operand Binding, and Computation—each producing distinct intermediate representations in specific layer bands. Using the same scaffold to analyze failure modes, the authors trace collapse caused by a single irrelevant clause to corruption of the Operation Planning stage, implemented by a set of attention heads whose causal role is validated bidirectionally.

7.0