Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #9422

Transformer

Related event timeline, sources and context from the news index.

EVENT TIMELINE

4

RESEARCH · 1 SOURCE · arXiv cs.AI

Modeling LLM reasoning dynamics with T/E/S structural representations (arXiv:2611.23749v1)

This arXiv paper introduces T/E/S (text, local events, spans of activity) to represent temporal structure in language-model reasoning. Across reasoning and information-seeking tasks, adding events and spans improves next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA; span duration and preceding events associate with transitions and retaining event history across spans helps prediction. The authors formalize budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, derive conditions for decision certification, and show in finite-process experiments that bounds which cancel shared continuation mass recover optimal actions while reducing mean inference CPU time by 77.7%. A compact structural observer also lowers Brier loss from 0.0836 to 0.0756 on 71 held-out HotpotQA and MuSiQue questions, but an annotation audit finds substantial errors in that observer, separating the computational findings from semantic validation of its features.

6.0

MODELS · 1 SOURCE · Apple Machine Learning Research

Attention to Mamba: two-stage recipe distills Pythia-1B Transformer into Mamba SSM

Authors propose a principled two-stage cross-architecture distillation: first distill a Transformer into a linearized-Attention model (using a kernel-trick adaptation), then distill that into a Mamba-style State Space Model without Attention. The distilled Mamba nearly matches the Pythia-1B teacher (perplexity 14.01 vs 13.86) across evaluation and includes ablations, scaling analyses, and a distillation scaling law for compute-optimal allocation.

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

ArXiv paper introduces 'Speed of Coherent Cognition' benchmark and Co^4 architecture for reliability under faster-changing realities

The paper formulates a model-agnostic challenge measuring how fast reality can change before an intelligent system stops behaving reliably, formalizing the metric Speed of Coherent Cognition (S_C^{eff}) relative to a Speed of Reality (S_R). As a proof of concept, reinforcement-learning agents trained at a baseline S_R are evaluated without retraining as S_R is increased up to 50× with fixed compute; an architecture called Co^4 inspired by cellular mechanisms attains substantially higher S_C^{eff} than Transformer and gated-MLP baselines, motivating S_C^{eff} as a new benchmark and coherent-cognition scaling as a design principle.

6.0

RESEARCH · 1 SOURCE · arXiv cs.AI

T-RoPE: time-aware rotary position embedding improves sequential recommendation

arXiv:2609.30576v1 introduces T-RoPE, a time-aware variant of Rotary Position Embedding (RoPE) for Transformer-based sequential recommendation that replaces index-only rotations with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. The paper proves standard RoPE is time-translation invariant, reports state-of-the-art gains across five public benchmarks (e.g., +78–130% HR@10 on sparse PixelRec; +8–12% on Amazon Books), large improvements on a 6B-interaction industrial e-commerce dataset (+13–82%), and small but positive online A/B lifts in the Shop app (+0.33% conversion, +0.63% order count); it also provides forward/backward algorithms with linear added cost in sequence length and head dimension.

8.0