Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #5474

Mamba

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

MODELS · 1 SOURCE · Apple Machine Learning Research

Attention to Mamba: two-stage recipe distills Pythia-1B Transformer into Mamba SSM

Authors propose a principled two-stage cross-architecture distillation: first distill a Transformer into a linearized-Attention model (using a kernel-trick adaptation), then distill that into a Mamba-style State Space Model without Attention. The distilled Mamba nearly matches the Pythia-1B teacher (perplexity 14.01 vs 13.86) across evaluation and includes ablations, scaling analyses, and a distillation scaling law for compute-optimal allocation.

7.0

MODELS · 1 SOURCE · arXiv cs.AI

DSRec: dual-interest cross-SSM for sequential recommendation

arXiv:2609.21548v1 introduces DSRec, a dual-interest cross-SSM architecture that disentangles long-term and short-term item roles by encoding items into long-term interest embeddings (processed by a full-sequence Mamba SSM) and a time-modulated short-term SSM, with a residual cross-fusion mechanism to align the two branches; experiments on public benchmarks report improvements over prior state-of-the-art methods.

5.0