On the Importance of Gating: how gating steers memorization vs in‑context learning in SSMs
Researchers show theoretically and empirically that the gating mechanism in state space models (SSMs) often drives training toward an in‑weights memorization solution and can delay or prevent convergence to correct in‑context learning, while also sometimes improving generalization to longer sequences. The paper analyzes how gating shapes both training dynamics and long‑sequence generalization in linear‑time sequence models.