RESEARCH · RESEARCH · #926
PaMER: pre-action memory control signals discovered in long-horizon language-model agents; method reduces context use
Researchers report that hidden states immediately before each action in long-horizon language-model agents encode signals indicating the need for memory compression and recall, patterns that vary with model depth and are not explained by simple context length or progress. They introduce Preaction Memory with Evidence Retrieval (PaMER) and PaMER+—state-guided compression plus selective historical evidence retrieval—and show on WorkBuddyBench that these approaches substantially cut context consumption while maintaining competitive task performance across multiple backbones and baselines.
KEY POINTS
- Researchers report that hidden states immediately before each action in long-horizon language-model agents encode signals indicating the need for memory compression and recall, patterns that vary with model depth and are not explained by simple context length or progress.
- They introduce Preaction Memory with Evidence Retrieval (PaMER) and PaMER+—state-guided compression plus selective historical evidence retrieval—and show on WorkBuddyBench that these approaches substantially cut context consumption while maintaining competitive task performance across multiple backbones and baselines.
- If pre-action memory needs can be detected and acted on, agents can reduce costly context retention and selectively retrieve only needed history, improving scalability of long-horizon LM agents.
WHY IT MATTERS
If pre-action memory needs can be detected and acted on, agents can reduce costly context retention and selectively retrieve only needed history, improving scalability of long-horizon LM agents.