RESEARCH · RESEARCH · #719
Embed-TTT: two-step test-time task embedding improves rule induction in VARC (arXiv:2609.21181v1)
The paper introduces Embed-TTT, a two-step test-time training protocol for Vision ARC (VARC) where a task embedding is first finetuned alone and then the backbone is finetuned. Across ARC-AGI-1, ConceptARC, and controlled rule-known datasets, Embed-TTT yields task embeddings that better match underlying rules, improves embedding-based retrieval, enables accurate linear probing of known rules, and—using embeddings that are <0.01% of model parameters—already solves a non-trivial fraction of tasks; the full two-step pipeline further boosts performance and shows compositional interpolation but not extrapolation.
KEY POINTS
- The paper introduces Embed-TTT, a two-step test-time training protocol for Vision ARC (VARC) where a task embedding is first finetuned alone and then the backbone is finetuned.
- Across ARC-AGI-1, ConceptARC, and controlled rule-known datasets, Embed-TTT yields task embeddings that better match underlying rules, improves embedding-based retrieval, enables accurate linear probing of known rules, and—using embeddings that are <0.01% of model parameters—already solves a non-trivial fraction of tasks; the full two-step pipeline further boosts performance and shows compositional interpolation but not extrapolation.
- This clarifies separation between rule induction and rule execution in ARC-like evaluations and suggests lightweight embedding adaptation can reveal whether models learn true rules, informing benchmark design.
WHY IT MATTERS
This clarifies separation between rule induction and rule execution in ARC-like evaluations and suggests lightweight embedding adaptation can reveal whether models learn true rules, informing benchmark design.