RESEARCH · RESEARCH · #2087
LEV (arXiv:2610.20951v1): looped decision models enable variable-depth thinking and higher accuracy
The paper introduces LEV, a decision-model architecture that reruns its layers in a loop and reads options after each loop step so a single checkpoint can be executed at different depths; all loop steps are trained jointly and a small learned ‘stopper’ can set depth per question. The authors report accuracy increasing with depth on three looped backbones (e.g., a 2.6B LEV rises from 63.2% at depth 1 to 76.2% at depth 4 on JevBench), on-demand depth saves roughly 15–24% of compute for under one percentage point of accuracy loss, and 1.4B/3B looped models outperform comparable dense models and cascades by 5.4–8.8 points at latencies below 50 ms.
KEY POINTS
- The paper introduces LEV, a decision-model architecture that reruns its layers in a loop and reads options after each loop step so a single checkpoint can be executed at different depths; all loop steps are trained jointly and a small learned ‘stopper’ can set depth per question.
- The authors report accuracy increasing with depth on three looped backbones (e.g., a 2.6B LEV rises from 63.2% at depth 1 to 76.2% at depth 4 on JevBench), on-demand depth saves roughly 15–24% of compute for under one percentage point of accuracy loss, and 1.4B/3B looped models outperform comparable dense models and cascades by 5.4–8.8 points at latencies below 50 ms.
- LEV provides a practical method for allocating inference compute by question difficulty in non-autoregressive decision models, improving hard-example accuracy and enabling useful compute/latency trade-offs without needing separate checkpoints.
WHY IT MATTERS
LEV provides a practical method for allocating inference compute by question difficulty in non-autoregressive decision models, improving hard-example accuracy and enabling useful compute/latency trade-offs without needing separate checkpoints.