reinforcement learning (RL)

2026-10-21

RESEARCH · 1 SOURCE · arXiv cs.AI

SpaRLax: sparse RL relaxation improves soft Chain-of-Thought reasoning

The arXiv paper (2610.27220v1) proposes SpaRLax, a sparse relaxation method for reinforcement-learning-style training of soft Chain-of-Thought (soft‑CoT). SpaRLax uses entmax to adaptively control sparsity so gradients can propagate across soft tokens over time; the method reportedly achieves the highest average results versus baselines across nine reasoning benchmarks and shows that adaptive sparsity and soft-token superposition contribute to gains.

6.0

2026-10-15

RESEARCH · 1 SOURCE · arXiv cs.AI

SAT-RL: training LLMs to interact with symbolic solvers via reinforcement learning

The paper introduces SAT-RL, an RL framework that trains LLMs to perform multi-turn interactions with symbolic satisfiability solvers (e.g., Z3). It combines synthetic data with verifiable solutions, supervised warm-up on solver-augmented traces, and solver-aware rewards, and reports substantial improvements on both in-domain and out-of-domain logical reasoning tasks; code is available at the linked GitHub repo.

7.0

2026-10-07

MODELS · 1 SOURCE · Hugging Face

Nemotron-3 specialists fine-tuned to gold level at IOI 2026 and IMO 2026

Teams starting from Nemotron 3 used supervised fine-tuning, reinforcement learning, and an inference loop (GenCorrect) to create specialist checkpoints. Nemotron-3-Ultra-CC scored 535.4/600 in a live IOI 2026 run (unofficial benchmark) and a combined IMO system scored 30/42 on IMO 2026 proofs judged by official graders, exceeding the gold threshold.

8.0

2026-09-25

RESEARCH · 1 SOURCE · arXiv cs.AI

DEEPO: dual-entropy enhanced policy optimization to reduce hallucination in MLLMs

The paper (arXiv:2609.28570v1) proposes DEEPO, a dual-stage RL enhancement for multimodal LLMs that combines semantic-entropy-triggered expert prefixes to restore advantage variance and advantage-sign-aware Renyi preconditioning to counteract logit saturation. Both components individually outperform GRPO and together yield statistically significant gains on the VideoMMMU benchmark (+4.0, 95% CI [1.1, 6.9]); DEEPO reduces hallucination while preserving accuracy and training stability.

6.0

2026-09-18

RESEARCH · 1 SOURCE · arXiv cs.AI

Compositional reasoning in LMs shows decomposed-to-composed asymmetry under RL post-training

This arXiv paper proposes a dependency-graph framework defining three compositionality levels and empirically studies compositional generalization of language models after reinforcement-learning post-training. Using deterministic data-structure tasks, the authors find a consistent decomposed-to-composed asymmetry—training on decomposed skills does not reliably transfer to composed tasks, while composed-task training transfers back more readily—provide a theoretical account, test length and structural shifts, and report a pilot on tool-calling benchmarks with preliminary evidence the asymmetry can appear in practical settings.

6.0

2026-09-15

RESEARCH · 1 SOURCE · arXiv cs.AI

Planning or Learning: Reliability and Cost in Multi-Asset Maintenance

This arXiv preprint empirically compares planning methods and reinforcement learning (RL) for multi-asset bearing maintenance using run-to-failure data. The authors find planning enforces reliability as a hard constraint and yields zero-failure policies whose total cost is largely insensitive to failure-penalty magnitude, while RL optimizes expected cost and often trades preventive maintenance for occasional failures (yielding lower costs under low-penalty regimes but persistent non-zero failures even when penalties are high); they also test lightweight constraint mechanisms (reward shaping, action masking) and provide a unified benchmark protocol for comparing decision approaches.

6.0