RESEARCH · RESEARCH · #2011
SAT-RL: training LLMs to interact with symbolic solvers via reinforcement learning
The paper introduces SAT-RL, an RL framework that trains LLMs to perform multi-turn interactions with symbolic satisfiability solvers (e.g., Z3). It combines synthetic data with verifiable solutions, supervised warm-up on solver-augmented traces, and solver-aware rewards, and reports substantial improvements on both in-domain and out-of-domain logical reasoning tasks; code is available at the linked GitHub repo.
KEY POINTS
- The paper introduces SAT-RL, an RL framework that trains LLMs to perform multi-turn interactions with symbolic satisfiability solvers (e.g., Z3).
- It combines synthetic data with verifiable solutions, supervised warm-up on solver-augmented traces, and solver-aware rewards, and reports substantial improvements on both in-domain and out-of-domain logical reasoning tasks; code is available at the linked GitHub repo.
- This work helps bridge flexible natural-language reasoning in LLMs with the scalability and verifiability of symbolic solvers, improving performance on harder logical tasks.
WHY IT MATTERS
This work helps bridge flexible natural-language reasoning in LLMs with the scalability and verifiability of symbolic solvers, improving performance on harder logical tasks.