SAT-RL: training LLMs to interact with symbolic solvers via reinforcement learning
The paper introduces SAT-RL, an RL framework that trains LLMs to perform multi-turn interactions with symbolic satisfiability solvers (e.g., Z3). It combines synthetic data with verifiable solutions, supervised warm-up on solver-augmented traces, and solver-aware rewards, and reports substantial improvements on both in-domain and out-of-domain logical reasoning tasks; code is available at the linked GitHub repo.