RESEARCH · RESEARCH · #2456
ThinkingBox-Training released on GitHub; RL fine-tune raises Qwen3.8-27B reliability using database-graded rewards
ThinkingBox-Training is now available on GitHub. The toolkit turns executable checks of terminal database state into RL rewards; fine-tuning Qwen3.8-27B with 50 GRPO steps raised average pass@1 from 51.70% to 60.78% and increased tasks solved on all 20 attempts from 38 to 107, using an offline policy-relative difficulty selection of training tasks and evaluation on the 507-task ThinkingBox-Bench.
KEY POINTS
- ThinkingBox-Training is now available on GitHub.
- The toolkit turns executable checks of terminal database state into RL rewards; fine-tuning Qwen3.8-27B with 50 GRPO steps raised average pass@1 from 51.70% to 60.78% and increased tasks solved on all 20 attempts from 38 to 107, using an offline policy-relative difficulty selection of training tasks and evaluation on the 507-task ThinkingBox-Bench.
- Using executable backend checks as rewards materially improves repeatable, production-relevant agent behavior and provides a measurable way to train for database safety and reliability beyond headline pass@1 metrics.
WHY IT MATTERS
Using executable backend checks as rewards materially improves repeatable, production-relevant agent behavior and provides a measurable way to train for database safety and reliability beyond headline pass@1 metrics.