ThinkingBox Turns Database Checks Into RL Rewards for Agent Training

ThinkingBox was built to catch agents that claim to be finished when the database state shows otherwise. Each benchmark task gives an agent a real backend, a set of MCP tools and a simulated LLM user with private context, then runs executable checks against terminal database records. In an earlier post, the team described a model closing a customer’s ticket as resolved while the carrier exception behind it remained open: nine competent-looking tool calls, one wrong database row.

The new work treats those final-state checks as reward functions. The team assembled a training pool from ThinkingBox-format tasks outside the benchmark and used their executable outcomes as rewards. It selected tasks by “offline policy-relative difficulty”: estimating each task’s success probability once with the base policy before training and keeping the pool fixed throughout the run. The aim was to avoid tasks already solved or hopeless for the model, which the authors said teach it nothing.

For the Qwen3.8-27B run, each of the 507 benchmark tasks was evaluated 20 times. pass@1 measured average success over all 10,140 attempts; pass@20 measured tasks solved at least once; observed 20/20 measured tasks that passed every recorded attempt. Besides the pass@1 and 20/20 gains, pass@20 fell by seven tasks, from 453, or 89.35%, to 446, or 87.97%. The authors described the result as a sharp increase in repeatable success with a small decline in at-least-once coverage.

The blog gave a hotel-booking example. A traveler asks for a room downgrade, but company policy requires manager approval first. Before training, the agent changed the booking and issued refunds without approval in 15 of 20 attempts; after fine-tuning, it preserved the booking and payment state in all 20, with all expected records matching in 19.

ThinkingBox-Bench checks backend records rather than specific tool calls. The authors said this encourages models to discover new solution paths instead of rediscovering a human-backed solution that may be correct but suboptimal. In the earlier 18-model evaluation, Kimi-K3 solved 93.89% of tasks at least once but completed only 68 of 507 tasks on all 20 attempts. Claude Opus 5 had narrower coverage and greater consistency, and Claude Opus 5.5 gained 0.66 percentage points on pass@1 over Opus 5 but passed the same number of tasks, 241, on all 20 attempts. ThinkingBox-Training is now available on GitHub, the blog said.