An agent can describe a completed job while leaving the underlying records wrong. Training against the final message risks rewarding that appearance. A backend check can instead ask whether the requested change happened and whether related records remained consistent.
The October 22 ThinkingBox-Training report uses executable terminal-state checks as reinforcement-learning rewards on tasks outside its benchmark. For Qwen3.8-27B after 50 GRPO steps, reported average success rises from 51.70% to 60.78%. Tasks passing all twenty observed attempts rise from 38 to 107 out of 507, while at-least-once coverage falls from 453 to 446.
Read the original source and its reported conditions.
The reward defines what counts as finished
A terminal-state test can accept several valid action sequences rather than requiring one prescribed route. Its weakness is equally concrete: an incomplete check may miss an unauthorized intermediate action or an external side effect. Final-state correctness and process constraints need separate treatment where both matter.
The authors selected training tasks using difficulty estimated under the starting policy, then held that pool fixed. That makes the selection procedure part of the intervention. Reproducing the result requires the task pool and reward definitions, not only the final checkpoint.
Consistency and breadth move differently
The mixed metrics resist a simple claim that the trained model became better at everything. Repeated success improved while some tasks lost their occasional successful path. An application owner should decide which failure pattern is tolerable before selecting a checkpoint.
This work offers an inspectable training direction for systems that change records. The practical next test is whether gains survive different tools, permissions and business rules, with independent checks for both correct outcomes and actions the agent must never take.
References
Questions, news tips or corrections? Contact [email protected].


