Training only on clean success traces teaches an agent what to do when everything goes well. A deployed terminal session rarely offers that guarantee.
ReTerm's October 13 paper builds recovery demonstrations and uses reinforcement learning to launch multiple attempts from the agent's own failure states. The authors report improved terminal-task success and recovery across error categories. Read the research paper.
The failed state is valuable supervision
Restarting every attempt from the beginning wastes the context that made recovery hard. A partially modified repository, a mistaken command and a misleading error message are part of the decision problem.
Forking the state lets training compare different repairs under a shared starting condition. It also risks overfitting to failure patterns the current policy already generates. New tools and unfamiliar errors remain important tests.
Measure whether recovery preserves valid work, not just whether the task eventually passes. An agent that discards useful changes and starts over may succeed while imposing unnecessary cost.
ReTerm supports treating mistakes as branch points for learning rather than as discarded samples. The lesson is especially relevant to self-improving agents: the ability to make a better first attempt and the ability to rescue a damaged ongoing attempt are distinct capabilities worth training separately.
Questions, news tips or corrections? Contact [email protected].
