An agent can revise its workflow until one difficult task succeeds. That experience may still do nothing for the next unfamiliar task.
Harness², introduced October 9, combines task-level harness edits with a higher-level process that extracts experience for future adaptation. Its authors report improvements across professional benchmarks and several model-harness configurations. Read the research paper.
A retained editing strategy is the second product
The first loop produces a better current execution. The second attempts to improve the method used to choose future edits. Calling both recursive improvement without separating them hides where transfer is supposed to occur.
Open-world tasks often lack a definitive verifier. Contrastive probing can supply feedback, but can also favour changes that look persuasive without improving the final artifact.
For evaluation, compare fresh tasks before and after the retained adaptation process changes, under the same search budget. Also inspect regressions in tasks the original harness handled well.
The published gains support the framework's tested settings. The useful distinction is between buying another round of effort and becoming better at allocating that effort. A persistent self-improvement claim should show the latter, not merely the success of a more thoroughly searched current answer.
Sakana AI and UC Berkeley’s MASS paper, posted October 8, takes another route: optimise multi-agent workflows, then fine-tune a shared model on their trajectories. Harness² retains improvements in the surrounding execution process; MASS also updates weights. Comparing them requires stating which artifact persists and counting the work needed to produce it. “Self-improvement” does not identify that mechanism by itself.
References
Questions, news tips or corrections? Contact [email protected].
