An agent can write a persuasive lesson after a successful task. That does not prove the lesson caused the success—or will help the next task.
An October 13 study finds large variation in self-written playbooks and proposes Randomized Block Curation. Related lessons are included or withheld at random on practice tasks, allowing their effect on success to be estimated. Curated playbooks outperform the unfiltered and agent-selected alternatives in the reported evaluations. Read the research paper.
Advice becomes an experimental treatment
This changes memory selection from literary criticism to an intervention. A lesson that sounds insightful can be dropped if outcomes do not improve when it is available.
Grouping lessons matters because individual effects may be too small to measure. But groups can also hide a harmful instruction inside helpful advice. Follow-up tests should split promising blocks and check interactions.
Practice tasks must remain separate from final evaluation. Otherwise curation can gradually fit the playbook to the same cases used to claim improvement.
The approach does not make every memory item causally measurable at low cost. It does provide a stronger standard for self-learning: retained advice should earn its place through changed behaviour, not through the agent's confidence in its own explanation.
Questions, news tips or corrections? Contact [email protected].
