If an agent improves after seeing past attempts and rewards, it seems natural to credit the rewards. Remove or scramble them, however, and that explanation becomes testable.
An October 8 paper examines direct in-context reinforcement learning across six models and four benchmarks. In the tested setting, altering or removing reward values leaves improvement curves largely unchanged. Corrupted trajectories can also produce similar effects. Read the research paper.
Reading feedback is not using its meaning
The result distinguishes visible reward tokens from a causally useful learning signal. A model may react to prompt structure or distributional cues without learning which actions the reward endorses.
The study concerns direct conditioning on raw trajectory-reward pairs. It should not be extended to every memory architecture, meta-trained learner or parameter-updating RL method.
For a self-learning agent, reverse rewards while holding trajectories fixed. If behaviour barely changes, improving the reward rubric may not address the actual mechanism.
The finding is valuable because an incorrect explanation leads to incorrect engineering. A system can benefit from accumulated context while failing to perform the reward-sensitive adaptation its designers assume. Calling that process reinforcement learning does not substitute for showing that the reward changes what it learns.
Questions, news tips or corrections? Contact [email protected].
