A response can be accurate but badly justified, or complete but poorly grounded. Collapsing a rubric into one reward number hides which property the learner should change.
CoCA's October 13 paper derives credit from criterion-level outcomes and distributes it using criterion-related changes in token probabilities. Its evaluated medicine and science benchmarks improve over the comparison rubric-RL approach. Read the research paper.
Feedback structure must survive optimisation
The rubric already contains a decomposition of quality. Uniform token advantages discard that information, potentially reinforcing correct and incorrect passages together.
Fine-grained credit is not automatically faithful. A token associated with a criterion may not have caused its failure; correlated phrasing can receive misplaced credit. Inspect targeted changes in responses, not only the total reward.
The reported domain benchmarks do not certify medical advice or scientific truth. They evaluate a training method under specific graders and tasks.
For open-ended post-training, this makes reward design only half the story. The optimiser must also preserve the distinctions the reward process worked to measure. A careful rubric adds little value if its feedback is flattened into the same update for every word.
References
Questions, news tips or corrections? Contact [email protected].
