Giving a model the answer or environmental feedback should make it a better teacher for itself. Reported training gains do not automatically show that the privileged information produced new reasoning ability.
An October 13 study revisits two self-distillation methods across code, science, tools and mathematics. In its experiments, gains do not consistently transfer to unseen problems and can reflect overfitting or existing behaviour transfer. Read the research paper.
A model can become a better imitator without learning more
Self-distillation offers an appealing picture of progress: give a model extra information, ask it to produce a better answer, and train the original model to imitate that improved version of itself. The extra information might be a correct solution or environmental feedback. The procedure can raise benchmark scores, but that observation leaves two explanations open. The student may acquire more general reasoning ability, or it may become more likely to reproduce behaviour already available within its existing repertoire.
The distinction affects how much confidence to place in another round of self-improvement. Only a test designed to separate these mechanisms can establish what the training signal actually contributed.
Privileged information needs a causal test
The study revisits Self-Distillation Policy Optimization and On-Policy Self-Distillation across code, science, tool use and mathematics. It examines Qwen3 models ranging from 1.7 billion to fourteen billion parameters and OLMo-3-7B. In the authors' experiments, reported gains do not consistently generalise to unseen problems. They attribute much of the observed improvement to overfitting or transfer of existing reasoning behaviour rather than effective use of the privileged information.
That finding does not make behaviour transfer worthless. Making useful behaviour more reliable can be valuable. It does, however, change the claim: redistributing existing competence is a narrower achievement than learning to solve genuinely new kinds of problems from feedback.
The direction of the update matters
The authors also compare self-distillation gradients with policy gradients that directly optimise expected reward. They find weak alignment. This provides another way to investigate why an apparently informative teacher response might fail to produce the desired improvement. A supervision signal can look persuasive in language while pushing parameters in a direction poorly matched to the actual objective.
Gradient comparison is not, by itself, a universal verdict on distillation. Different objectives can produce useful updates without identical gradients. Here it is part of a broader audit that includes generalisation and the role of privileged information. Read together, those tests challenge the explanation for the gains rather than merely reporting a disappointing leaderboard result.
The audit does not cover every hybrid using a teacher
The authors isolate the self-distillation objective in SDPO and OPSD. Methods retaining a reinforcement-learning objective while using teacher scores only to reweight advantages fall outside that focus. So do some newer harness-based, contrastive and aggregated-feedback approaches. This boundary matters because a disappointing result for imitation-driven updates is not automatically a result about every system containing a self-conditioned teacher.
The study also concentrates on verifiable domains. Open-ended tasks can motivate distillation for reasons that do not reduce to the same reward tests. The appropriate response is therefore to make a method’s source of improvement explicit: which information changes the target, which objective changes the policy, and where the learned effect transfers. That is a more useful standard than treating self-distillation as one uniform technique.
What survives a stricter comparison
A convincing follow-up should retain unseen problem families, meaningful controls for the extra information, and an account of the teacher's computation. It should report where existing capabilities became easier to elicit as well as where new capabilities appeared. Mixing those outcomes into one accuracy number makes methods hard to compare and encourages claims about learning that the experiment cannot resolve.
Recursive training loops need independent evaluation between rounds. A rising training score can otherwise become evidence fed back into the next round of the same loop. The value of self-distillation depends on what remains when familiar tasks, helpful context and the model's existing answer patterns no longer carry the evaluation.
Questions, news tips or corrections? Contact [email protected].
