A useful instruction can still produce an update that clashes with what a model is learning now. Selecting examples once cannot fully control that interaction.
GRADE's October 6 paper combines state-aware sample selection with a gate for potentially destructive updates during low-rank adaptation. Across the tested backbones and instruction mixture, the authors report improved accuracy and robustness over their comparisons. Read the research paper.
A narrow update space can fill unevenly
LoRA constrains adaptation to a limited parameter subspace. Gradients from different tasks can cancel or overwrite useful directions, so example quality alone does not determine the final effect.
The proposed selector follows the evolving multi-task gradient field, while the gate addresses harmful steps near saturation. That makes timing part of curation rather than leaving it entirely to the optimiser.
A fair comparison must include the overhead of measuring alignment and rejecting steps. It should also inspect minority tasks: an apparently coherent average direction could suppress a legitimate but less common objective.
GRADE's results support adaptive admission in its evaluated setting. The broader insight is that a dataset is not just a bag of lessons. Training turns it into a sequence of interventions, and the same lesson can help or interfere depending on when it is applied.
Questions, news tips or corrections? Contact [email protected].
