An agent removes an obsolete paragraph from its working context. Later cached tokens may still encode information computed while that paragraph was present.
An October 12 paper studies budgeted KV-cache repair. Its Semantic-Unit Refresh recomputes coherent, relevant regions rather than isolated tokens, recovering much of full-recomputation performance in the evaluated deletion and question-answering tasks. Read the research paper.
A text edit is also a state edit
The key-value cache is a causal history of computation. Editing the displayed prompt without repairing dependent states can leave the system acting on a past version of the evidence.
Coherent regions matter because dependencies cross token boundaries. Refreshing individually high-scoring positions may leave their neighbours carrying inconsistent histories.
A partial repair remains an approximation. Compare it against exact full recomputation, especially when removing instructions, corrections or sensitive claims with broad downstream effects. This method's task-recovery results should not be described as a general guarantee of data erasure.
The practical implication reaches beyond performance engineering. Context-management systems need a contract linking visible edits to computational state. “The agent no longer sees that sentence” is not a sufficient statement if the cache still contains what the sentence caused the model to compute.
Questions, news tips or corrections? Contact [email protected].
