A cached document is often treated like a preprocessed file. Its key-value states also contain the influence of the prefix and earlier documents, so moving it can change the answer.
SAFE-KV's October 10 study repairs the causal suffix after non-prefix reuse and compares answers with exact full prefill. It reports strong preservation after repair, but its single-request hardware study finds faithful repair expensive enough to favour full prefill for every evaluated workload. Read the research paper.
A negative operating result is useful engineering
The result prevents a misleading conclusion: demonstrating a repair mechanism does not demonstrate profitable reuse. Correctness can reveal that the proposed shortcut is not worthwhile in a particular setting.
The paper does not rule out useful reuse under other workloads, batching arrangements or hardware. It identifies the causal-state contract those systems need to satisfy.
For RAG serving, compare identical prompts and answer preservation, then include every recomputed suffix token in the cost. Reusing a nominally large block can still save little if dependent states must be rebuilt.
The important advance is a clearer boundary between a valid optimisation and a merely fast approximation. Sometimes the right outcome of careful systems research is knowing when to decline the shortcut.
Questions, news tips or corrections? Contact [email protected].
