One attention head concentrates on a few tokens. Another spreads attention across a long document. Giving both the same cache budget can waste storage in one and erase useful evidence in the other.
ChampionKV, posted October 13, allocates head-wise budgets using a lightweight concentration proxy. It improves its underlying compressors in the authors' evaluations, with stronger results on the tested larger model and mixed outcomes on smaller ones. Read the research paper.
Uniform budgets hide unequal demand
The method changes allocation rather than the base token-selection procedure. That separation makes it possible to ask whether a compressor's weakness lies in choosing tokens or deciding how many tokens each head may keep.
Attention concentration is not a perfect importance signal. A broad head may be diffuse because it is unhelpful, while a concentrated head may need one rare token that calibration misses.
The authors also report that another global allocation method retains advantages in some settings. That limitation belongs in the conclusion: adaptive allocation is useful, not universally dominant.
For deployment, compare quality at fixed total memory and inspect per-head behaviour on your own prompts. A cache budget is a resource-allocation policy, and the best allocation can change with the input rather than merely with model size.
References
Questions, news tips or corrections? Contact [email protected].
