A drafter predicting several future tokens simultaneously cannot condition each guess on the final identities of its neighbours. Individually plausible tokens can therefore form an implausible sequence.
DCOCO, introduced October 8, keeps multiple candidates and scores compatibility between adjacent draft positions in parallel. The authors report throughput gains across tested verifier models, decoding settings and concurrent request loads. Read the research paper.
Correct the relationship, not just the token
The mechanism targets the gap between parallel computation and sequential language dependence. A stronger guess at one position is insufficient if it assumes a different preceding token from the one eventually chosen.
Local compatibility cannot recover every long-range dependency. The target model's verification remains essential, and the corrector's overhead must stay below the work it saves.
Concurrent serving is an important test because a correction method that helps one request can consume resources needed by others. Single-stream speedups are not a complete service result.
The paper suggests a productive compromise: retain parallel proposal generation while cheaply repairing some of the coordination it loses. Faster decoding need not choose between fully independent guesses and a complete return to one-token-at-a-time computation.
Questions, news tips or corrections? Contact [email protected].
