Few-shot prompting has a recurring bill: every new query rereads the demonstrations. With images, even a short example library can consume a large number of visual tokens.
STAVE, introduced October 6, replaces demonstrations with two task-specific vectors added to existing embeddings. One affects answer-producing tokens; the other affects structural context tokens. The vectors are trained from labelled prompts, while the backbone remains frozen. Read the research paper.
Demo-free inference still has an adaptation cost
The useful distinction is between removing examples at serving time and learning without examples. STAVE does the first. Its compact task state is obtained through training, so describing the entire process as zero-shot would obscure where supervision entered.
A serving team should compare repeated prompt processing with the cost of fitting and maintaining those vectors. Stable, frequently used tasks are different from one-off requests whose instructions change every time.
The two-vector design also changes the intervention surface. Rather than adding tokens that compete for attention, it alters how the existing prompt is represented. That can be efficient, but makes mismatched task states less visible to a human inspecting the text.
Version the vectors alongside task definitions and test them when the backbone changes. A compact representation saves tokens only if it keeps representing the same job.
References
Questions, news tips or corrections? Contact [email protected].
