Most demonstration libraries show a model what success looks like. SMILE asks whether the model needs to see where its own interpretation previously went wrong.
The October 12 paper records a target vision-language model's errors in an offline pool. At inference, its current prediction guides retrieval of related mistakes. Across the authors' evaluated models and benchmarks, this improves visual in-context learning with lower latency than their comparison method. Read the research paper.
The model's guess becomes a retrieval clue
A mistaken prediction is useful here because it points toward a recurring confusion. The same image can be easy for one model and misleading for another; a generic nearest-neighbour library overlooks that difference.
But retrieval conditioned on a prediction also risks reinforcing a poor initial interpretation. If the first guess selects the wrong family of errors, the demonstrations may answer a question the image did not pose. A deployment check should compare prediction-guided retrieval with image-only retrieval specifically on severe misclassifications.
SMILE's results support model-specific correction, not universal transfer of one model's error pool. Updating the backbone may invalidate which examples are helpful. The broader lesson is that a useful few-shot library can be a record of the recipient's blind spots. Its value comes from the relationship between the example and the model, not from the example's apparent quality in isolation.
References
Questions, news tips or corrections? Contact [email protected].
