A tool call contains braces, field names and constrained values. Generating it as if every next token were unrestricted prose can waste the structure already known to the system.
SchemaWeaver's October 12 paper uses schema-guided parallel decoding for on-device tool calling, without retraining the language model. Its authors report latency reductions across evaluated mobile processors and model families. Read the research paper.
Syntax is useful advance knowledge
An output schema narrows what can appear at particular positions. That creates opportunities to produce or check multiple tokens per pass while preserving the required form.
Valid structure is only one requirement. A perfectly formed request can call the wrong tool or place the wrong value in a valid field. Measure semantic execution success alongside decoding speed.
The benefit also depends on schema complexity and how much output is genuinely constrained. Free-form arguments may offer less exploitable structure than fixed enumerations.
For mobile agents, benchmark complete tool interactions, including validation and retries. SchemaWeaver's contribution is to use knowledge the application already possesses as a serving optimisation. The schema stops being merely a final-format checker and becomes information available during computation.
Questions, news tips or corrections? Contact [email protected].
