A cheaper language model can make an AI session more expensive if switching to it discards work the current model has already done. Price comparisons usually quote the cost of fresh input and output tokens. A running agent has a history, and part of that history may already be sitting in a reusable prompt cache.
A router sold as an inference endpoint
Fireworks introduced FireRouter with Opus on September 28, 2026. The service is available through its command-line tools and as a standalone serverless endpoint. At launch, it chooses among Claude Opus 5.5, GLM 5.3 and GLM 5.3 Flash; the company says that set will change as models launch.
Each user turn is assessed for the models suited to handling it. The router then estimates the cost, including prompt-cache effects, before selecting an option. An application can therefore delegate model selection to the endpoint while continuing to use an existing agent harness. The current setup documentation adds an important condition: a route using a closed model also needs the matching provider credential. Without one, FireRouter serves turns with open models. A Fireworks account alone does not unlock paid Opus access.
Switching has a memory bill
Prompt caching lets a provider reuse computation for input it has already processed. An agent repeatedly supplying a long conversation or code context can benefit from those saved computations. Moving the next turn to another model can lose that advantage, even if the alternative has a lower advertised token price.
FireRouter explicitly considers whether a switch is worth that loss. Its decision is therefore about the state of a session as well as the difficulty of the next request. Two identical-looking requests could sensibly go to different models if their preceding contexts have different cache histories.
This is where the release adds something to the familiar proposal of sending easy work to small models. Routing is also a scheduling problem across resources that retain useful state. A per-request price comparison leaves that state out of the accounting.
What the internal experiment actually measured
Fireworks randomly assigned internal coding sessions to the router or an Opus-only control, across the same workloads and users. Reported cost per session fell from $15.36 to $6.63, a 57% reduction. The company gives an uncertainty of plus or minus 19 percentage points for that reduction.
Accuracy was not unchanged. Fireworks graded whether the agent completed the work, reached the correct answer and needed a correction on the next turn. The router scored 78.7% of graded turns, compared with 80.2% for Opus alone. Calling that 98.1% of Opus accuracy describes a ratio between two scores; it does not mean the router answered 98.1% of tasks correctly.
The router’s cache-hit rate was 94.2%, against 97.8% in the control. Those figures show that some cache reuse was sacrificed while total spending fell. They do not isolate how much of the saving came from cache awareness rather than switching to cheaper models. A router without cache awareness would be a useful additional comparison.
Session economics need session-level tests
Most traffic in this experiment came from Fireworks’ own coding work, where routine turns offered opportunities for open models. The result should not be carried directly into a medical assistant, a document pipeline or another company’s coding workload. Their context lengths, repeated prompts and tolerance for corrections can differ substantially.
A deployment test also needs to preserve the session. Randomly scattering isolated prompts between systems can miss cache persistence and the extra turns required after a weak answer. The relevant bill includes completing the job, recovering from errors and meeting the user’s time expectations.
FireRouter makes that broader bill an explicit product concern. The next competition in model serving may involve providers deciding when to keep useful computation in place, rather than winning every turn with the lowest sticker price. Buyers can now test whether those decisions save money on their own histories, with the accuracy trade-off stated alongside the saving.
References
Questions, news tips or corrections? Contact [email protected].
