A support queue does not always need another paragraph. It needs a defensible choice about which team should receive a request, and a way to recognize cases that should go to a person. That makes uncertainty part of the workflow rather than an optional explanation appended after the decision.
Baseten’s October 21 release introduces mammoth-1 and elephant-1, post-trained from Qwen3.5 models with 27 billion and nine billion parameters respectively. The company says it used 6.21 million decision examples. Both return probabilities over bounded answers rather than generating an unconstrained response. Baseten reports strong results on the Jev Decision Index and Artificial Analysis Decision Index; those are named evaluation results, not evidence that every customer queue can be automated safely. Read the primary source.
Confidence needs a local meaning
A probability of 0.9 is useful only if decisions receiving that score are correct at an appropriate frequency on the relevant workload. A threshold selected on easy examples may fail when product terminology, languages or the balance of rare cases changes. Measuring accuracy without the corresponding escalation rate would hide how much work still falls to people.
A practical evaluation should keep a reviewed set of difficult and ambiguous requests, compare calibration across task groups, and count both wrong automatic decisions and unnecessary escalations. Latency and infrastructure cost should include the fallback path. These are proposed acceptance checks, not tests independently performed on the released models.
Open availability gives teams a concrete candidate to compare with their existing classifier or generative model. Over time, bounded decision interfaces could simplify high-volume routing, provided confidence remains reliable as inputs change. The next useful evidence is workload-specific calibration and sustained results after deployment, rather than a leaderboard rank alone.
References
Questions, news tips or corrections? Contact [email protected].
