The case for small models in the field
A model that runs on the device you already own beats a better model you cannot reach.
The frontier of model capability and the frontier of deployable capability have almost nothing to do with one another. A clinic with six hours of no signal per shift cannot call an API, however good the model behind it. The relevant question is never which model is best, but which model fits.
Quantised speech, translation and vision models now run acceptably on a mid-range tablet. Acceptably is doing real work in that sentence: a small model that answers in the local language, offline, in two seconds, is more useful than a large one that answers beautifully when a signal happens to exist.
The question is never which model is best, but which model fits.
There is a second argument, which is cost structure. An on-device model has no per-token cost, which means an organisation can offer the thing to everyone rather than rationing it to the cases that seem to justify a call. Rationing is invisible in reports and enormous in practice.
The tradeoffs are real and worth naming. Small models are narrower, drift without curation, and need a maintainer for the update path. All three are solvable by ordinary engineering; none are solved by choosing a larger model.
Our default is therefore the smallest model that clears the task, running on hardware the organisation already owns, with a cloud path as an enhancement rather than a dependency.