AI
Small open models are good enough for many agency tasks
The largest model is not always the right one. For classification, extraction, and grounded answering, smaller models running in the agency's own environment often do the job.
There is an assumption that any AI project needs the largest, newest model from a major provider. For many agency tasks, it does not. We evaluate smaller open-weight models first, and they frequently win on the combination of accuracy, cost, speed, and control.
Where smaller models do well
- Classifying service requests, emails, and documents into known categories
- Extracting fields from forms and reports
- Answering questions grounded in retrieved agency documents
- Summarizing documents of moderate length
- Detecting personal information for redaction
These tasks have a defined scope and can be measured. A model that scores well on the agency's test set is good enough, whatever its size.
Where larger models earn their cost
Long, complex reasoning; drafting that requires broad knowledge; handling highly varied inputs without task-specific tuning. Even then, the largest model is not automatically the best; evaluate.
Why it matters for agencies
Control. Open models run in the agency's environment, in the region and under the controls the agency chooses, with no data leaving.
Cost. Self-hosted or provider-hosted open models can be much cheaper per request at agency volumes.
Stability. The agency decides when the model changes.
Latency. Smaller models respond faster, which matters for interactive use.
How we evaluate
A test set from the agency's real data with known answers. Several candidate models, including open and commercial. Accuracy, latency, and cost per thousand requests measured. The results, not the brand, decide.
For procurement
Ask vendors which model a product uses, whether it can be swapped, and whether the agency can run it in its own environment. Products that hard-wire one commercial model have a dependency the agency should understand.