Public Sector
Five terms we think every agency AI contract should include
Short, specific contract language that protects agencies buying AI-enabled software.
We read a lot of contracts for AI-enabled products. Most were written for ordinary software and miss the issues that AI raises. These five terms address them, in language an agency can adapt.
1. No training on agency data
"Contractor shall not use Agency data, including prompts, documents, and outputs, to train, fine-tune, or improve any model, whether for Contractor or third parties, without Agency's prior written consent."
Without this, the agency's records may become part of a vendor's product.
2. Notice and testing before model changes
"Contractor shall provide at least thirty days' written notice before any material change to the models or algorithms used in the Service and shall provide test results demonstrating performance on the agreed evaluation set."
Models change under the hood. The agency should know and should see evidence that the change did not degrade results.
3. Performance measured on agency data
"Contractor's performance commitments shall be measured against the evaluation set provided by Agency, using the methodology in Exhibit X, and reported quarterly."
Benchmark numbers from marketing do not describe your documents.
4. Explanation and audit support
"For any output that informs a decision affecting an individual, the Service shall provide the sources or factors on which the output was based, and Contractor shall support Agency in responding to audits, appeals, and public records requests concerning such outputs."
Agencies must be able to explain their decisions.
5. Disclosure rights
"Agency may publicly disclose its use of the Service, the purposes for which it is used, and the general nature of the models involved."
Transparency obligations are growing, and a vendor should not be able to block them.
Also worth including
Data location and deletion, security requirements including prompt injection testing, accessibility conformance, and a defined human-review role.
These are starting points, not legal advice. Agency counsel should adapt them. But every one addresses a problem we have seen in a real contract.