Acceptance criteria that a tester can execute and a contract can rely on

March 4, 2026 · 5 min read · HQT

"The system shall work as intended" survives every review because nobody objects to it. It also cannot be tested, cannot be failed, and cannot be enforced. When a deliverable arrives and the agency believes it is not finished, the acceptance criteria settle the argument, and that sentence gives the agency nothing to stand on.

Acceptance criteria are the point where a requirement turns into a test. We write ours before we build, because the exercise surfaces disagreements that would otherwise appear during user acceptance testing, when they cost more to fix.

Why most criteria cannot be enforced

Weak criteria usually fail in one of four ways.

No observable result. "Shall be user friendly" describes an impression, not an output. Two reasonable reviewers will disagree and both will be right.
No actor or starting state. "Reports can be generated" does not say by whom, with what permissions, or from what data.
No threshold. "The page shall load quickly" has no number, no measurement point, and no stated conditions.
No method. "The application shall be accessible" names no standard, no tool, and no review procedure.

Hand the sentence to someone who has never seen the project and ask for a pass or a fail. If they have to ask a question first, it is not finished.

The shape of something a tester can run

We write criteria as preconditions, action, observable result. Each one names the role, the state of the data beforehand, the step taken, and what must be true afterward, including side effects such as records written and notifications sent.

A criterion in that form reads like this: given a clerk account with the records role and a case in closed status, when the clerk searches by case number and selects export, the system produces a file containing the fields listed in the appendix and writes an audit entry with the user, timestamp, and case identifier.

That is longer than "the system shall support case export." Length is not the problem in acceptance criteria. Ambiguity is.

Put numbers in, and attach a method to each one

Performance and availability belong in criteria, but a number without a method is unenforceable. A response time target needs a percentile, a concurrent load, a named environment, and a measurement point, because timing at the server and timing in a browser on an agency workstation produce different answers. An availability target needs a measurement window, a definition of an outage, and a rule for maintenance.

Choose numbers the agency is willing to measure and pay for. Every threshold is a cost driver, and one nobody measures is one nobody meets.

Non-functional criteria deserve the same discipline

Security and accessibility are where vague language does the most damage, because both carry obligations that stay with the agency whatever a vendor asserts.

For accessibility, name the standard, its version, and the conformance level. Federal agencies buy against Section 508 of the Rehabilitation Act, whose implementing standards incorporate WCAG 2.0 Level AA. State and local governments are covered by the Department of Justice rule under Title II of the Americans with Disabilities Act, which sets WCAG 2.1 Level AA for web content and mobile applications, with compliance dates that differ by the size of the entity. Then name the evidence: an automated scan with no critical findings, plus a manual keyboard and screen reader review of named workflows, since automated tools catch only part of the standard.

For security, name the scan, the severity levels that block acceptance, the remediation window, and the events that must appear in the audit log. If the system has to meet a control baseline such as NIST SP 800-53, cite the controls and say which artifact demonstrates each one. "Follows industry best practices" is not something a tester can execute or a contracting officer can invoke.

Criteria a contract can rely on

Testability is necessary and not sufficient. To carry weight, criteria need mechanics around them. A criterion that cannot be failed is a preference.

Traceability. Each requirement maps to criteria, each criterion to a test case, and each test case to the deliverable and payment milestone it supports.
Execution. State who runs the test, in which environment, and against what data. Agency staff should execute at least the criteria tied to payment.
Evidence. State what the vendor submits: test results, scan output, and review notes.
Failure handling. State the retest window, the cure period, and what happens to the milestone when a retest fails.
Review time. If the contract treats a deliverable as accepted when nobody responds within a set number of days, reserve staff time inside that window.

How we do it on our own builds

On internal exercises and reference builds, criteria come before implementation and serve as the definition of done. Someone other than the developer runs them against synthetic data, and whatever can be automated moves into the pipeline.

What to do next

Read the criteria in your next solicitation and strike every sentence that cannot produce a pass or a fail. Ask bidders to propose measurable replacements, and treat the answers as evidence about how they work. Keep the criteria in the contract rather than in a kickoff deck, and reserve tester time in the schedule.