Pilot and proof of concept agreements: scope, cost, and exit

Reference article · 6 min read

A pilot is a purchase of information. The agency spends staff time, and sometimes money, to learn whether a product does what the vendor says it does, in the agency's own environment, with the agency's own data and users. That is a reasonable thing to buy. Trouble starts when the pilot is documented loosely, runs past its intended end, accumulates real records, and quietly becomes the production system that nobody competed.

The fix is not to avoid pilots. It is to write a short agreement that states the question being answered, the conditions for answering it, and exactly what happens on the last day. Most of the terms below fit in a few pages and can sit under an existing master agreement, a small purchase authority, or a standalone evaluation agreement, depending on your jurisdiction's rules.

Start with the question the pilot has to answer

A pilot should test something that a demonstration and a reference call cannot settle. Common examples: whether the product can read your actual data without heavy cleanup, whether it integrates with a specific system of record, whether field staff will use it without constant support, or whether performance holds at your transaction volume.

Write the success criteria before anything is installed, and have both the program owner and IT sign them. Useful criteria include:

• The measure. What will be observed, in plain terms, such as records processed without manual correction or time from intake to disposition.
• The baseline. How the current process performs today. Without this, any result can be argued either way.
• The threshold. What counts as good enough to proceed, and what counts as a clear failure.
• The judge. Who reviews the evidence and makes the recommendation, and by what date.
• Permission to fail. State in writing that a negative result is an acceptable outcome and ends the engagement.

Keep the pilot from becoming a sole source justification

The most expensive pilot mistake is procedural. After months of configuration, the argument writes itself: switching now would waste the investment, so award it to the incumbent. Sole source authority in most jurisdictions turns on whether a good or service is genuinely available from only one source, not on convenience or prior familiarity, and rules differ by state and locality. Confirm the standard with your purchasing authority and counsel before assuming a pilot supports it.

Two protections help. First, state in the agreement that the pilot creates no expectation of award, no exclusivity, and no obligation to purchase. Second, decide up front whether the pilot vendor may help draft requirements for a later solicitation. Many agencies exclude that vendor from specification writing, or make all pilot artifacts available to every bidder so no competitor is disadvantaged.

Data handling, security, and deletion

Use synthetic or de-identified data when the question can be answered that way. When production data is necessary, limit it to the minimum records the test requires, and confirm whether any of it carries obligations under criminal justice, health, education, or tax confidentiality rules. Those regimes change, so verify current requirements with the data owner and counsel rather than relying on a prior project's paperwork.

Put these in writing:

• No secondary use. The vendor may not use agency data to train models, build benchmarks, or develop products without separate written permission.
• Named subprocessors. List who else touches the data, including hosting and support locations.
• Deletion and certification. Set a deadline after the end date for deletion of agency data from production and backups, with written certification of completion.
• Records obligations. Assume pilot outputs may be public records subject to disclosure and to your retention schedule. A pilot does not suspend either.

Who owns configuration and outputs

Agencies often discover at the end that the workflows, rules, mappings, report definitions, and tuning they built are treated as vendor property. Specify that agency data and any outputs derived from it belong to the agency, and that the agency may keep and use those outputs whether or not it proceeds. Require a full export in a documented, non-proprietary format before the end date, not on request afterward. The vendor keeps its underlying software and prior intellectual property, which is fair, but the agency's operational knowledge should not become a switching cost.

Price protection if the agency proceeds

Attach a pricing schedule to the pilot agreement at signing, valid for a stated period after the pilot ends. Lock the definition of the billing unit, whether that is named users, devices, cases, transactions, or API calls, since redefining the unit is how a favorable rate becomes unfavorable. Get implementation, data migration, interface, and training costs quoted at the same time. A free pilot with unquoted conversion costs is not free.

A clean end date with no automatic conversion

The agreement should end on a fixed date with no renewal or conversion language of any kind. Allow at most one extension, in writing, with a stated reason and a new end date. Before that date, run an offboarding checklist: export data, disable accounts and integrations, remove agency credentials from vendor systems, and confirm deletion. Close with a short decision memo recording what was tested, what was observed against the criteria, and the recommendation, because that memo is the record a future auditor will ask for.

What to require and what to ask

Before signing, ask the vendor these questions and keep the answers with the file:

1. What exactly is out of scope during the pilot, and what would require a change order?
2. What agency staff time do you assume, by role and hours per week?
3. Where does our data reside, who administers it, and who else can access it?
4. What does conversion to production cost, including migration and interfaces?
5. What happens to our configuration and exports if we do not proceed?

Require, at minimum: written success criteria, a fixed end date, no automatic renewal, no exclusivity or award expectation, data deletion with certification, export rights in a documented format, and a fixed pricing schedule for a defined window.

HQT works with state and local agencies on software development, integration, and technology advisory work, including the scoping of evaluations that precede a full procurement.

Frequently asked questions