Public Sector
The questions that expose a weak architecture before award
A question set for reviewing a vendor software architecture before award, and the answers that show a design has not been thought through.
Agencies buy architecture more often than they buy code. A proposal arrives with a diagram, a bill of materials, and a hosting story, and someone has to decide whether the system described will still be serviceable years after award. That decision usually falls to the people who will operate the system but who did not design it. A structured architecture review gives those people a way to interrogate a design while it is still a proposal.
We use the question set below on our own reference builds. The questions matter less than the shape of the answers they produce.
Start with the work, not the diagram
A diagram shows components. It does not show whether the system does the job. The first questions are about the workload: which users, which transactions, at what volume, and at which times of year. Property tax billing, elections, permit deadlines, and open enrollment produce load that looks nothing like an average day. A good answer names specific workflows and specific peak periods, and ties them to a sizing assumption.
Trouble is a description of the technology stack offered in place of a description of the work. If nobody can say what the busiest hour looks like, nobody has sized anything.
Where the data lives and who can reach it
Ask for a data inventory before asking about servers: what is stored, what is sensitive, and which records carry separate handling rules. Criminal justice information, protected health information, student records, and state tax confidentiality statutes each impose their own requirements, and one system often holds more than one category. Then trace every path to that data, including vendor support access, backups, analytics pipelines, log aggregation, and any AI or search feature that sends records to another service.
Trouble sounds like "the data stays in our secure cloud." That is a brand claim, not an architecture. Ask for the hosting region, the tenancy model, who holds the encryption keys, and a direct answer about whether vendor staff can read agency records in production, under what process, and with what audit trail. Cloud authorization programs such as FedRAMP, and the state and local programs modeled on it, are useful context, but an authorization covers a defined service boundary rather than an entire company or an entire design. Read the boundary, and check whether the components the proposal depends on sit inside it.
What happens when something fails
Ask what breaks when the identity provider is unavailable, when the payment processor times out, when the database fails over, and when an integration partner returns bad data for most of a working day. A good answer is boring and specific: this queue absorbs it, this retry policy applies, this screen tells the counter clerk what to do, this runbook covers the rest.
The answer that signals trouble is a recovery time objective quoted with no exercise behind it. Recovery time and recovery point objectives in a proposal are commitments. Ask when failover was last rehearsed and what the rehearsal found. A design that has never been tested against failure is a hypothesis.
How a change reaches production
Architecture decays through the change process, so ask to see the path from a developer workstation to production: version control, automated tests, dependency and vulnerability scanning, environment promotion, approvals, and rollback. Ask specifically how a security patch to a base image reaches a running system, and how long that normally takes.
Trouble sounds like a reference to DevOps practice with no pipeline named, or a deployment that requires a person to log into a server and copy files. Test coverage described but never demonstrated belongs in the same category.
What the agency owns at the end
Every design encodes an exit, whether or not anyone wrote it down. Ask who owns the source code, who owns the data model, what the export format is, whether an export includes documents and audit history as well as table rows, and what the system looks like if the vendor relationship ends. Building custom logic inside a proprietary runtime can be a reasonable decision. It should be a knowing one.
The answer that signals trouble is migration described as a service the vendor would be happy to quote, rather than as a documented capability of the product.
The general shape of a weak answer
- Confidence without artifacts: no sizing worksheet, no threat model, no test results.
- A diagram with no numbers on it.
- Everything described as configurable, which often means everything is custom.
- Compliance answered with a certificate instead of a scope statement.
- No named owner for operations after go-live.
Putting the questions to use
These questions work best inside the solicitation, where vendors answer them in writing and every bidder answers the same thing. Ask for artifacts by name rather than for assurances, and give the reviewer access to the people who designed the system rather than only the people who sell it. Treat the result as a list of risks to manage rather than a verdict, because most designs come back with findings. The useful outcome is knowing which findings the agency is accepting on purpose, and writing the ones that matter into the contract.