AI & Automation
Retrieval-augmented generation: AI assistants that cite agency sources
How an assistant can answer questions from the municipal code, policies, and records without inventing answers.
A language model on its own answers from what it learned in training, which does not include your agency's code, policies, or contracts, and it may produce confident answers that are wrong. Retrieval-augmented generation solves this by finding the relevant passages in the agency's own documents first and having the model answer only from those, with citations.
How it works
- Agency documents are split into passages and indexed, including a vector index that captures meaning rather than only keywords.
- A user asks a question.
- The system retrieves the passages most relevant to the question.
- The model is instructed to answer using only those passages and to cite them.
- The answer and citations are returned, and the interaction is logged.
Why it fits agencies
The knowledge lives in documents that change: ordinances, policies, procedures, fee schedules. Updating the index is far simpler than retraining anything. Citations let staff verify the answer against the source, which is the behavior you want in a public setting.
Typical agency uses
- Staff assistant over the municipal code and policy manuals
- Contract and procurement file search with question answering
- Internal IT and HR help desk over knowledge base articles
- Public-facing assistant over published web content, with strict grounding
- Research support for records and legal staff
Design decisions that matter
Source scope. Only approved, current documents, with a process for adding and retiring them.
Access control. The assistant must respect document permissions; a user should not get an answer from a document they cannot open.
Grounding strictness. If the sources do not answer the question, the system should say so rather than guess.
Evaluation. A test set of real questions with known-good answers, scored before launch and periodically after.
Hosting. For sensitive content, models and indexes run inside the agency's cloud boundary. For public content, managed services may be appropriate.
Logging and records. Prompts and answers are records; retention and public records handling are decided up front.
What a procurement should ask for
Citations on every answer, refusal behavior when sources are silent, permission-aware retrieval, evaluation results, hosting and data handling details, and documentation. Vendors that cannot describe these are selling a demo.