Automating document intake: classification, routing, and exception queues

Reference article · 6 min read

Every agency receives documents it did not design. Paper mail arrives at a central address, residents email PDFs and phone photographs to a general inbox, and online portals accept uploads that may or may not be the form the page asked for. Somewhere behind all of this, a person opens each item, decides what it is, and decides who should have it. That decision is usually fast, usually correct, and almost never recorded. When volume rises or staff turn over, the decision becomes the bottleneck, and the only visible symptom is that constituents wait longer without anyone being able to say where the time went.

Automated document intake replaces the routine part of that judgment with a classifier and a routing rule, and keeps the hard part with a human. Done well, it is unglamorous infrastructure: most items land in the right queue without anyone touching them, a minority land in a review queue with a reason attached, and the agency can see both numbers. Done poorly, it creates a new invisible backlog in a place nobody owns. The difference is mostly in design decisions made before procurement.

What classification actually decides

Classification answers one narrow question: what kind of document is this. Not what it says, not whether it is complete, not whether the applicant qualifies. Keeping the scope narrow is what makes the rest tractable.

Practical systems usually combine several signals:

• Layout and visual structure. Standard agency forms have recognizable shapes. A model trained on your own forms is generally more reliable than one trained on generic document types.
• Text after OCR. Form numbers, titles, and header language are strong indicators, when the scan quality supports them.
• Channel and context. A file uploaded to a specific portal step, or emailed to a dedicated address, carries information the document itself may not.
• Deterministic rules. Some cases should never be probabilistic. If a barcode or form number is present and readable, honor it and skip the model.

Separate classification from extraction in your design and in your contract. A system can be good at telling a business license renewal from a zoning appeal and still be unreliable at pulling a parcel number off a fax. Conflating the two makes accuracy claims meaningless.

Confidence thresholds and the straight-through decision

Most classifiers return a score alongside a label. The threshold is where that score becomes a decision, and it is a policy choice, not a technical one. Set it high and more work lands on humans. Set it low and more misroutes reach downstream staff who may not notice for days.

Two things make thresholds manageable:

1. Set them per document type, not globally. A type with distinctive layout and low consequence tolerates a lower bar than one that triggers a statutory clock or involves protected information.
2. Let program staff adjust them. Thresholds should be a setting an authorized staff member changes, with the change logged, not a ticket to the vendor.

Some document types should never go straight through regardless of score. Anything that starts a legal deadline, involves a minor, or carries records with heightened confidentiality is a reasonable candidate for mandatory review. Decide that list with your records officer and counsel, and note that retention and confidentiality obligations vary by state and record type.

Routing rules staff can edit

The routing layer maps a classification to a destination queue. This is where agencies most often lose control. Rules embedded in vendor-maintained code mean every reorganization, every new form, and every seasonal reassignment becomes a change request with a cost and a wait.

Insist that rules live in configuration that authorized staff can read and change, and that the design supports:

• Plain-language rule expression. A program manager should be able to read the rule set and tell whether it matches current practice.
• Versioning and audit. Who changed which rule, when, and what it was before.
• Queue ownership by role, not person. Routing to a named individual breaks when that person leaves.
• A test mode. Run proposed rules against recent traffic and see what would have changed before committing.
• A defined fallback. Every unmatched document goes somewhere specific and owned.

Designing the exception queue

The exception queue is the part that determines whether the system helps or quietly hurts. An exception queue with no owner, no aging visibility, and no service expectation becomes a place documents go to be forgotten.

Build it with the same care as a primary queue:

• Named ownership and coverage. A role owns it, with backup for absence.
• Visible aging. Oldest item first, with the age displayed, not buried in a report.
• Reason codes on entry. Low confidence, unreadable scan, ambiguous between two types, and unrecognized type are different problems with different fixes.
• Escalation on age. Items past a defined threshold surface to a supervisor automatically.
• Feedback into the model. Every human correction is labeled training data. If corrections do not flow back, accuracy stays flat.

Watch for the opposite failure too. A queue that is always empty may mean thresholds are set too low and errors are moving downstream silently.

Measuring misroutes

Classifier accuracy measured on a vendor test set is not the number you need. The operational number is the misroute rate: documents that reached a queue and had to be moved.

Track it by capturing every reroute as an event with the original label, the corrected label, and who corrected it. Then review the confusion pairs, since misroutes cluster rather than scatter, and two types that look alike will account for much of the volume. Also track the lag between arrival and correction, because a misroute found in an hour is an annoyance and one found in three weeks may be a compliance problem. Sample straight-through items periodically as well, since misroutes nobody noticed never generate a correction event.

Questions to ask before you buy

• Can our staff create a new document type and its routing rule without vendor involvement? Ask for a live demonstration, not an assurance.
• How are confidence thresholds configured, by whom, and is the change logged?
• What happens to a document the classifier has never seen? Show the path.
• Are human corrections captured as training data, and do we own that data and the extracted text?
• What does the exception queue look like with a hundred aged items in it?
• If we leave, what do we take with us: rules, labeled corrections, documents, and audit history, in what format?

Require in the contract that configuration is exportable, that correction data belongs to the agency, and that accuracy reporting is available to you continuously rather than on request.

HQT works with state and local agencies on document intake, workflow automation, and the integration work that connects intake systems to existing records platforms.

Frequently asked questions