Entity resolution: matching people and businesses across agency systems

Reference article · 6 min read

Agencies rarely have a single identifier that follows a person or business from one system to the next. A resident who pulls a building permit, opens a water account, receives a code enforcement notice, and appears in a case management file usually exists as four unrelated records, created by different staff, at different times, under slightly different spellings. Social Security numbers are restricted and often absent. Driver license numbers are not collected everywhere. A business may be registered with the state under a legal name that local permitting systems never captured because the applicant wrote the trade name instead.

Entity resolution is the work of deciding which records refer to the same real thing: the same person, business, parcel, or vehicle. It is not a one-time data cleanup project. It is a standing process with rules, scores, thresholds, human review, and an audit trail, and it needs a named owner in the same way a system of record does.

Define the entity and the decision before you match

Match quality is judged against a purpose, not in the abstract. Linking utility accounts to parcels for a delinquency report tolerates different error than linking case records for an eligibility determination or a code enforcement action. Write down the entity type, the systems in scope, and the specific decision the linked view will support. Then look for anchors that already exist, because a real identifier beats any amount of clever scoring:

• Parcel. The assessor parcel number is usually the strongest anchor a local government has, and permitting, code enforcement, and utility systems often carry it already, sometimes in a free text field.
• Address. Use an authoritative address point or site address file rather than the strings typed into each application. Unit designators, rural route addresses, and new subdivisions are the usual trouble.
• Vehicle. VIN is strong, plate is weaker because plates transfer between vehicles and owners over time.
• Business. The state entity registration number, where captured, plus the registered agent address, beats name matching on its own.

Deterministic matching: rules you can explain

Deterministic matching applies explicit rules. Normalize first: case, punctuation, whitespace, nicknames, suffixes such as Jr and III, parsed address components, and a consistent date format. Then define rules in order of strength, for example an exact match on a normalized anchor, or an exact match on last name plus date of birth plus address point.

Deterministic rules are fast, auditable, and easy to defend when someone asks why two records were joined. They are brittle against typographical errors, transliteration, name changes after marriage, and shared household attributes. Most agencies should keep a deterministic layer for the high-confidence cases and send the rest onward rather than trying to write a rule for every variant.

Probabilistic scoring: weighing agreement and disagreement

Probabilistic record linkage, in the tradition of the Fellegi and Sunter model, scores each candidate pair field by field. Agreement on a field contributes positive weight, disagreement contributes negative weight, and missing values contribute little or nothing. The weight depends on how often a field agrees among true matches and how often it agrees by chance among non-matches, which is why agreement on a rare surname is stronger evidence than agreement on a common one, and why agreement on a city in a small jurisdiction carries almost no weight at all.

Candidate pairs are generated by blocking or indexing, since comparing every record to every other record does not scale. Blocking choices are a quiet source of missed matches: if you block on ZIP code, you will never find the pair where the ZIP was mistyped.

Thresholds and clerical review

Scoring produces a continuum, not an answer. Set an upper threshold above which pairs are linked automatically, a lower threshold below which they are rejected, and treat the band between them as a clerical review queue. Set those thresholds by consequence rather than by a single accuracy figure, and expect to tune them per use case. A review queue only works if it is staffed, sized to what staff can actually clear, and equipped with the context reviewers need: source records side by side, the field level comparisons, and the ability to record a reason.

The two failure directions

• False merge. Two people become one record. Someone sees another person's utility history, medical or benefits detail, or enforcement record. This is a disclosure event, and depending on the data involved it may trigger obligations under state breach notification law or program rules governing health, education, tax, or criminal justice information. Requirements vary by data type and jurisdiction, so confirm with your privacy officer or counsel.
• False split. One person or business stays fragmented. The repeat code violator looks like three first-time violators, duplicate benefits go undetected, the pattern behind a series of complaints never surfaces, and counts used for planning are inflated.

Neither error is universally worse. For eligibility, enforcement, and public safety uses, a false merge usually carries more harm. For outreach or workload forecasting, a false split is often the costlier one.

What to keep in the match audit log

Store links rather than overwriting source records, so every merge can be reversed. For each decision, retain:

• Source system, record identifier, and the version of the record as compared.
• The rule set or model version and its configuration, including thresholds in effect at that moment.
• Field level comparison outcomes and the resulting score.
• The decision, whether it was automatic or manual, the reviewer identity for manual decisions, and a reason code.
• The full merge and unmerge history for the resulting cluster identifier.
• A retention period consistent with your records schedule, since match logs may themselves be public records.

What to ask vendors and how to start

Ask how candidate pairs are generated and what blocking is used. Ask whether rules, weights, and thresholds are visible and editable by agency staff. Ask what the clerical review interface looks like and who is expected to staff it. Ask how unmerge works and what happens downstream to records already created under a merged identity. Ask what the system does with missing fields, and whether it produces an explanation for each individual link.

In contract terms, require that rules, weights, thresholds, and the resulting match table are agency owned and exportable in an open format, that audit logs are retained and retrievable, that the match logic is documented in writing, and that the agency may sample and audit decisions at any time.

To start, pick two systems and one decision. Assemble a sample of pairs and have staff who know the data label them by hand. Measure both error directions against that sample, tune thresholds, then widen the scope. Resist the temptation to launch across every system at once.

HQT works with state and local agencies on data integration, record linkage, and analytics engineering.

Frequently asked questions