Deidentification and suppression before releasing agency data

Reference article · 5 min read

A request arrives for data the agency already holds: incident reports, disease counts by ZIP code, absenteeism by school, or open caseloads by region. The hard question is what can be released without exposing an individual, and how to answer that the same way every time.

Deidentification is not one step of deleting names. It is a set of decisions about which fields identify people, how coarse the numbers should be, which cells are too small to show, and what else is already public. Write each decision down, because the agency will be asked to explain the method later.

Direct identifiers and indirect identifiers

Two kinds of fields matter: those that name a person outright, and those that narrow the population until one person is left.

Direct identifiers. Name, Social Security number, address, phone, email, medical record number, case or booking number, license plate, photograph, exact coordinates. Removing the columns is necessary and never sufficient, and free text narratives usually still contain them.
Indirect identifiers, or quasi-identifiers. Date of birth or exact age, sex, race and ethnicity, ZIP code or census tract, exact date and time of an event, school, employer, unit of assignment. Individually harmless. In combination, a few can single out one person.
Sensitive attributes. The diagnosis, finding, or status the release is about, which sets how much harm a reidentification causes.

Aggregation is the primary control

Most release questions are answered better by counts than by records. Aggregation publishes the number of events in a group rather than a row per event, and the group is chosen deliberately: county rather than tract, quarter rather than exact date, age bands rather than exact age, broad category rather than specific code. Pick the coarsest level that answers the question asked, not the finest the system can export.

Small cell suppression and complementary suppression

Aggregated tables still leak. A cell holding one or two people, in a table crossed by geography and demographics, points at identifiable individuals.

Primary suppression. Withhold any cell whose count falls below a stated threshold. Some federal and state programs set a threshold in rule or guidance, and where one exists it governs. Otherwise the agency sets one in policy and applies it consistently.
Complementary suppression. If a total is published with all but one of its cells, the suppressed value can be recovered by subtraction, so a second cell in that row and column must also go. Multi-dimensional tables need this checked systematically, not by eye.
Small denominators and complete cells. A rate computed on a handful of people discloses as much as a count, and a cell equal to its group total says the attribute applies to every member. Suppress both.

Rounding, and what it does not fix

Rounding counts to a fixed base, controlled or random, lowers precision without leaving visible holes. It is useful and weaker than it looks: rounded figures often fail to add to their totals, forcing them to add can reveal true values, and repeated rounded releases can be averaged back toward the truth. Top and bottom coding collapses extreme values into one band.

Reidentification risk comes from combination

A release is never evaluated alone. Assess it against what someone could place beside it.

Other public records. Voter files, property records, court dockets, permits, obituaries, news coverage. A date, a ZIP code, and an age band can match one name.
Your own prior releases. Monthly files that sum to a published annual total, a dashboard whose counts shift when one case is added, last year's table at a different geography. Differencing stays invisible unless someone logs what has gone out.
Small populations. Rare conditions, small districts, and small units make ordinary attributes identifying. A table can be safe statewide and unsafe in one county.
Interactive tools. A dashboard that lets users filter and export is a query system. Suppression must apply after filters combine, and to exports and APIs.

When to aggregate further and when to withhold

Treat full release and denial as the ends of a ladder: broaden the geography, broaden the time period, collapse categories, suppress cells, publish counts instead of rates, or answer in narrative form. Restricted access under a data use agreement may serve when publication cannot. Withhold when suppression would empty the table, or when any cross tabulation of a small population identifies people.

Keep the legal question separate from the technical one. Deidentification does not by itself make a confidential record releasable, and public records law may compel release regardless of method. Requirements differ by state and data type and they change, so confirm with counsel first.

Making it repeatable, and what to ask vendors

Put the method in a short written release standard: identifier inventory, threshold, suppression and rounding rules, review roles, and who signs off. Log what was published and at what granularity. Where a reporting product or open data portal sits in the release path, ask the vendor:

How suppression is configured, whether the threshold is agency settable, and whether complementary suppression runs across every dimension of a table.
Whether suppression holds on filtered views, drill downs, exports, scheduled reports, and API responses. Ask for a demonstration, not a statement.
What documentation the product produces that you can attach to the release record.

In the contract, require a prohibition on attempting reidentification, limits on redisclosure, deletion at termination, incident notification, audit rights, and the right to test suppression before go live.

HQT builds and supports data and reporting systems for state and local agencies, including release workflows where suppression rules must be applied consistently.

Frequently asked questions

Is there a standard minimum cell size for public data releases?

There is no single national threshold that applies to every release. Some federal and state programs set a threshold in rule or guidance for their own data, and where one applies it governs. Where none applies, the agency picks a threshold, writes it into policy, and uses it the same way on every release so the method can be explained and repeated. Requirements change, so confirm the applicable rule with the program's official source and with counsel.

Does removing names and Social Security numbers make a dataset deidentified?

No. Direct identifiers are only part of the problem. Combinations of indirect identifiers such as exact date of birth, ZIP code, sex, and the exact date of an event can single out individuals, especially in small populations, and free text narrative fields often still contain names and addresses. A defensible method also addresses granularity, cell sizes, and what else is already public.

Our dashboard lets the public filter the data. Do suppression rules still apply?

Yes, and they are harder to enforce there. A dashboard that allows filtering is effectively a query system, so a combination of filters can produce a cell far smaller than anything visible on the default view. Require that suppression run after filters combine and that it also apply to drill downs, exports, scheduled reports, and API responses. Ask the vendor to demonstrate that behavior rather than describe it.

We published a monthly table and an annual table. Why is that a problem?

Related tables can be differenced. If a suppressed monthly cell can be recovered by subtracting the published months from a published annual total, the suppression did not work. The same happens across geographies, across years published with different groupings, and across versions of a dashboard. Keep a log of what has already been published so each new release can be checked against it.

When should we withhold data rather than aggregate further?

Withhold when suppression would remove most of the table anyway, when the population is small enough that any cross tabulation points at individuals, or when the underlying records are confidential by statute. Deidentification does not by itself convert a confidential record into a releasable one. Disclosure duties can also run the other way, since public records law may require release, so treat the legal analysis as separate from the statistical one and involve counsel before responding to the requester.