How to plan a data migration you can defend

Data migration consumes more effort than almost anyone plans for, and it is the part of a system replacement most likely to produce a dispute after go-live. The defense is a documented mapping, repeatable runs, and reconciliation evidence that someone outside the project can follow.

Treat conversion as a workstream with its own schedule and owner, not as a task at the end of implementation.

Steps

1. 1Profile the source data before designing anything

   Count records, check completeness by field, find duplicates, and identify values that violate the rules the new system will enforce. Expect surprises in historical data entered under older practices.

2. 2Decide what actually moves

   Not all history belongs in the new system. Agree with the business and with records staff what converts, what is archived in a readable form, and what is destroyed under the retention schedule. Document the decision.

3. 3Write the field-level mapping

   For every target field, record the source, the transformation, the default when the source is empty, and the rule for values that do not fit. This document is the contract between the agency and the implementer.

4. 4Resolve identity and duplicates

   Decide how the same person, parcel, account, or asset appearing in several sources becomes one record. Agree the matching rules and who adjudicates the cases the rules cannot decide.

5. 5Build the conversion as repeatable code

   Scripted and version controlled, so a run can be repeated identically. Manual cleanup that is not captured in code cannot be reproduced at cutover.

6. 6Run trial conversions on a schedule

   Several full rehearsals, each followed by business validation. Track defect counts per run; the trend tells you whether cutover is realistic.

7. 7Reconcile with evidence

   Record counts, control totals for financial values, and targeted spot checks by the staff who know the data. Keep the reconciliation output as the acceptance record.

8. 8Plan the cutover window

   Freeze source changes, run the final conversion, reconcile, and decide go or no-go against criteria agreed in advance. Write the rollback procedure and confirm who can invoke it.

9. 9Keep the source readable after cutover

   Retain a queryable copy or an exported archive of the source system for the period the records schedule requires, and document how to read it.

What you end up with

A documented mapping, repeatable conversion code, reconciliation evidence, and an archive of the source. If anyone asks in two years why a record looks the way it does, the answer exists.