Skip to main content

Search the Commons

Type at least 2 letters. Use the arrow keys to move through results, Enter to open, Escape to close.

Ask a question instead: answers drawn from across the Commons, with links to the source. Or browse the site map.

Manage and improve · Legacy and modernisation

Don't carry data debt into the new system

Legacy systems hold much of the public sector's most important data, and much of its data debt. Modernisation is the best chance to fix it, and the easiest moment to lose it. Find your data debt, put data quality gates into the migration, and prove the move with reconciliation.

In short: a plain English summary
  • This page is about old computer systems.
  • Moving data from old systems can cause problems.
  • It shows how to check data before and after a move.

Why it matters

Data debt patterns, and how to find them

PatternExampleFind it with the toolkitFix

Data debt self-assessment

10 questions, about 5 minutes. Your answers stay in this browser.

Data quality gates for a migration

MoSCoW: data quality in a modernisation programme

Put the obligations into the new supplier's contract with the clause builder, and write the target rules as a data contract.

Reconcile two versions of a dataset

Compare before and after a migration, or a supplier's extract with your own copy. Checks row counts, records missing or new, duplicate keys, totals of number columns, and values that changed. Both files stay in your browser. The same check runs in Python: dqkit reconcile old.csv new.csv --key id.

Hold Ctrl (or Cmd) to choose more than one.

Sources