Data Asset Management (DAM) capability playbooks
Have your sayHelpSite map

Sector and domain supplements Level: Practitioner

Sector supplements

Extra governance, catalogue fields and quality rules for kinds of data the core playbooks treat only in general terms.

  • 15 supplements
  • 51 quality rules
  • Linked to the rulebook

In brief

  • Answer a few questions to see which supplements apply to your asset.
  • Each supplement adds to the three capabilities; it never replaces the core method.
  • Quality rules download in a format the Data Quality Commons kit can run.

Which supplements apply to your asset?

Tick everything that is true of the critical data asset

Supplements that apply

Tick a statement to see which supplements apply.
Which supplements apply
If this is true of the assetUse these supplements
It holds records about individual peopleData about people Data sharing and legal gateways
It holds health or care recordsHealth and care data Adult and children's social care data
It holds addresses, locations or map geometryGeospatial and address data
It is used to produce official statisticsOfficial statistics and administrative sources
It is used, or may be used, for AI, machine learning or automated decision supportAI-ready data
Personal data in it is shared with, or acquired from, another organisationData sharing and legal gateways
It is used to detect fraud or errorFraud and error
It supports law enforcement, justice, defence or securityJustice, policing and national security
It flows to or from councils or the devolved governmentsDevolved governments and local government Adult and children's social care data
It relies on code lists or identifiers maintained by other organisationsReference data and identifiers
It must be kept, archived or destroyed on a scheduleRecords, retention and disposal
Other systems or teams build on its structure, or it has a formal data modelData architecture and models
It is time-series or sensor data, such as environmental monitoringEnvironmental, sensor and time-series data
It holds records about pupils or learnersPupil and learner data

All supplements

Sector rule library

Every rule across the supplements. Download them all as CSV, or each supplement's rules as YAML.

Every sector quality rule
IDSupplementData elementRuleDimensionCheck
HLT-01Health and care dataNHS numberNHS number is recordedcompletenessnot_null
HLT-02Health and care dataNHS numberNHS number is 10 digits with a valid modulus 11 check digitvalidityidentifier
HLT-03Health and care dataDate of birthDate of birth is not in the futurevaliditynot_future
HLT-04Health and care dataOrganisation codeProvider organisation code is in ODS formatvalidityidentifier
HLT-05Health and care dataDiagnosisPrimary diagnosis is in ICD-10 code formatvalidityregex
HLT-06Health and care dataEthnicityEthnic category is not unknown or not statedcompletenessnot_unknown
HLT-07Health and care dataNHS number by groupNHS number completeness does not differ by more than 5 points between ethnic groupscompletenessgroup_disparity
SOC-01Adult and children's social care dataCouncil codeSupplying council is a valid GSS codevalidityidentifier
SOC-02Adult and children's social care dataNHS numberNHS number is valid where present, so care and health records can be linkedvalidityidentifier
SOC-03Adult and children's social care dataEvent dateEvent dates are not in the futurevaliditynot_future
SOC-04Adult and children's social care dataSubmissionEach council's latest submission is no more than one quarter and 30 days oldtimelinessfreshness
SOC-05Adult and children's social care dataPerson and eventEach event is recorded once for each personuniquenessunique
GEO-01Geospatial and address dataUPRNUPRN is recordedcompletenessnot_null
GEO-02Geospatial and address dataUPRNUPRN is 1 to 12 digitsvalidityidentifier
GEO-03Geospatial and address dataPostcodePostcode is in Royal Mail formatvalidityidentifier
GEO-04Geospatial and address dataEastingEasting is within the British National Grid rangevalidityrange
GEO-05Geospatial and address dataNorthingNorthing is within the British National Grid rangevalidityrange
GEO-06Geospatial and address dataArea codeArea code is a GSS codevalidityidentifier
STA-01Official statistics and administrative sourcesReference periodReference period is complete with no missing monthscompletenessno_gaps
STA-02Official statistics and administrative sourcesCountsMonthly totals are not implausible outliersaccuracyoutlier
STA-03Official statistics and administrative sourcesGeographyArea codes are valid GSS codesvalidityidentifier
STA-04Official statistics and administrative sourcesUnitsEvery measure column names its unitconsistencyheader_has_unit
SHR-01Data sharing and legal gatewaysAgreement referenceEvery shared extract carries the agreement referencecompletenessnot_null
SHR-02Data sharing and legal gatewaysExtract dateExtracts arrive on the agreed scheduletimelinessfreshness
SHR-03Data sharing and legal gatewaysColumnsOnly the columns in the agreement are presentvalidityrequired_columns
AI-01AI-ready dataLabelTraining labels are completecompletenessnot_null
AI-02AI-ready dataLabel by groupLabel completeness does not differ by more than 5 points between groupscompletenessgroup_disparity
AI-03AI-ready dataRecordEach record appears once, so training and test sets do not overlapuniquenessunique
AI-04AI-ready dataFeaturesNumeric features have no implausible outliersaccuracyoutlier
FRD-01Fraud and errorNational Insurance numberNational Insurance number is in a valid formatvalidityidentifier
FRD-02Fraud and errorPersonEach person appears once in the matching baseuniquenessunique
FRD-03Fraud and errorDate of birthDate of birth is recorded, so matches are not made on name alonecompletenessnot_null
JUS-01Justice, policing and national securityClassificationEvery record carries a classification marking from the allowed listvalidityin_set
JUS-02Justice, policing and national securityEvent dateEvent dates are not in the futurevaliditynot_future
PER-01Data about peopleNameFamily name is recordedcompletenessnot_null
PER-02Data about peopleDate of birthDate of birth is not in the futurevaliditynot_future
PER-03Data about peoplePersonEach person is recorded onceuniquenessunique
PER-04Data about peopleEthnicityEthnicity is not unknown, where it is lawfully collectedcompletenessnot_unknown
REF-01Reference data and identifiersArea codeArea code is a GSS codevalidityidentifier
REF-02Reference data and identifiersCompany numberCompany number is in Companies House formatvalidityidentifier
REF-03Reference data and identifiersLegal entity identifierLEI is valid, including its check digitsvalidityidentifier
REF-04Reference data and identifiersCharity numberCharity number is in a valid formatvalidityidentifier
REC-01Records, retention and disposalRecord dateRecords are not older than the retention period in your schedule (this example uses 7 years)timelinessfreshness
ARC-01Data architecture and modelsEntity keyEach entity's key is uniqueuniquenessunique
DEV-01Devolved governments and local governmentNationNation is recorded from the allowed listvalidityin_set
DEV-02Devolved governments and local governmentArea codeArea code is a GSS codevalidityidentifier
ENV-01Environmental, sensor and time-series dataTimestampNo gaps in the expected reading intervalcompletenessno_gaps
ENV-02Environmental, sensor and time-series dataUnitsEvery measure column names its unitconsistencyheader_has_unit
ENV-03Environmental, sensor and time-series dataReadingReadings have no implausible outliersaccuracyoutlier
EDU-01Pupil and learner dataSchoolSchool unique reference number is 6 digitsvalidityidentifier
EDU-02Pupil and learner dataPupilEach pupil appears once per censusuniquenessunique

Supplements in full

SUP-HLT · Must have

Health and care data

Governance Cataloguing Quality

For critical data assets that hold patient-level or care records. Adds the health roles, duties and coding standards the core playbooks do not cover.

For: NHS England and health bodies; Department of Health and Social Care ALBs. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Map the data owner and steward to the Caldicott Guardian, the SIRO and, for health IT, the clinical safety officer. Record who holds each role for every asset.
  • Record the legal basis for each use, the common law duty of confidentiality position and whether the national data opt-out applies.
  • Check the organisation's latest Data Security and Protection Toolkit assessment covers the systems that hold the asset.
  • Apply the Caldicott principles when deciding what each user of the asset needs: the minimum necessary, justified for each purpose.
  • Record the clinical terminology and exchange standards the asset follows, such as SNOMED CT, dm+d and HL7 FHIR UK Core, in conformsTo, so the MEM record points to the sector standard.
  • Where a health IT system holds the asset, record its clinical safety position: DCB0129 for the supplier and DCB0160 for the organisation that deploys it.
  • If confidential patient information is used without consent, record the legal basis that sets aside the duty of confidentiality, such as section 251 support, separately from the data protection lawful basis.
  • If software or AI that uses the asset has a medical purpose, check whether it is a medical device under the Medical Devices Regulations 2002, using the MHRA's guidance.
  • If NHS health and social care data in the asset is used for research or analysis beyond direct care, give access through a secure data environment, as DHSC's secure data environment policy guidelines require, and record which environment in the catalogue record.

Risks to check

  • Pseudonymised data is still personal data; access controls and agreements still apply.
  • Opt-outs and local suppression can bias coverage for some groups; check coverage against an independent count.
  • Clinical codes can be valid in format but wrong in meaning; format checks do not replace clinical coding audit.

Extra catalogue fields

Catalogue fields to add for health and care data
FieldSuggested termHow to record it
legalBasisdpv:hasLegalBasisThe lawful basis and, for health data, the special category condition
optOutAppliedlocal extensionWhether the national data opt-out has been applied to this distribution (yes, no or not applicable)
deidentificationlocal extensionIdentifiable, pseudonymised or anonymised, and the method used
conformsTodcterms:conformsToThe NHS data set or Data Dictionary standard the data follows

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for health and care data
IDData elementRuleDimensionCheckParameters
HLT-01NHS numberNHS number is recordedcompletenessnot_nullcolumn=nhs_number
HLT-02NHS numberNHS number is 10 digits with a valid modulus 11 check digitvalidityidentifiercolumn=nhs_number, scheme=nhs_number
HLT-03Date of birthDate of birth is not in the futurevaliditynot_futurecolumn=date_of_birth
HLT-04Organisation codeProvider organisation code is in ODS formatvalidityidentifiercolumn=provider_code, scheme=ods_code
HLT-05DiagnosisPrimary diagnosis is in ICD-10 code formatvalidityregexcolumn=primary_diagnosis, pattern=[A-Z][0-9]{2}(\.?[0-9X])?
HLT-06EthnicityEthnic category is not unknown or not statedcompletenessnot_unknowncolumn=ethnic_category
HLT-07NHS number by groupNHS number completeness does not differ by more than 5 points between ethnic groupscompletenessgroup_disparitygroup=ethnic_category, tolerance=5, inner=not_null(nhs_number)

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • NHS number: 10 digits; the last is a modulus 11 check digitK: General knowledge, not checked against a source in this release
  • ODS code: 3 to 6 letters and digits identifying an NHS or care organisationK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: an outpatient activity extract shared with a central department. Critical data elements are NHS number, provider code, attendance date and primary diagnosis. Baseline shows NHS number completeness of 97% overall but 91% for one ethnic group; HLT-07 records that as its own issue so it is not averaged away.

Checklist

Checklist done0 of 5 done
  1. Caldicott Guardian, SIRO and clinical safety officer roles are recorded for the asset
  2. Legal basis, confidentiality position and opt-out status are recorded on the catalogue record
  3. NHS number validity and completeness rules are in the DQAP
  4. Quality is broken down by ethnic group where lawful
  5. Access for research and analysis goes through a secure data environment, recorded on the catalogue record

Legislation, standards and frameworks that apply

Health and care
The Caldicott PrinciplesV: Verified against the primary source on 2026-11-05
Data Security and Protection ToolkitV: Verified against the primary source on 2026-11-05
HL7 FHIR UK CoreS: Found by web search; the site blocked automated checking, so open it before relying on it
SNOMED CTS: Found by web search; the site blocked automated checking, so open it before relying on it
DCB0129: Clinical risk management in the manufacture of health IT systemsS: Found by web search; the site blocked automated checking, so open it before relying on it
DCB0160: Clinical risk management in the deployment and use of health IT systemsS: Found by web search; the site blocked automated checking, so open it before relying on it
Confidential patient information and section 251S: Found by web search; the site blocked automated checking, so open it before relying on it
The Medical Devices Regulations 2002S: Found by web search; the site blocked automated checking, so open it before relying on it
Medical devices: software applications (apps)S: Found by web search; the site blocked automated checking, so open it before relying on it
Data saves lives: reshaping health and social care with dataV: Verified against the primary source on 2026-11-05
Secure data environment for NHS health and social care data - policy guidelinesV: Verified against the primary source on 2026-11-05
Data access policy updateV: Verified against the primary source on 2026-11-05
Legislation
Data Protection Act 2018V: Verified against the primary source on 2026-11-05

SUP-SOC · Should have

Adult and children's social care data

Cataloguing Quality

For assets built from council returns, such as adult social care client level data and children's social care census returns, where the source systems sit outside the policy's scope.

For: Local authorities (voluntary adoption); Department of Health and Social Care; Department for Education. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record which councils supply the data and that they are not bound by the DAM Policy; agree quality expectations through the collection's own guidance.
  • Name the owner for the national asset and a contact route for suppliers, not individuals.
  • Record changes to collection specifications as decisions, because they change what the asset means.

Risks to check

  • Councils use different case management systems, so the same field can be captured differently.
  • Late or missing submissions make national figures look better or worse than they are.
  • Linking to health data depends on NHS number quality in council systems.

Extra catalogue fields

Catalogue fields to add for adult and children's social care data
FieldSuggested termHow to record it
spatialdcterms:spatialThe councils and nations covered, using GSS codes
accrualPeriodicitydcterms:accrualPeriodicityThe submission cycle, for example quarterly
conformsTodcterms:conformsToThe collection specification and its version

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for adult and children's social care data
IDData elementRuleDimensionCheckParameters
SOC-01Council codeSupplying council is a valid GSS codevalidityidentifiercolumn=la_code, scheme=gss_code
SOC-02NHS numberNHS number is valid where present, so care and health records can be linkedvalidityidentifiercolumn=nhs_number, scheme=nhs_number
SOC-03Event dateEvent dates are not in the futurevaliditynot_futurecolumn=event_start_date
SOC-04SubmissionEach council's latest submission is no more than one quarter and 30 days oldtimelinessfreshnesscolumn=submission_date, expected=P3M, lag=P30D
SOC-05Person and eventEach event is recorded once for each personuniquenessuniquecolumns=['person_id', 'event_id']

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • GSS code: One letter (E, K, L, M, N, S or W) followed by 8 digitsK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: a quarterly client level data asset. SOC-04 shows two councils have not submitted for two quarters; the owner records the gap in the quality statement rather than imputing it.

Checklist

Checklist done0 of 3 done
  1. Supplying organisations and the collection specification are recorded
  2. Timeliness by supplier is measured and reported
  3. NHS number validity is checked where linkage is planned

Legislation, standards and frameworks that apply

Health and care
Adult social care client level data, England: background quality and methodologyV: Verified against the primary source on 2026-11-05
Adult social care outcomes framework: handbook of definitionsV: Verified against the primary source on 2026-11-05
Care data matters: a roadmap for better adult social care dataV: Verified against the primary source on 2026-11-05
Children and education
Children in need census: guide to submitting dataV: Verified against the primary source on 2026-11-05

SUP-GEO · Must have

Geospatial and address data

Cataloguing Quality

For assets that hold addresses, coordinates or map geometry. Adds geospatial metadata, open identifiers and positional quality.

For: Geospatial Commission; Ordnance Survey; HM Land Registry; environment bodies. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record the licence for any third-party geospatial data the asset depends on, and who may use derived data.
  • Use the open identifiers for property and street (UPRN and USRN) as the join between assets.
  • Apply the Geospatial Commission's guidance on ethical use of location data when location could identify people.

Risks to check

  • A valid UPRN format does not prove the property exists or is the right one; check match rates against the authoritative address base.
  • Coordinates in mixed reference systems silently misplace features.
  • Small-area location data can identify people even when names are removed.

Extra catalogue fields

Catalogue fields to add for geospatial and address data
FieldSuggested termHow to record it
spatialdcterms:spatialThe area covered, by GSS code or bounding box
crslocal extension (GEMINI spatial reference system)The coordinate reference system, for example British National Grid (EPSG:27700)
lineagedcterms:provenanceHow the geometry was captured and processed
conformsTodcterms:conformsToUK GEMINI where the asset is geospatial

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for geospatial and address data
IDData elementRuleDimensionCheckParameters
GEO-01UPRNUPRN is recordedcompletenessnot_nullcolumn=uprn
GEO-02UPRNUPRN is 1 to 12 digitsvalidityidentifiercolumn=uprn, scheme=uprn
GEO-03PostcodePostcode is in Royal Mail formatvalidityidentifiercolumn=postcode, scheme=uk_postcode
GEO-04EastingEasting is within the British National Grid rangevalidityrangecolumn=easting, min=0, max=700000
GEO-05NorthingNorthing is within the British National Grid rangevalidityrangecolumn=northing, min=0, max=1300000
GEO-06Area codeArea code is a GSS codevalidityidentifiercolumn=area_code, scheme=gss_code

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • UPRN: Unique property reference number, up to 12 digitsK: General knowledge, not checked against a source in this release
  • USRN: Unique street reference number, up to 8 digitsK: General knowledge, not checked against a source in this release
  • GSS code: One letter followed by 8 digitsK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: a register of public buildings. GEO-02 passes but 6% of UPRNs do not match the address base used by the receiving department; the owner logs a consistency issue and agrees a match-rate target.

Checklist

Checklist done0 of 4 done
  1. UPRN and USRN are used as join keys where they exist
  2. The coordinate reference system and lineage are on the catalogue record
  3. Positional checks and match rates are in the DQAP
  4. Licences for third-party geospatial inputs are recorded

Legislation, standards and frameworks that apply

Service delivery and technology
Identifying property and street information (UPRN and USRN)V: Verified against the primary source on 2026-11-05
Geospatial
UK GEMINI metadata standardV: Verified against the primary source on 2026-11-05
How FAIR are the UK's national geospatial data assets?V: Verified against the primary source on 2026-11-05
Best practice guidance and tools for managing geospatial dataV: Verified against the primary source on 2026-11-05

SUP-STA · Should have

Official statistics and administrative sources

Quality Cataloguing

For assets used to produce official statistics. Joins the DQAP to quality assurance of administrative data, so quality is reported once for both purposes.

For: Office for National Statistics; departmental statistics teams; Office for Statistics Regulation. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record the statistics produced from the asset and the producer responsible for them.
  • Agree with the statistics producer how supplier changes are communicated before they happen.
  • Use the DQAP as evidence for the assurance level set under quality assurance of administrative data.

Risks to check

  • Operational changes in the supplier's system can look like real-world change.
  • Two quality reports for the same data drift apart unless one feeds the other.

Extra catalogue fields

Catalogue fields to add for official statistics and administrative sources
FieldSuggested termHow to record it
isReferencedBydcterms:isReferencedByThe official statistics that use the asset
conformsTodcterms:conformsToSDMX structure or statistical classification used, where one applies

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for official statistics and administrative sources
IDData elementRuleDimensionCheckParameters
STA-01Reference periodReference period is complete with no missing monthscompletenessno_gapscolumn=period
STA-02CountsMonthly totals are not implausible outliersaccuracyoutliercolumn=total, method=iqr, k=3
STA-03GeographyArea codes are valid GSS codesvalidityidentifiercolumn=area_code, scheme=gss_code
STA-04UnitsEvery measure column names its unitconsistencyheader_has_unit

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • GSS code: One letter followed by 8 digitsK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: an administrative benefits extract used for quarterly statistics. STA-02 flags a sudden fall in one month that turns out to be a supplier system change; it is logged as a known issue in both the DQAP and the statistical release.

Checklist

Checklist done0 of 3 done
  1. Statistics that depend on the asset are recorded on the catalogue record
  2. A supplier change notice process is agreed
  3. DQAP results are used as evidence for administrative data quality assurance

Legislation, standards and frameworks that apply

Statistics
Code of Practice for StatisticsV: Verified against the primary source on 2026-11-05
Quality assurance of administrative dataS: Found by web search; the site blocked automated checking, so open it before relying on it
Standards
SDMXV: Verified against the primary source on 2026-11-05

SUP-SHR · Must have

Data sharing and legal gateways

Governance

For every critical data asset that includes personal data, because critical data assets are defined by being shared. Adds a legal gateway record to the governance and catalogue records.

For: All departments. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Keep a sharing decision record for each flow: purpose, lawful basis, any special category condition, legal power or gateway, DPIA reference, agreement reference and review date.
  • Check each flow against the Data (Use and Access) Act 2025 changes to UK GDPR at the next review.
  • Use a data sharing agreement for routine sharing, as the ICO code recommends, even where it is not mandatory.
  • Name the owner on both sides of every flow, by role.
  • If the flow uses a Part 5 power in the Digital Economy Act 2017, check the agreement is on the published register.
  • Record the agreement with the data sharing agreement template (T-GOV-6), and name the legal gateway in it and in the catalogue record.

Risks to check

  • Extra fields creep into extracts over time, beyond what the agreement allows.
  • Lawful basis recorded once at the start of a flow can go out of date when the law or the purpose changes.

Extra catalogue fields

Catalogue fields to add for data sharing and legal gateways
FieldSuggested termHow to record it
accessRightsdcterms:accessRightsRestricted, with the agreement reference
legalGatewaylocal extensionThe statutory power or legal basis for the flow
rightsdcterms:rightsConditions on onward sharing and use

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for data sharing and legal gateways
IDData elementRuleDimensionCheckParameters
SHR-01Agreement referenceEvery shared extract carries the agreement referencecompletenessnot_nullcolumn=agreement_ref
SHR-02Extract dateExtracts arrive on the agreed scheduletimelinessfreshnesscolumn=extract_date, expected=P1M, lag=P7D
SHR-03ColumnsOnly the columns in the agreement are presentvalidityrequired_columnscolumns=['person_id', 'agreement_ref', 'extract_date']

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: a monthly flow of eligibility data between two departments. The sharing decision record names the gateway, the DPIA and the agreement; SHR-03 fails when an extra free-text column appears, which is fixed at source before the next transfer.

Checklist

Checklist done0 of 3 done
  1. A sharing decision record exists for each flow, with a review date
  2. Agreement references appear on the catalogue record and in each extract
  3. Flows were reviewed against the Data (Use and Access) Act 2025 changes

Legislation, standards and frameworks that apply

Legislation
Data (Use and Access) Act 2025V: Verified against the primary source on 2026-11-05
Digital Economy Act 2017V: Verified against the primary source on 2026-11-05
Register of information sharing agreements under the Digital Economy Act 2017S: Found by web search; the site blocked automated checking, so open it before relying on it
ICO data sharing code of practiceS: Found by web search; the site blocked automated checking, so open it before relying on it
Data Protection Act 2018V: Verified against the primary source on 2026-11-05
Data protection
Data sharing: a code of practice - data sharing agreementsS: Found by web search; the site blocked automated checking, so open it before relying on it
Health and care
Confidential patient information and section 251S: Found by web search; the site blocked automated checking, so open it before relying on it

SUP-AI · Must have

AI-ready data

Quality Governance Cataloguing

For assets used, or likely to be used, for AI or automated decision support. Turns the GOV.UK AI-ready data guidance into checks at the annual review.

For: All departments using data for AI or automated decision support. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record the AI task the data supports, whether outputs affect individuals, and which uses are out of scope, such as automated decisions.
  • Name the owner who approves AI uses, and decide what happens if a use changes.
  • Publish an algorithmic transparency record where a tool supports decisions about people.
  • Use the Data and AI Ethics Framework for new or higher-risk uses.

Risks to check

  • Open or available data is not the same as AI-ready data.
  • Bias in labels or coverage becomes bias in outputs.
  • Without exact versioning, a model cannot be retrained or audited.

Extra catalogue fields

Catalogue fields to add for ai-ready data
FieldSuggested termHow to record it
aiPermittedUseslocal extensionTraining, fine-tuning, inference or evaluation: which are allowed
sensitiveFieldslocal extensionFields that are personal, confidential or special category
provenancedcterms:provenanceWhere the data comes from and how it is produced
versiondcat:versionVersion, so a training set can be reconstructed exactly

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for ai-ready data
IDData elementRuleDimensionCheckParameters
AI-01LabelTraining labels are completecompletenessnot_nullcolumn=label
AI-02Label by groupLabel completeness does not differ by more than 5 points between groupscompletenessgroup_disparitygroup=group, tolerance=5, inner=not_null(label)
AI-03RecordEach record appears once, so training and test sets do not overlapuniquenessuniquecolumns=['record_id']
AI-04FeaturesNumeric features have no implausible outliersaccuracyoutliercolumn=feature_value, method=zscore, threshold=4

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Self-assessment areas from the GOV.UK AI-ready data guidanceV: Verified against the primary source on 2026-11-05

Purpose and legitimacy (4)
  • The task the data supports is known
  • It is known whether outputs affect individuals
  • Out-of-scope uses are stated
  • A named owner approves AI uses
Legal, rights and governance (4)
  • Permitted uses (training, fine-tuning, inference, evaluation) are known
  • Sensitive fields are flagged
  • Permitted working environments are known
  • There is a plan if the use case changes
Metadata fitness (6)
  • Tables and fields are understood
  • Event time, update frequency and history are clear
  • Schema changes and versions are tracked
  • Provenance is known
  • Known issues and biases are documented
  • Stable identifiers exist or their absence is stated
Unstructured data pipelines (4)
  • Data is machine readable or has an extraction pipeline
  • Rights, redaction and sensitivity travel with extracted content
  • Outputs can be traced back to source files
  • Humans review training data and outputs
Operating model and sustainability (5)
  • Someone owns quality, drift and bias monitoring
  • Breaks, drift and anomalies are detected
  • Fixes are re-approved
  • Training sets can be reconstructed exactly
  • There is a process if AI causes service, cost or trust issues

Summarised from Making government datasets ready for AIV: Verified against the primary source on 2026-11-05.

Worked example. Illustrative, synthetic: a casework text archive proposed for a summarisation tool. The review finds redaction does not travel with extracted text, so the asset is marked not yet AI-ready and a pipeline fix is added to the DQAP.

Checklist

Checklist done0 of 4 done
  1. AI permitted uses and sensitive fields are on the catalogue record
  2. The five areas of the GOV.UK self-assessment are reviewed at the annual review
  3. Group disparity rules are in the DQAP
  4. A transparency record is published where a tool supports decisions about people

Legislation, standards and frameworks that apply

Data and AI
Making government datasets ready for AIV: Verified against the primary source on 2026-11-05
Algorithmic Transparency Recording Standard hubV: Verified against the primary source on 2026-11-05
AI Playbook for the UK GovernmentV: Verified against the primary source on 2026-11-05
Data and AI Ethics FrameworkV: Verified against the primary source on 2026-11-05

SUP-FRD · Should have

Fraud and error

Governance Quality

For assets used to detect fraud or error, where identity matching quality decides who is flagged.

For: HMRC; DWP; Public Sector Fraud Authority. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Require a person to review every automated flag before action is taken against someone.
  • Record the legal power used for each fraud data flow.
  • Measure and report false positives, not only detections.

Risks to check

  • Poor identity data produces false positives that fall on real people.
  • Names transliterated differently can be missed or mismatched, affecting some communities more than others.

Extra catalogue fields

Catalogue fields to add for fraud and error
FieldSuggested termHow to record it
purposelocal extension (one of the seven purposes)Fraud and error
matchingMethodlocal extensionHow records are matched, and the known match error rate

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for fraud and error
IDData elementRuleDimensionCheckParameters
FRD-01National Insurance numberNational Insurance number is in a valid formatvalidityidentifiercolumn=nino, scheme=nino
FRD-02PersonEach person appears once in the matching baseuniquenessuniquecolumns=['person_id']
FRD-03Date of birthDate of birth is recorded, so matches are not made on name alonecompletenessnot_nullcolumn=date_of_birth

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • National Insurance number: Two letters, six digits and a letter A to D, with some prefixes not usedK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: a matching exercise between benefit and payroll data. FRD-03 shows date of birth missing for 4% of records; those records are excluded from automated matching and routed to manual review.

Checklist

Checklist done0 of 3 done
  1. Human review of flags is required and recorded
  2. False positive rates are measured
  3. Identity field validity rules are in the DQAP

Legislation, standards and frameworks that apply

Fraud and error
Public Sector Fraud AuthorityV: Verified against the primary source on 2026-11-05
Legislation
Digital Economy Act 2017V: Verified against the primary source on 2026-11-05

SUP-JUS · Could have

Justice, policing and national security

Governance Cataloguing

For assets in law enforcement, justice, defence or security settings, where only OFFICIAL assets are in scope and metadata itself may be sensitive.

For: Ministry of Justice; Home Office; police; defence. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Confirm the classification of the asset and of its metadata; only OFFICIAL assets are in scope of the policy.
  • Record whether processing falls under Part 3 of the Data Protection Act 2018 (law enforcement processing).
  • Restrict catalogue record visibility where the description itself is sensitive, and record the handling marking.

Risks to check

  • Describing an asset in detail can reveal sensitive capabilities.
  • Assets above OFFICIAL must not be described in this portal or reported through it.

Extra catalogue fields

Catalogue fields to add for justice, policing and national security
FieldSuggested termHow to record it
securityClassificationlocal extension (MEM securityClassification)OFFICIAL, with any handling marking such as OFFICIAL-SENSITIVE
accessRightsdcterms:accessRightsInternal, with the vetting level needed to access the data

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for justice, policing and national security
IDData elementRuleDimensionCheckParameters
JUS-01ClassificationEvery record carries a classification marking from the allowed listvalidityin_setcolumn=classification, values=['OFFICIAL', 'OFFICIAL-SENSITIVE']
JUS-02Event dateEvent dates are not in the futurevaliditynot_futurecolumn=event_date

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: a court listings data asset. The catalogue record is visible across government, but the distribution is restricted, and JUS-01 confirms no records carry a higher marking.

Checklist

Checklist done0 of 3 done
  1. Classification of the asset and its metadata is confirmed
  2. Part 3 processing is identified where it applies
  3. Catalogue record visibility matches the sensitivity of the description

Legislation, standards and frameworks that apply

Legislation
Data Protection Act 2018V: Verified against the primary source on 2026-11-05
Security
Government Security ClassificationsV: Verified against the primary source on 2026-11-05

SUP-PER · Should have

Data about people

Cataloguing Quality

For any asset that holds records about individual people. Makes sure forms and rules let people describe themselves accurately.

For: All departments holding data about people. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Collect only the personal fields the purpose needs, and record why each is needed.
  • Follow current statistical guidance for questions about sex and gender, ethnicity and disability, and offer 'prefer not to say'.
  • Watch for the cross-government person data and vulnerability data standards and adopt them when published.

Risks to check

  • Forms that cannot record names, addresses or identities accurately create bias at source.
  • Completeness that looks good overall can hide gaps for some groups.

Extra catalogue fields

Catalogue fields to add for data about people
FieldSuggested termHow to record it
personalDatalocal extensionWhether the asset holds personal or special category data
populationlocal extensionWho the records describe, and known gaps in coverage

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for data about people
IDData elementRuleDimensionCheckParameters
PER-01NameFamily name is recordedcompletenessnot_nullcolumn=family_name
PER-02Date of birthDate of birth is not in the futurevaliditynot_futurecolumn=date_of_birth
PER-03PersonEach person is recorded onceuniquenessuniquecolumns=['person_id']
PER-04EthnicityEthnicity is not unknown, where it is lawfully collectedcompletenessnot_unknowncolumn=ethnicity

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: a customer contact data set. PER-01 fails for records from a form that forced a single name field; the fix is to change the form, not to clean the data.

Checklist

Checklist done0 of 3 done
  1. Each personal field has a recorded purpose
  2. Forms allow 'prefer not to say' and names in full
  3. Completeness is broken down by group where lawful

Legislation, standards and frameworks that apply

Accessibility and equality
Public sector equality dutyV: Verified against the primary source on 2026-11-05
Legislation
Equality Act 2010V: Verified against the primary source on 2026-11-05
Data Protection Act 2018V: Verified against the primary source on 2026-11-05

SUP-REF · Should have

Reference data and identifiers

Governance Cataloguing Quality

For assets that depend on code lists and identifiers maintained by other organisations, such as area codes, organisation codes and company numbers.

For: All departments. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Name the authoritative source and owner of each code list the asset uses.
  • Record the version of each code list and how changes are picked up.
  • Prefer recommended open standards for identifiers.

Risks to check

  • Code lists change; stale codes pass format checks but point to retired areas or organisations.

Extra catalogue fields

Catalogue fields to add for reference data and identifiers
FieldSuggested termHow to record it
conformsTodcterms:conformsToEach code list and its version

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for reference data and identifiers
IDData elementRuleDimensionCheckParameters
REF-01Area codeArea code is a GSS codevalidityidentifiercolumn=area_code, scheme=gss_code
REF-02Company numberCompany number is in Companies House formatvalidityidentifiercolumn=company_number, scheme=company_number
REF-03Legal entity identifierLEI is valid, including its check digitsvalidityidentifiercolumn=lei, scheme=lei
REF-04Charity numberCharity number is in a valid formatvalidityidentifiercolumn=charity_number, scheme=charity_number

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • Company number: 8 digits, or 2 letters and 6 digitsK: General knowledge, not checked against a source in this release
  • LEI: 20 characters with 2 check digits (ISO 17442)K: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: a grants register. REF-01 fails after a boundary change because the code list was not updated; the fix is to subscribe to the code list's change notices.

Checklist

Checklist done0 of 2 done
  1. Every code list has a named source and version
  2. Changes to code lists are picked up on a schedule

Legislation, standards and frameworks that apply

Service delivery and technology
Open standards for governmentV: Verified against the primary source on 2026-11-05
Identifying property and street information (UPRN and USRN)V: Verified against the primary source on 2026-11-05

SUP-REC · Should have

Records, retention and disposal

Governance

For every critical data asset that must be kept, archived or destroyed on a schedule. Joins data asset management to records management.

For: All departments; knowledge and information management teams. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record the retention schedule and disposal action for each asset in the ownership register.
  • Agree with the records manager when an asset is retired, archived or transferred.
  • Keep catalogue records for retired assets, marked as retired, so dependants can see what changed.
  • When an asset needs to be kept longer than its schedule allows, record the reason and who approved it in the exception register (T-GOV-4), with a review date.

Risks to check

  • Data kept past its retention period is a data protection risk; data destroyed early is a records risk.
  • There is no single government-wide retention period. The National Archives describes short, medium and long periods and each organisation sets its own schedule, so check yours before using an example such as seven years.

Extra catalogue fields

Catalogue fields to add for records, retention and disposal
FieldSuggested termHow to record it
retentionlocal extensionRetention period and disposal action
isReplacedBydcterms:isReplacedByWhat replaces a retired asset

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for records, retention and disposal
IDData elementRuleDimensionCheckParameters
REC-01Record dateRecords are not older than the retention period in your schedule (this example uses 7 years)timelinessfreshnesscolumn=created_date, expected=P7Y, lag=P30D

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: a case management extract kept for seven years. REC-01 flags records past retention, which the owner agrees to dispose of through the records management process.

Checklist

Checklist done0 of 3 done
  1. Retention and disposal are on the ownership register
  2. Retired assets keep a marked catalogue record
  3. The retention fields in the ownership register (T-GOV-2) and catalogue record (T-CAT-2) are filled in from your organisation's schedule

Legislation, standards and frameworks that apply

Records
Information management guidanceV: Verified against the primary source on 2026-11-05
Records and information management
Advice on retentionS: Found by web search; the site blocked automated checking, so open it before relying on it
Manage information: guidance for governmentV: Verified against the primary source on 2026-11-05
Public Records Act 1958V: Verified against the primary source on 2026-11-05

SUP-ARC · Should have

Data architecture and models

Cataloguing Governance

For critical data assets whose structure others build on. Describes the entities, how they relate and which system is the authoritative source, and links them to the catalogue.

For: All departments; design authorities and data architects. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Name the authoritative source for each entity, and the role that approves changes to its structure (the design authority in a change programme).
  • Keep the entity and relationship list (T-CAT-4) with the asset, and review it when the data model changes.

Risks to check

  • Without an agreed authoritative source, two systems can each claim to be the master copy, and returns count assets twice.
  • Diagrams drawn by hand drift from the real model; keep the list as the source and draw the diagram from it.

Extra catalogue fields

Catalogue fields to add for data architecture and models
FieldSuggested termHow to record it
conformsTodcterms:conformsToLink to the published schema or data model
dataModellocal extensionLink to the entity and relationship list or diagram

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for data architecture and models
IDData elementRuleDimensionCheckParameters
ARC-01Entity keyEach entity's key is uniqueuniquenessuniquecolumn=entity_id

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: a case management asset with people, cases, appointments and outcomes. The worked entity and relationship list and the diagram drawn from it are with the entity and relationship list template (T-CAT-4); ARC-01 checks each entity key is unique.

Checklist

Checklist done0 of 3 done
  1. Every entity has a key, a description and an authoritative source
  2. Relationships state their cardinality
  3. The catalogue record links to the list or diagram

Legislation, standards and frameworks that apply

Standards
Using metadata to describe data assets in a data catalogueS: Found by web search; the site blocked automated checking, so open it before relying on it

SUP-DEV · Should have

Devolved governments and local government

Governance Cataloguing Quality

For assets that flow to or from the devolved governments or councils, which are not bound by the policy but often depend on it.

For: Scottish Government; Welsh Government; Northern Ireland Executive; local authorities. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record which nations the asset covers and which bodies supply or receive it.
  • Where a devolved government or council supplies data, agree quality expectations in the sharing agreement, because the policy does not bind them.
  • Offer councils the playbooks as a voluntary route to adoption.

Risks to check

  • Presenting England-only rules as UK-wide misleads users in other nations.
  • Differences in definitions between nations break comparability.

Extra catalogue fields

Catalogue fields to add for devolved governments and local government
FieldSuggested termHow to record it
spatialdcterms:spatialNations covered, using GSS codes (England E, Scotland S, Wales W, Northern Ireland N)
languagedcterms:languageLanguages of the data and documentation, including Welsh where it applies

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for devolved governments and local government
IDData elementRuleDimensionCheckParameters
DEV-01NationNation is recorded from the allowed listvalidityin_setcolumn=nation, values=['England', 'Scotland', 'Wales', 'Northern Ireland']
DEV-02Area codeArea code is a GSS codevalidityidentifiercolumn=area_code, scheme=gss_code

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Four nations view

Who the policy applies to, by nation and tier
WhoPosition
UK government departments and ALBsMandatory under the DAM PolicyV: Verified against the primary source on 2026-11-05
ScotlandNot bound by the policy; own approach to data sharing governanceV: Verified against the primary source on 2026-11-05
WalesNot bound by the policy; own approach to data sharing governanceV: Verified against the primary source on 2026-11-05
Northern IrelandNot bound by the policy; own approach to data sharing governanceV: Verified against the primary source on 2026-11-05
CouncilsNot required; may adopt voluntarilyV: Verified against the primary source on 2026-11-05

Source: Identifying and recording critical data assetsV: Verified against the primary source on 2026-11-05

Worked example. Illustrative, synthetic: a UK-wide dataset assembled from four national returns. DEV-01 shows records from one nation coded inconsistently; the fix is agreed with that nation's data team.

Checklist

Checklist done0 of 2 done
  1. Nations covered are on the catalogue record
  2. Quality expectations for devolved or council suppliers are in the agreement

Legislation, standards and frameworks that apply

Local government
Local Digital DeclarationV: Verified against the primary source on 2026-11-05

SUP-ENV · Could have

Environmental, sensor and time-series data

Cataloguing Quality

For monitoring and sensor data where readings arrive continuously and gaps, units and drift matter more than missing fields.

For: Defra group; environment agencies. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Name who owns calibration of instruments as well as who owns the data.
  • Record how gaps and suspect readings are flagged and whether they are corrected.

Risks to check

  • Filling gaps silently makes data look complete when it is not.
  • Sensor drift passes format checks; it needs calibration records.

Extra catalogue fields

Catalogue fields to add for environmental, sensor and time-series data
FieldSuggested termHow to record it
temporalResolutiondcat:temporalResolutionThe interval between readings
spatialdcterms:spatialMonitoring sites and their locations

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for environmental, sensor and time-series data
IDData elementRuleDimensionCheckParameters
ENV-01TimestampNo gaps in the expected reading intervalcompletenessno_gapscolumn=reading_time
ENV-02UnitsEvery measure column names its unitconsistencyheader_has_unit
ENV-03ReadingReadings have no implausible outliersaccuracyoutliercolumn=value, method=iqr, k=3

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Worked example. Illustrative, synthetic: river level readings every 15 minutes. ENV-01 finds a six-hour gap from a failed sensor; the gap is flagged, not filled, in the published data.

Checklist

Checklist done0 of 2 done
  1. Reading interval and units are on the catalogue record
  2. Gaps and outliers are flagged, not silently corrected

Legislation, standards and frameworks that apply

Geospatial
Best practice guidance and tools for managing geospatial dataV: Verified against the primary source on 2026-11-05
Standards
Data Catalog Vocabulary (DCAT) version 3V: Verified against the primary source on 2026-11-05

SUP-EDU · Could have

Pupil and learner data

Quality

For assets about pupils and learners, which are long-lived and linked across schools, colleges and nations.

For: Department for Education and its ALBs. A synthesis of the sources belowO: This playbook's own synthesis of published guidance.

Governance overlay

  • Record the collection the data comes from and its specification version.
  • Treat records about children with the extra care data protection law expects.

Risks to check

  • Dual registration and moves between schools create apparent duplicates.

Extra catalogue fields

Catalogue fields to add for pupil and learner data
FieldSuggested termHow to record it
conformsTodcterms:conformsToThe census or collection specification and version

"Local extension" means no agreed cross-government term exists yet; record it in your catalogue's own field.K: General knowledge, not checked against a source in this release

Quality rule library

Starter quality rules for pupil and learner data
IDData elementRuleDimensionCheckParameters
EDU-01SchoolSchool unique reference number is 6 digitsvalidityidentifiercolumn=school_urn, scheme=school_urn
EDU-02PupilEach pupil appears once per censusuniquenessuniquecolumns=['pupil_id', 'census_date']

Download these rules for the Data Quality Commons kit (YAML). Column names are examples; map them to your data.

Identifiers

  • School URN: 6 digitsK: General knowledge, not checked against a source in this release

Worked example. Illustrative, synthetic: a termly pupil census extract. EDU-02 finds duplicate pupils for dual-registered learners; the rule is refined to count the main registration only.

Checklist

Checklist done0 of 2 done
  1. The collection specification and version are recorded
  2. Duplicate and dual registration rules are agreed

Legislation, standards and frameworks that apply

Legislation
Data Protection Act 2018V: Verified against the primary source on 2026-11-05

Page owner: the portal's service owner (a role) · Published 5 November 2026 · Last updated 2026-11-06 · Next review 2027-02-05

Was this page useful? Tell us Tell us privately, without GitHub