The public sector's data problem is not about data

Twenty-eight data-sharing agreements were set up between government bodies to tackle fraud under the powers in the Digital Economy Act 2017. Four of them became business as usual. That count, current to July 2025, appears in the Public Accounts Committee’s report on government use of data analytics on error and fraud, published on 27 March 2026, and it is the most informative number in British public sector data policy.

It is informative because of what it is not. It is not a statement about data quality, or about analytical capability, or about whether departments understand machine learning. Twenty-eight teams got far enough to negotiate a legal basis for sharing records with another part of the state. Twenty-four of those arrangements did not survive into routine operation. The Public Sector Fraud Authority’s own indicative timeline suggested a data-sharing agreement should take around twenty weeks to conclude. The auditors found the process could run for months or years, and that nobody was formally monitoring whether the twenty weeks was ever met.

The prize is stated, in public, by the people who would collect it

Ask whether the public sector is doing enough with its data and the honest first move is to establish what enough would be worth. Unusually, that has a published answer.

The National Audit Office estimates fraud and error cost the taxpayer between £55 billion and £81 billion in 2023-24. Roughly £40 billion of that sits in tax and around £10 billion in welfare, with the remainder, between £5 billion and £31 billion, spread across everything else government does. Those are gross figures before recovery activity, and the width of the bands is itself a finding: for large parts of the state the loss is an estimate rather than a count.

Against that, the Government Digital Service has estimated that effective use of data analytics could save up to £6 billion a year. DSIT told the committee that this estimate helped it make its case in the most recent spending review. So the department responsible put a number on the opportunity, used the number to win money, and the auditors then concluded that savings achieved to date have been modest against the potential.

That sequence is worth pausing on, because it disposes of the usual explanations. This is not a case of a public body failing to recognise the value of its records. The value was quantified, argued and funded. The failure is downstream of belief.

What actually binds

Three constraints show up repeatedly in the evidence, and none of them is analytical.

The first is legal architecture written for a different technical era. The National Fraud Initiative, created in 1996, matches datasets across the public sector to find duplicate and improper payments. Local authorities are required to take part. Central government is not, because the legislation that created the exercise was aimed at local government and has not been amended since. Thirty-six central bodies including arm’s length bodies joined the most recent exercise in 2024-25, up from twenty-two in the previous round, which is progress by voluntarism. The Public Sector Fraud Authority’s preliminary estimate, supplied to the committee in writing on 12 March 2026, was that mandating participation for central departments and amending the law to permit more matching could realistically save between £120 million and £150 million per two-year cycle, with a net uplift of about £60 million even on its most conservative modelling.

The second is legislation that forbids the specific analysis a modern counter-fraud team would want to run. Under the Local Audit and Accountability Act 2014 there is no permission for profiling of individuals’ behaviours. In practice that means data gathered under the Act can be used to look for fraud but not to support a live investigation. Nor may it mark out a person with a fraud already proved against them as a risk indicator in future work. The authority’s evidence noted that in other industries treating a proven fraud as a predictor of further fraud is standard. Data collected for the National Fraud Initiative may also be retained only for the two years of each biennial exercise, which rules out the longitudinal view that would make the matching sharper.

The third is time. Agreements that take years to negotiate are not agreements in any operational sense. A programme with a three-year business case cannot depend on a permission that may arrive in year four.

Notice what is absent from that list. No shortage of records. No shortage of techniques. The barrier is the arrangement of permission, and permission is a legislative and administrative product rather than a technical one.

The capability argument is real, and it is smaller than it looks

There is a genuine skills gap and it is measured. The Cabinet Office told the committee in March 2025 that it was aiming for 10% of civil servants to have digital expertise, an ambition tied to saving up to £500 million a year by reducing the need for digital consultants. In April 2025 only 5.5% of the civil service met that threshold. Some departments have already passed 10%; others are far behind, which is the pattern in almost every measure of British public sector digital capacity.

The State of digital government review, published on 21 January 2025, sets out why that gap is expensive rather than merely embarrassing. It recorded an average contractor cost of £182,000 a year against £61,000 for the average civil servant, roughly three times as much, and a workforce split in which contractors are 18% of digital headcount but a far larger share of digital cost. An organisation staffed that way does not lack people who can build a data pipeline. It lacks people who will still be there when the pipeline needs changing, and who can be given custody of a data-sharing agreement for its whole life.

The same review found only 27% of its survey respondents believed their data infrastructure enabled a comprehensive view of operations, and 70% said their data landscape was not well co-ordinated. Those are self-assessments and should be read as such. They describe how the people running the estate feel about it, which is useful, and they do not establish what is actually true of the records.

Leadership was reorganised rather than strengthened

January 2025 brought the digital centre of government into DSIT. The blueprint issued at the time envisaged a chief digital officer for government at second permanent secretary rank, with the standing to act across departments.

The post was not created. Two director general roles took its place, splitting responsibility between the products the Government Digital Service runs itself and the wider transformation agenda, both answering to the departmental permanent secretary and both filled on an interim basis at the start of 2026. The committee’s verdict was that this is a shortcoming, and its recommendation was to appoint at permanent secretary level after all, with levers attached.

Whether seniority alone would fix anything is arguable. What is not arguable is the mechanism the committee is pointing at. Data sharing between two departments is a negotiation between two accounting officers with different statutory duties, different risk appetites and different legal advice. Nothing below permanent secretary level settles that quickly, which is precisely why twenty-four of twenty-eight pilots did not become routine.

Transparency is the part that can be checked from outside

Government bodies are required to record their use of algorithms, artificial intelligence and machine learning in decision making through the Algorithmic Transparency Recording Standard. As at February 2026 the hub held 110 records for central government. Eleven of those mentioned fraud. None of the examples the auditors had identified as good practice appeared on the register at all.

DSIT acknowledged the record is incomplete while saying it believed most cases had been captured. Departments told it they find the process difficult because they do not want to publish information that would help fraudsters, which is a real tension rather than an excuse. Some counter-fraud logic genuinely cannot be described in public without weakening it.

But the gap between eleven fraud records and a state that has been running analytics on payments for years is not explained by operational sensitivity. It is explained by a register that nobody is required to complete on pain of anything. Registers with no consequence for omission converge on the same completeness everywhere, which is partial.

This matters more than it sounds, because the register is the only route by which a citizen, a journalist or a select committee can find out what is being run against their records without submitting a freedom of information request and waiting. It is the entire public accountability surface for a large and growing category of government decision.

The library that is still being designed

The government’s answer to the co-ordination problem is the National Data Library. Its progress update of 26 January 2026 records over £100 million allocated to it within a £1.9 billion investment in DSIT across the spending review period, a completed discovery phase, an expert advisory group, five kickstarter projects covering energy bill support, long-term health support, adult social care, legal guidance from the National Archives and weather and climate data, and an exercise evaluating how data.gov.uk performs, assessing the range of datasets available on it and the functionality it already has. Further details were promised for spring 2026. No decision about what data.gov.uk becomes is announced, and it is worth resisting the assumption that it becomes the library’s front door, because the update does not say so.

Read that list carefully. It is a discovery phase, a governance structure, a set of pilots and a review of a website that already exists. Those are reasonable things to have after a year. They are not yet a mechanism that shortens a data-sharing agreement from years to weeks, which is the specific failure the audit trail identifies.

There is a precedent worth holding in mind. When the National Audit Office examined challenges in using data across government on 21 June 2019, its finding was that government had lacked clear and sustained strategic leadership on data, that quality was often inadequate, and that there was a culture of tolerating and working around poor records. The Comptroller and Auditor General warned that the national data strategy then being prepared would need to force a change in leadership rather than restate the ambition. Seven years later a different central body is publishing a different plan against a recognisably similar diagnosis.

The difference this time is that one adjacent thing has been built and does work. The Digital Economy Act projects that did conclude have delivered benefits in excess of £200 million in reduced fraud. Four agreements out of twenty-eight produced that. The arithmetic of fixing the pipeline rather than adding to the top of it is not subtle.

So, is it doing enough

No, and the reason is not the one usually given.

The public sector is not short of data, short of enthusiasm for data, or short of strategies about data. It is short of a route by which two public bodies can agree to match two datasets in a timeframe shorter than the career of the person who proposed it. Everything else in this field, the platforms, the libraries, the registers and the capability targets, sits downstream of that.

Which suggests a test for any future announcement in this area, and it is not a test of ambition. Ask what the announcement does to the elapsed time between two organisations deciding to share records and actually doing so. If the answer is nothing, the announcement is about something else.

Related coverage sits in public sector technology, and the underlying records problem, which is not confined to government, is examined in you cannot fine-tune your way out of bad records.