Pilots clear. Rollouts do not

In February 2017 the Infrastructure and Projects Authority assessed the government’s flagship identity verification platform and produced a sentence that ought to be pinned above every programme board in Britain. GOV.UK Verify, it said, had been an innovative technical success and was performing to specification, but it was not producing the promised benefits, which relied on large numbers of people signing up.

Both halves of that are accurate, and the fact that they can both be accurate at once is the whole problem. The thing worked. Nobody used it.

The National Audit Office investigated in March 2019, and the record it left is the clearest account available of how a British transformation programme stalls after the first deployment. It is worth reading as a general mechanism rather than as a story about digital identity, because almost none of the causes are about identity.

The gap between the plan and the estate

The 2016 business case set two targets. Twenty five million people would use Verify by 2020, and 46 government services would be accessible through it by March 2018.

By February 2019, 3.6 million people had signed up. If the trend of the time had continued, around 5.4 million would have signed up by 2020. Nineteen government services were using it, less than half the number expected a year earlier, and the NAO recorded that at least 11 of those 19 could also be accessed through other online systems.

That last figure is the one to sit with. For more than half the services that had adopted the platform, the platform was an alternative rather than the route. The programme had not replaced anything. It had added an option.

The mechanism, in four parts

The first department’s costs are the programme’s costs. The second department’s are its own.

The Government Digital Service intended Verify to be largely self-funding by the end of March 2018. It was not. Average prices paid to providers remained above £20 for new verifications, so GDS continued to subsidise departments for using it. Note the word average, which the NAO uses and which matters: it is a mean across a mixed book, not a floor under every transaction. And then the detail that says more about cross-organisation adoption than any survey of stakeholder sentiment: HMRC paid £6.7 million for its usage, and between 2016-17 and 2018-19 no other department paid at all, despite being issued invoices by the Cabinet Office. The NAO’s comment is a model of restraint. It is unclear why some departments have not paid these invoices.

This is not resistance and it is not politics. It is the absence of a mechanism that makes adoption rational for the adopter. A department asked to reconfigure its systems, retrain its staff, absorb the failure cases and then pay an invoice, in exchange for a benefit that will be booked centrally, has been asked to make a loss on behalf of somebody else’s business case. Most will decline politely and slowly, which looks identical to being busy.

The commercial model had assumed the opposite. Under the original design, GDS expected the price paid per sign-up to fall over time as user numbers increased, which is the standard shape of a platform business case and works only if the volume arrives. Volume did not arrive, so unit prices stayed high, so departments needed subsidy, so the platform could not become self-funding, so the case for it weakened, which further discouraged adoption. Every step of that is a consequence of the step before it and none of them is a mistake in isolation.

The supply side responded to the same signal. Royal Mail and CitizenSafe, part of GBG, had both been identity providers and decided not to continue on the commercial terms applying from October 2018. When the demand assumption in a platform business case fails, the suppliers who priced against that assumption leave first, and they leave quietly, in a paragraph of a report nobody outside the programme reads.

The benefits were non-cash releasing, so nobody’s budget depended on them.

GDS originally estimated Verify’s benefits at £873 million for the four years from 2016-17 to 2019-20. It revised that to £217 million, 75% lower. The NAO recorded that it had not been able to replicate or validate the revised estimate on the evidence made available, and that a significant proportion of the expected financial benefits came from avoided building costs, meaning money departments would otherwise have spent constructing their own identity systems.

Benefits classified as non-cash releasing produce departmental efficiencies without necessarily reducing a budget. Which means that if adoption is zero, no finance director is short of money, no one has to explain a variance, and the programme’s failure is visible only to the programme. Cash releasing benefits create an enemy who will chase you. Non-cash releasing benefits create nobody.

The measure that mattered was not a delivery milestone.

GDS reported a verification success rate of 48% at the beginning of February 2019, against a 2015 projection of 90%. The NAO adds two qualifications that make the real position worse. The measure counts people who succeed in a single attempt out of all those who try, and some failures are not counted at all, such as people who drop out before finishing their application. And it says nothing about whether a successfully verified person could then actually access the service they wanted.

For the largest customer the effect was severe. Universal Credit remained Verify’s biggest government user, and only 38% of Universal Credit claimants could successfully verify their identity online, of the 70% of claimants who attempted to sign up through Verify. Departments consequently had to undertake more manual processing than they anticipated, which increased their costs.

A completion rate for the single most common user journey is available from the first week of a pilot. It is almost never on a programme dashboard, because dashboards report milestones, and a milestone is a thing the programme controls.

And the specification stayed met while the point was missed.

This is why the assurance rating did not save anyone. The platform performed to specification. Adoption was not in the specification, because adoption is somebody else’s behaviour, and a delivery specification describes what the delivery organisation will produce. Verify was subject to over 20 internal and external reviews, a number the NAO says in part reflects GDS’s efforts to re-evaluate and reset the programme. In July 2018 the IPA recommended that it be closed as quickly as practicable, bearing in mind Universal Credit’s critical dependency on it. In October 2018 the Cabinet Office announced that government funding would stop in March 2020, capping spending in the interim at £21.5 million.

The total, from 2011-12 to September 2018, was at least £154 million on Verify and its predecessor programme, of which £58 million went to providers, and the NAO points out that this is likely an underestimate of costs across government, since it excludes what departments spent reconfiguring their own systems.

What this is not

It is not a story about optimism, although the NAO’s concluding remarks do describe Verify as exhibiting failings often seen in major programmes, including optimism bias and failure to set clear objectives.

It is not a story about the technology being wrong, and treating it as one is how the lesson gets lost. A programme in which the assurance function says the product works and does not produce benefits has discovered that it is in a different business from the one it was staffed for. GDS had built a product and then needed to sell it to buyers who could decline, with no price mechanism, no sales function and no authority to compel. That is market entry. It was resourced as delivery.

Every large organisation running a shared platform is in the same position and usually has not noticed, because internally the buyers are colleagues and it feels rude to describe them as a market.

The end state makes the point sharper, and it is worth stating at the strength the NAO stated it. At the time of the report GDS was still considering what the commercial model would look like after April 2020 and how private sector providers would take over control and management. One possibility recorded was that departments would procure identity verification directly from the market. Departments had not been paying their full usage costs and would have to under a market-based model. GDS would no longer set prices after April 2020, so it could not guarantee what the market would determine, and the NAO recorded the consequent risk that the price could be unaffordable for the departments still using it.

A shared service that could not persuade its internal customers to pay a subsidised price was, on that route, heading for a market that would charge them more. That is not a criticism of the decision, which had few alternatives by then. It is an illustration of how narrow the options become once adoption has been treated as a communications problem for three years.

Four things to establish before the second rollout

Whose budget pays, and has the money already moved. Not whether it has been agreed in principle. Whether the transfer has happened, because a benefit that has not been taken out of somebody’s budget is a benefit nobody is obliged to deliver.

What the second adopter’s alternative is, and what it costs them. Eleven of Verify’s nineteen services had another way in. If the alternative is cheaper for them than adopting, adoption will lose, regardless of the total cost to the organisation.

What the adoption assumption is, expressed as a rate for a named population, and who has ever hit that rate. Ninety per cent verification success was projected in 2015. Nobody had achieved it. An adoption assumption with no precedent behind it is a wish with a decimal point.

And what happens to the programme’s rating if adoption is zero. If the answer is nothing, the rating is measuring the wrong thing, and it will keep saying so confidently until the funding decision arrives.

The warning sign that is visible from outside

The four signals that a public programme is in trouble are all matters of record, and this one is the most reliable of the four: a delivery confidence assessment that describes the product favourably and the benefits unfavourably in the same paragraph. It is not a contradiction and it is not a drafting compromise. It is an assurance function correctly reporting that the thing built is not the thing that was supposed to produce the return.

When that sentence appears, the useful question is not how to improve the product. It is who has to change their working week for the benefit to arrive, and what has been offered to them, which is the argument in nobody embraces change. The running cost of keeping an adopted system correct in its second year is a separate and equally underestimated number, set out in what automation costs to run, and the rest of this coverage sits under digital transformation.