The open data failures of the coalition and how it can be put right

Written by Simon Briscoe on . Posted in Features

Which parts of the public sector are battling to open up data, and which are resisting it in the UK? There are so many moving parts it’s hard to say. To try and answer this, I produced a couple of open data maps to show how busy the coalition government had been with public sector open data.

The previous government set up many initiatives to open up data and it felt great at times – there was certainly fantastic ambition. The trouble is that there was no real plan and a shortage of skills, which led to a lack of delivery. Unforgivably, there was very little openness about what was going on and it looks like not much has been delivered. We now have a new government, so it’s a good time to think about what is needed if the supposed benefits of public sector open data are to be realised.

{mbox:significance/2015/graphs/open-data-map-briscoe1.png|width=300|height=222|caption=Click to enlarge|title=Open data holders and users}

This first diagram (see right) sets the scene. The many silos in the public sector have loads of information that is by and large not made public. There are lots of people – mainly outside the public sector but also in it – who would happily use it for one purpose or another. Open data is about enabling that sharing.

The second ‘map’ is more complex. It shows the main players in the UK public sector data – and the links between them. Solid lines imply a closer or more controlling relationship than a dotted line.

The main heavy outer line represents the public sector. The dotted line inside that is central government. The departments of state with a major role in open data are identified as are the main agencies and the principle initiatives set up by them.

Of course, virtually all the departments and agents of government release some information. What they publish or make available varies greatly between them. The triangles show the main conduits of information as part of the open data movement. Some data that used to be public sector data has been privatised and, as things stand, are now outside the public sector.

{mbox:significance/2015/graphs/open-data-map-briscoe2.png|width=300|height=223|caption=Click to enlarge|title=Key public sector players in open data}

As Guiseppe Sollazzo, an ex-colleague on ODUG, has written. The British government has '… gone from four to zero advisory panels on open data issues in the space of two years.' He was referring to the Data Strategy Board (wound up in 2013), APPSI and ODUG (wound up earlier this year) and the Public Sector Transparency Board (thought to have been appointed until April 2015).

Are these bodies to be resurrected, replaced or done without?

The buzz about open data in the UK has been ignited by inspiring individuals. The government has talked the talk (and written some cheques) and thousands of disparate developers in start-ups, bedrooms and the public sector, have done things that would have been unimaginable a decade ago. The UK tops a prestigious table for the performance with regard to open government data.

Despite the long list of initiatives, I still don’t have the data I would like to have. I would like to see information, statistics or data that:

  • Sheds light onto the state and changing nature of the UK economy and society. This is an area pretty much unaffected by the recent open data activities – we have no new stats of note and little additional insight. Access to existing data has not got much better, indeed has got worse in some cases.
  • Describes my local area – about schools, street cleaning, car ownership or parking tickets. This is partly to monitor local services and engage local communities but also to make life easier. These just don’t seem to have become available. If there are apps that would improve my life, I haven’t found them.

Open data in this context is not about personally disclosive data and it only relates to information held by the public sector – those data whose collection has been paid for by the people (tax payers) and are, in aggregate, about them and their activities.

In considering all this, I conclude the following themes:

Lots of talk and little action or strategy

There is an enormous amount of talk yet there’s been a mixed reaction from the public sector. A relatively small proportion of the information that could have been released has been released.

It’s hard to pin down what the government’s open data strategy is. It seems to be fragmented and lacking coherence. Reports are produced and not followed through in any robust way and various groups have been launched, only to then fade away.

Departments leading the transparency agenda are not transparent

The same applies to the various supporting transparency groups: they are lacking transparency in their structures (who works there, how much does it cost, how do you contact them and what do they do?) and often publish no basic admin info such as minutes of meetings. It is sometimes left to independent individuals or volunteer groups to publish minutes and to use FoI law to extract information.

Inconsistent enthusiasm for opening up data

Some government departments are far more willing to engage than others. I have been told that my characterisation is unfair but I see the Cabinet Office as pro open data (but often not very effective) and BIS largely against (and rather more effective).

With some exceptions, many other departments – and notably the ONS and UKSA – seem indifferent. The prime minister is apparently keen and has given good speeches but how much action has resulted? Rare are the cases where departments or their leaders have driven change – it seems to be individuals in departments that have produced new products. But the bottom up approach is not going to be effective on the really big issues.

Barely any core data

There has been a failure to progress big issues of core reference data – maps, weather, and corporate and land registers – from Public Data Group assets. The National Information Infrastructure has stalled under Cabinet Office leadership.

Inadequate promotion of open data benefits

The government has done little to promote the results of open data. More generally on access, there is no place for the public to find outputs or apps. And if data are poor or inaccurate there is no common or reliable way for users’ comments to be fed back. In addition, the website has been a disaster and makes it very hard to find information.

Apps have been for amusement not action. Some apps have made life easier or more entertaining at the margin, but they are mostly of fringe appeal and there are very few new statistics and little gain for policy analysts and the public.

Government expertise is lacking

Skill levels are too low so defensive attitudes lead to inaction. Within government, the cross cutting, inter-departmental professions (statisticians, economists, social scientists etc) are showing little interest. Few individuals in government use open data, so these practices are heavily under resourced and a long way from being business as usual.

Much of the interesting public sector data is now not under direct ministerial control and arm’s length bodies are in denial. Some ex-utilities have been privatised and this is a major issue to be addressed. There’s little interest in the private sector to open up data under their own steam and it’s hard for the centre to apply pressure.

There are some areas of considerable achievement in the public sector’s open data world, but the direction of travel is often painfully slow and has probably slowed since 2012. The new government needs a coherent strategy to deliver real change or the open data movement will stall in the face of modest achievements and campaigners’ exhaustion.


This is an edited version of two related posts on Simon's personal blog.

Open Data

Join the RSS

Join the RSS

Become part of an organisation which works to advance statistics and support statisticians

Copyright 2019 Royal Statistical Society. All Rights Reserved.
12 Errol Street, London, EC1Y 8LX. UK registered charity in England and Wales. No.306096

Twitter Facebook YouTube RSS feed RSS feed RSS newsletter

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies and Terms of Use.