Tag: caat

Focussing on open data where it matters: accountability and action

A lot of talk of open data proceeds as if all data is equal, and a government dataset is a government dataset. Some open data advocates fall into the trap of seeing databases as collections of ‘neutral facts’, without recognising the many political and practical judgements that go into the collection and modelling of data. But, increasingly, an awareness is growing that datasets are not a-political, and that not all datasets are equal when it comes to their role in constituting a more open government.

Back in November 2010 I started exploring whether the government’s ‘Public Sector Information Unlocking Service’ actually worked by asking for open data access to the dataset underling the Strategic Export Controls: Reports and Statistics Website. Data on where the UK has issued arms export licenses is clearly important data for accountability, and yet, the data is kept in obfuscated in an inaccessible website. 14 months on, and my various requests for the data have seen absolutely zero response. Not even an acknowledgement.

However, today Campaign Against the Arms Trade have managed to unlock the Export License dataset, after painstakingly extracting inaccessible statistics from the official government site, and turning this into an open dataset and providing an online application to explore the data. They explain:

Until now the data, compiled by the Export Control Organisation(ECO) in the Department for Business, Innovation and Skills (BIS), was difficult to access, use and understand. The new CAAT app, available via CAAT’s website, transforms the accessibility of the data.

The salient features are:

    • Open access – anyone can view data without registering and can make and refine searches in real time.
    • Data has been disaggregated, providing itemised licences with ratings and values.
    • Comprehensive searchability (including of commonly-required groupings, for example by region of the world or type of weaponry).
    • Graphs of values of items licensed are provided alongside listings of licences.
    • Revoked licences are identified with the initial licence approvals.
    • Individual pages/searches (unique urls) can be linked to directly.
    • The full raw data is available as csv files for download.
And as Ian Prichard, CAAT Research Co-ordinator put’s it:

It is hard to think of an area of government activity that demands transparency more than arms export licensing. 

The lack of access to detailed, easy-to-access information has been a barrier to the public, media and parliamentarians being able to question government policies and practices. These practices include routine arming of authoritarian regimes such as Saudi Arabia, Bahrain and Egypt.

As well as providing more information in and of itself, we hope the web app will prompt the government to apply its own open data policies to arms exports. and substantially increase the level and accessibility of information available.

Perhaps projects like CAAT’s can help bring back the ‘hard political edge’ Robinson and Yu describe in the heritage of ‘open government’. They certainly emphasise the need for a ‘right to data’ rather than just access to data existing as a general policy subject to the decisions of those in power.

Open Arms? Unlocking raw data

[Summary: Exploring the process of requesting access to a raw dataset]

Update 22nd December: Almost a month on, and whilst my post on the OPSI Data Unlocking Service has had 30 votes in favour (more than any other request I can see by far) I’ve not heard from either OPSI or the data owner/data.gov.uk in response to my comments/requests for raw data. So far, it looks like requesting new raw data through the advertised routes doesn’t meet with much action. I’ll wait till the Open Up competition closes in the New Year to see what results that might bring – and then it’s time to start looking at what other ways there might be to request this data…

A lot of the open government data that has been released in recent years is only available locked up in PDFs and website interfaces. As this definition seeks to explain this radically limits the potential uses of that data.

Following a recent event organised by Campaign Against the Arms Trade I was curious about who the UK issues Export Control Licenses to, so I took a look on data.gov.uk. Sure enough, the Strategic Export Controls: Reports and Statistics Website is listed on the Data.gov.uk catalogue. But on closer investigation it turn out that the Strategic Export Controls: Reports and Statistics Website (a) requires registration before you can access it; (b) predominantly provides data as PDFs; (c) has a very complex search interface that generates reports in the background ready for download later – but reports which don’t include key information such as the month a license was issued. All the data is clearly in the system – as you can search by date – but in it’s current form, to extract meaningful information about where UK companies have gained arms export licenses (or been refused) would be a long and slow job.

I’ve heard about the OPSI Data Unlocking Service, and I’ve been in a number of presentations hearing senior government officials and Ministers talking about the commitment of government to releasing raw data, so I thought this would provide a good opportunity to test the process of requesting raw data.

So – as of this morning, I’ve tried three routes to ask for access to this data:

  1. Adding a comment to the package on Data.gov.uk requesting access to the data. I’ve also sent a copy of the comment via the ‘Feedback Form’ listed under ‘Contact Details’ for each dataset. From past experience, I think the comment form gets forwarded to the Data.gov.uk team who forward it on to the department – but I’m not certain where that message has gone, or who reads the comments on datasets.
  2. Submitting a request to the OPSI Data Unlocking Service. This appeared to submit an e-mail form to the OPSI webmaster, who is, I understand, supposed to check the request and  then add it to the OPSI website for others to vote on – as well as – I presume, to someone inside OPSI to review and act upon – although the process by which a request could lead to data is fairly unclear. My request is not yet on the website.
  3. Adding an idea submission to the TSO Open Up Competition which you can see here. As I understand, the TSO are working closely with government on open data projects, although don’t have authority to open access to data themselves. However, there does appear to be an interest from the competition in what datasets people want to see – so I figured a request via here can’t harm.

I suspect a fourth route might be to submit a Freedom of Information Request, but I’m keen to explore in the first place how these open data requesting channels work in practice. Have I missed any? How else should be requesting access to raw data? Do you have experience of requesting data? What worked and what didn’t?

I’ll report back on any updates on the process of getting access to this data…