Open Arms? Unlocking raw data

[Summary: Exploring the process of requesting access to a raw dataset]

Update 22nd December: Almost a month on, and whilst my post on the OPSI Data Unlocking Service has had 30 votes in favour (more than any other request I can see by far) I’ve not heard from either OPSI or the data owner/data.gov.uk in response to my comments/requests for raw data. So far, it looks like requesting new raw data through the advertised routes doesn’t meet with much action. I’ll wait till the Open Up competition closes in the New Year to see what results that might bring – and then it’s time to start looking at what other ways there might be to request this data…

A lot of the open government data that has been released in recent years is only available locked up in PDFs and website interfaces. As this definition seeks to explain this radically limits the potential uses of that data.

Following a recent event organised by Campaign Against the Arms Trade I was curious about who the UK issues Export Control Licenses to, so I took a look on data.gov.uk. Sure enough, the Strategic Export Controls: Reports and Statistics Website is listed on the Data.gov.uk catalogue. But on closer investigation it turn out that the Strategic Export Controls: Reports and Statistics Website (a) requires registration before you can access it; (b) predominantly provides data as PDFs; (c) has a very complex search interface that generates reports in the background ready for download later – but reports which don’t include key information such as the month a license was issued. All the data is clearly in the system – as you can search by date – but in it’s current form, to extract meaningful information about where UK companies have gained arms export licenses (or been refused) would be a long and slow job.

I’ve heard about the OPSI Data Unlocking Service, and I’ve been in a number of presentations hearing senior government officials and Ministers talking about the commitment of government to releasing raw data, so I thought this would provide a good opportunity to test the process of requesting raw data.

So – as of this morning, I’ve tried three routes to ask for access to this data:

  1. Adding a comment to the package on Data.gov.uk requesting access to the data. I’ve also sent a copy of the comment via the ‘Feedback Form’ listed under ‘Contact Details’ for each dataset. From past experience, I think the comment form gets forwarded to the Data.gov.uk team who forward it on to the department – but I’m not certain where that message has gone, or who reads the comments on datasets.
  2. Submitting a request to the OPSI Data Unlocking Service. This appeared to submit an e-mail form to the OPSI webmaster, who is, I understand, supposed to check the request and  then add it to the OPSI website for others to vote on – as well as – I presume, to someone inside OPSI to review and act upon – although the process by which a request could lead to data is fairly unclear. My request is not yet on the website.
  3. Adding an idea submission to the TSO Open Up Competition which you can see here. As I understand, the TSO are working closely with government on open data projects, although don’t have authority to open access to data themselves. However, there does appear to be an interest from the competition in what datasets people want to see – so I figured a request via here can’t harm.

I suspect a fourth route might be to submit a Freedom of Information Request, but I’m keen to explore in the first place how these open data requesting channels work in practice. Have I missed any? How else should be requesting access to raw data? Do you have experience of requesting data? What worked and what didn’t?

I’ll report back on any updates on the process of getting access to this data…

4 thoughts on “Open Arms? Unlocking raw data”

  1. Hi Tim,

    One query i can clear up is the role of data.gov.uk [I’m project managing the front-end development; thanks for your recent enhancement suggestions as well by the way 🙂 ] – the requests sent via the feedback form on individual datasets do come through to us and in this case are then sent onto the relevant departments for actioning. We also monitor comments added directly to the site and try to respond to as many as possible.

    It does look as if this specific request will be best dealt with by OPSI and i’ll be interested to see their response in due course.

    Hope this helps a little,

    Paul

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.