Questionable consultation

I don’t get the feeling Oxford City Council really value my view. At least, the consultation on local priorities they’ve just published certainly fails to inspire me with confidence that the council are aiming for a meaningful public dialogue on their corporate priorities.

Oxford City Council ConsultationI’m asked to indicate, from a list of issues plucked right out of a corporate plan, whether the issues are a ‘High Priority’, ‘Medium Priority’ or ‘Low Priority’ for me. There is no explanation of what indicating my priorities will lead to; no links to further information on what the council is already doing against a given priority; and only a minor attempt to tranlate a few into non-council-speak.

At the end of the online form I’m asked to go back and give the numbers of the priorities that are my top overall priorities – again with no explanation of how this input will be used.

Oxford City Council ConsultationInterestingly ‘Improviding dialogue and engagement with citizens’ is not amongst the priorities I’m invited to evaluate in this consultation. I would suggest to Oxford that perhaps it should be.

I’m left with the distinct feeling that this consultation could end up an exercise in statistical manipulation, with ‘high priority’ scores used to justify expenditure on services the council is already planning to resource, and lower scores used to justify cuts to other services – when, given two services side-by-side and asked to choose which they would prefer funded or cut, residents actual preferences may not match up at all with those that may be derived from this consultation exercise.

Creating good consultation

Of course, I prefer not to rant in the negative, so below are a few quick thoughts and reflections on what I’ve found makes a good consultation over the years:

  • Be clear about the scope and impact – tell respondents in plain English how their views will be used. Perhaps show an example of a previous consultation. Or include a short video clip of the person who will process the responses explaining the consultation. If you can’t explain how there is potential for a consultation to influence decision making and create change…well, stop and rethink whether you should be running a consultation until you can!
  • Provide context. When designing an online consultation think about how you can provide extra information on the different topics you are consulting on, so that citizens can read up more about what you mean by a given question, and have the chance to be deliberative in their decision making.

    Videos, photos, links and other context can all prove useful.

  • Think carefully about your questions. If you don’t know how you will use the answers to a question – don’t ask it. If you think that the question could be interpreted in many different ways – clarify it.

    Think as well about how questions answers could be ‘gamed’. If, for example, I suspect that all the responses from Oxford’s consultation will be simply totalled up in a spreadsheet, and there are certain elements of the consultation I have a strong interest in, I’m better off exaggerating my high priorities and rating everything else lower to help highlight my particular preferences. Is your consultation designed to avoid this sort of skewing?

  • Be honest about your limitations – and build the limitations into your consultation. Presumably even if I say everything in Oxford’s corporate plan is a high priority for me, some things will be allocated more resources than others. The consultation is at least in part about distributing limited resources. So instead of asking me if things are high-priority or not – why not ask me to rank the priorities between certain services? Or show some realistic sense of what rating a given service as a high priority might entail so I can think carefully about my priorities in the context of the real limitations around priority setting.

    Asking anyone ‘What do you want?’ is foolish without framing. ‘What do you want us to do with the resources we have and without significantly raising taxes?’ is a far better question.

  • Check your process. Think about how you will analyse responses before you start. Would someone who completed your consultation recognise the analysis  you have made of their input? If not – think about how you can improve the whole consultation process so that what comes out the far end at least approximately represents the input that citizens contributed in the first place.

  • Design and test your forms. Ask a number of people to test your consultation. Watch them complete your forms. Ask them what could be improved to make the process smoother. Far better to spend an extra day taking the rough edges off, than to have hundreds of responses which can’t be used.

    (For example, in the Oxford Council consultation above I was far more likely to indicate priorities from the bottom of the list as my overall top priorities (or at least as my third overall priority… I had a pretty clear idea of the first two I was asked for…) as I had read them most recently and could still see them on screen without scrolling when I came to reply to the questions which requested their numbers. If this is true for a large number of people completing the consultation then that could introduce a significant bias…)

  • Consult to start a conversation. Most important of all – think beyond consultation. How can your consultation questions provide the start of a wider discussion? How can you provide platforms for ongoing dialogue beyond the simple asking of consultation questions?

What other tips would you provide to someone designing a consultation?

2 thoughts on “Questionable consultation”

  1. Tim,

    As someone who works in the world of engagement and consultation in local government I think you’ve raised a range of interesting points, most of which I would be inclined to agree with.

    However I think these comments should be balanced with a number of other considerations that need to be factored in. I completely agree with you point about providing context. However, there is a limit to the amount of background information you can provide participants before this acts as a disincentive to completing the questionnaire…And I think it’s fair to say adding context to corporate priorities could get quite wordy!

    Alongside this I’ve always found consulting on strategic priorities to be somewhat problematic. I’d be lying if I said I’ve never constructed a similar looking question, but equally when I have I’ve also been aware of the constraints of the approach. Although I’d always ask ‘why do we need to ask this?’ It often has to be balanced against wider internal and/or external political pressures. Perhaps in defence of my colleagues at Oxford I’d hope they at least acknowledged these dilemmas when agreeing their approach, even if in the world of evidence based decision making concessions do have to be made!

    Overall I guess this all links back to recognising your limitations.

    Anyway, keep up the good work!

    Ed

  2. Hey Ed

    Thanks for the really useful comments. I certainly agree that there are many dilemmas and issues to be balanced in designing a consultation and engagement process – and it is always a case of compromise rather than perfect process (as underlying a consultation are political decisions, inevitably decisions needing negotiation and compromise).

    Providing context is tough – but I’m not sure that needs to be because it can become too wordy. A well designed short video clip, or a few slides written in plain English can give a sense of what a question is about (of course, care needs to be taken to make sure these don’t introduce bias into the question asking), and these can in the online environment be provided as optional extras to view for the consultee – rather than hurdles to be leapt over before a question is answered.

    One theme certainly worth exploring is what a fair online consultation looks like when we see taking consultation online not as just putting paper based forms on the internet – but we take advantage of a rich online environment to seek better quality deliberation and citizen input into decision making.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.