Questionable consultation

I don’t get the feeling Oxford City Council really value my view. At least, the consultation on local priorities they’ve just published certainly fails to inspire me with confidence that the council are aiming for a meaningful public dialogue on their corporate priorities.

Oxford City Council ConsultationI’m asked to indicate, from a list of issues plucked right out of a corporate plan, whether the issues are a ‘High Priority’, ‘Medium Priority’ or ‘Low Priority’ for me. There is no explanation of what indicating my priorities will lead to; no links to further information on what the council is already doing against a given priority; and only a minor attempt to tranlate a few into non-council-speak.

At the end of the online form I’m asked to go back and give the numbers of the priorities that are my top overall priorities – again with no explanation of how this input will be used.

Oxford City Council ConsultationInterestingly ‘Improviding dialogue and engagement with citizens’ is not amongst the priorities I’m invited to evaluate in this consultation. I would suggest to Oxford that perhaps it should be.

I’m left with the distinct feeling that this consultation could end up an exercise in statistical manipulation, with ‘high priority’ scores used to justify expenditure on services the council is already planning to resource, and lower scores used to justify cuts to other services – when, given two services side-by-side and asked to choose which they would prefer funded or cut, residents actual preferences may not match up at all with those that may be derived from this consultation exercise.

Creating good consultation

Of course, I prefer not to rant in the negative, so below are a few quick thoughts and reflections on what I’ve found makes a good consultation over the years:

  • Be clear about the scope and impact – tell respondents in plain English how their views will be used. Perhaps show an example of a previous consultation. Or include a short video clip of the person who will process the responses explaining the consultation. If you can’t explain how there is potential for a consultation to influence decision making and create change…well, stop and rethink whether you should be running a consultation until you can!
  • Provide context. When designing an online consultation think about how you can provide extra information on the different topics you are consulting on, so that citizens can read up more about what you mean by a given question, and have the chance to be deliberative in their decision making.

    Videos, photos, links and other context can all prove useful.

  • Think carefully about your questions. If you don’t know how you will use the answers to a question – don’t ask it. If you think that the question could be interpreted in many different ways – clarify it.

    Think as well about how questions answers could be ‘gamed’. If, for example, I suspect that all the responses from Oxford’s consultation will be simply totalled up in a spreadsheet, and there are certain elements of the consultation I have a strong interest in, I’m better off exaggerating my high priorities and rating everything else lower to help highlight my particular preferences. Is your consultation designed to avoid this sort of skewing?

  • Be honest about your limitations – and build the limitations into your consultation. Presumably even if I say everything in Oxford’s corporate plan is a high priority for me, some things will be allocated more resources than others. The consultation is at least in part about distributing limited resources. So instead of asking me if things are high-priority or not – why not ask me to rank the priorities between certain services? Or show some realistic sense of what rating a given service as a high priority might entail so I can think carefully about my priorities in the context of the real limitations around priority setting.

    Asking anyone ‘What do you want?’ is foolish without framing. ‘What do you want us to do with the resources we have and without significantly raising taxes?’ is a far better question.

  • Check your process. Think about how you will analyse responses before you start. Would someone who completed your consultation recognise the analysis  you have made of their input? If not – think about how you can improve the whole consultation process so that what comes out the far end at least approximately represents the input that citizens contributed in the first place.

  • Design and test your forms. Ask a number of people to test your consultation. Watch them complete your forms. Ask them what could be improved to make the process smoother. Far better to spend an extra day taking the rough edges off, than to have hundreds of responses which can’t be used.

    (For example, in the Oxford Council consultation above I was far more likely to indicate priorities from the bottom of the list as my overall top priorities (or at least as my third overall priority… I had a pretty clear idea of the first two I was asked for…) as I had read them most recently and could still see them on screen without scrolling when I came to reply to the questions which requested their numbers. If this is true for a large number of people completing the consultation then that could introduce a significant bias…)

  • Consult to start a conversation. Most important of all – think beyond consultation. How can your consultation questions provide the start of a wider discussion? How can you provide platforms for ongoing dialogue beyond the simple asking of consultation questions?

What other tips would you provide to someone designing a consultation?

Have you tested that online survey?

If you consult young people by running a series of face-to-face workshops then the chances are that after you’ve run the first workshop there will be things you’ll want to adapt for future sessions. Unless you’re working with a very fixed research methodology, you may even adapt the workshop as you go – responding to the prior knowledge and needs of the young people you are working with.

I’ve often had to add new activities into a workshop, or take some out to accommodate the particular levels of interest, background knowledge and literacy of the young people I’m working with.

But when you consult with an online survey, the same ‘feedback loop’ that allows you to check if you are pitching the questions right, and to adapt, doesn’t always exist. And that makes it really important to get young people involved in the design of your online survey, or at least to try out a draft with members of your target audience for the real thing.

Otherwise you end up with examples like this – packed full of assumed knowledge, jargon and questions structured entirely around a policy agenda rather than young people’s lives.

When it comes to delivering consultation online, thinking about accessibility matters more than ever.

SMSPoll – Accessible big screen voting

If you're looking for a way to run a quick mobile phone based consultation – or you've been wanting to use interactive voting at a conference or participation event – but haven't been able to afford expensive e-voting equipment – then you might want to take a look at SMSPoll.net.

 The service (which I just discovered this evening and have only briefly tested) lets you set up quick polls – which anyone you tell about them can vote on by sending a text message to a UK number. The results are updated in near real-time on the website – meaning you can get an 'ask the audience' style effect if you project the graphs that SNSPoll generates onto a big screen.

It's free for small polls (25 votes or less) and is very cheap for larger polls (from £5 month). 

When using it with groups of young people you would need to think carefully about any text message costs it might lead to – but even so – giving each young person £1 toward text message costs and having a few mobile phones to loan to those without them (or more likely, with no credit) at an event could potentially overcome those barriers.

If you do make use of SMSPoll – then I'd love to hear about how it works for you. Or if you've used any alternative systems perhaps you could share details of those and how they worked in the comments…

Online consultation on science and society

Thanks to tweets from Dave Briggs and a blog post from Simon over at Puffbox I've just been exploring Steph Gray's quite fantastic innovations over the Science and Society consultation website.

Not only does the site work hard to make a complex consultation more accessible through the use of video introductions and a blog format – but it sets the consultation free, and let's any visitor select questions from the Consultation to make available on their own websites via a custom widget – which feeds information right back into the core consultation.
So – bloggers, schools, youth services – anyone with a web presence where they can add in a little widget code can help their users and audience engage with the Science and Society consultation in a manageable way.

For example, the widget below (you may not see this if viewing in an RSS reader – so click through to the main post to take a look) displays a couple of questions I thought readers of this blog might be interested in answering.

Science and Society: your views

Please visit the Science and Society consultation site to join the debate.

This approach of enabling citizens to easily take, remix and re-publish government consultations to their networks is worth exploring in many more contexts – not least in promoting positive activities, enabling young people to take, remix and share information about positive activities in their areas with their networks.

Consultation games in the real world

I've explored the role of games in consultation before, but never bringing together the mix of playful real-world games, person-to-person interaction, and digital consultation and dialogue in quite the way that the process Kevin Harris describes in this recent blog post on 'Community engagement by treasure hunt'.

Kevin combined a quiz-book based treasure hunt around the site of the library due to be redeveloped with opportunities to speak to architects, chances to text-in ideas and an invitation to record reflections in the quiz-book. Kevin writes:

A key advantage of the treasure hunt was that it avoided those inactive ‘pools’ and conceptual congestion that you can get, where people stand around repeating the same points based on their own advance agenda. We have tried in the consultation exercises so far to be clear about what is negotiable and what is not, to avoid the risk that people get frustrated asking for something that is not on offer for whatever reason.

Another point is to see it more in terms of engagement than consultation. The exercise was only partially about the latter and we hold no presumptions about the depth of comment to be found in the eBooks and certainly not in the SMS exercise. These devices and processes are part of the mix of engagement which goes on and hopefully will strengthen and bear fruit

The whole excercise looks like a brilliant exercise in creative and playful consultation, building the constraints of a process in from the start, and equipping people with ongoing tools for future dialogue.

I'll certainly be looking at how I can learn from this sort of creative consultation in future.

(eBook image from Kevin's blog)

One page guide: online surveys

Online surveysI wrote this one page guide on running an online survey in response to a suggestion from Damien at ChangeMakers Virtual Volunteering programme, and to go towards a section on online consultation and participation I've been putting together for Participation Works.

You can download the guide for printing here (PDF), or for editing here (Word doc).

The guide gives an overview of setting up and running an online survey with SurveyGizmo.com. The sharp-eyed reader will notice that in fact the screen-shot in the guide is of a SurveyMonkey survey. This is no particular reason for this other than I had that particular survey open at the time. And it shows diversity.

As with all the guides in this series, it is aimed at someone who has perhaps heard of online surveys (or blogs, rss and wikis etc.), but doesn't really know what they have to offer or how to get started. The guide is designed to at least show that it's not that scary – and that these online tools have real practical applications.

I'm planning to experiment with some more 'platform agnostic' guides in the near future – but so far I've found that because every provider names things slightly differently ('analyse responses', 'create report' etc.) it gets quite difficult to create something that will help a new user feel secure rather than worried…


Attachment: Online Surveys.pdf
Attachment: Online Surveys.doc

The twitter post: txt for conferencing and consultation

(The twitter post: Well, it had to come sooner or later…)

Twitter from YOMO Event

I've just returned from an event in Chester (YOMO's Practical Ideas for Participation gathering) where we were making use of a tool called twitter to collect and share instant feedback throughout the event direct from people's mobile phones. The image above shows the feedback we got at the end of the event, all sent in by text message. With Twitter you can...

This has been my first large-scale experiment with twitter, and so shared below you will find:

  • A quick account of how we used twitter and a creative commons briefing you can adapt for using twitter at your conferences.
  • A reflection on the potential for twitter as a consultation/participation tool, and an invitation to suggest a pilot project.

Conference twitter for feedback
We set up a conference twitter account, and asked delegates to follow our account via mobile phone (by sending two sign-up text messages).

Throughout the event we were able to send instant text messages to all delegates – letting them know about what was coming up next, and inviting feedback. And delegates were able to text in their reflections, questions and feedback – with their views instantly appearing on the 'twitter wall' projected up on the main room, and on tickers running along the top of each powerpoint presentation being given.

Twitter briefingIt cost us nothing to set up. And it provided some really insightful gut-reaction instant feedback throughout the event.

The briefing paper I used to get people started using twitter at the event is attached at the bottom of this post.

It's not quite the same as the rest of the 'One pager…' series, as you will need to adapt this to your context if you want to use it. You will find comments in the margins giving you information on what you need to get set up for that.

Community twitter for consultation and participation

Twitter is a very flexible platform for building social networks. In general, it will work something like this:

  • People opt to follow your updates via the web, their mobile phone, or an instant messenger (gtalk).
  • You write an update.
  • Your followers receive your update on the web, by instant message or by text.
  • They can reply to you by instant message, web or text message either public ally, or privately.
  • You can read all the responses by phone, on the web or by instant message.
  • It doesn't cost anything more than the standard cost of any text messages involved.
  • If you are asking for public replies, then it would be possible to share the question and replies with others by pointing them to your twitter page on the web.

Some twitter users treat it as a way of keeping in touch with a geographically dispersed team. Some twitter users micro-blog using it to alert others to what they are up to.

From twitter.com

But, if you're thinking what I'm thinking – you might spot that there is a powerful tool for youth participation here. Imagine this scenario:

  • People opt to follow your updates via the web, their mobile phone, or an instant messenger (gtalk) – you ask young people across the community to follow your updates by phone, building up a large groups of 250 'followers' across the community.
  • You write an update – when you need to gauge ideas in the area on a particular issue. You pose a short question.
  • Your followers receive your update on the web, by instant message or by text – hopefully as many as possible receive the message by text soon after you send it.
  • They can reply to you by instant message, web or text message either public ally, or privately – you ask for public replies and within an hour you have short text feedback and ideas from 90 young people. You send a text an hour later thanking everyone from feedback and letting them know you no longer need replies.
  • You can read all the responses by phone, on the web or by instant message – instantly gaining a deeper insight into different young people's views on an issue. If this helps you make a decision or make a change, you can send an update to provide instant feedback,
  • It doesn't cost anything more than the standard cost of any text messages involved.
  • If you are asking for public replies, then it would be possible to share the question and replies with others by pointing them to your twitter page on the web – you could send a link to the views to a local councilor to ask them to read young peoples views directly.

I'm not aware of any groups making use of twitter in this way yet (though I would be suspired if there aren't some out there applying it like this – do get in touch) and I would be very interested in supporting a pilot project.

Other applications

For more on applications of Twitter, you might want to check out


Attachment: Twitter briefing for conferences – draft.doc

Consultation responses: putting some backbone into it

One of the joys I find in blogging is that just when I'm strugging to find a way to express an idea, I stumble across an idea with similar roots elsewhere that can, hopefully, help make sense of what I was thinking about.

Just such a thing occured with this post from Annecdote about 'Story Spines'. As Shawn explains:

I asked the groups to grab an issue and tell a story explaining what happened. People busily jumped into the activity but I noticed they were just writing dot points detailing their opinions about what had happened. No one wrote a story.

It seems that they didn't know what to do to write a story. I had just assumed that everyone else thinks about stories like I do and has a sense what one looks like. Big mistake!

My next opportunity was at another knowledge strategy workshop but this time with a government department in Canberra. I had remembered Andrew introducing us to story spines so I dug out the blog post. Here is the simple story spine (Viv's example is more elaborate).

Once upon a time…
Everyday…
But one day…
Because of that… (repeat three times or as often as necessary) Until finally…
Ever since then…
And the moral of the story is…(optional)

The Story Spine helps groups to structure their responses to a question or request into the format that is needed to move things forward.

What has this got to do with consultation you might ask? Well, as I was working with colleagues to analyse the notes from flipcharts created at the 3D Dialogues we came to realise that a lot of responses lacked verbs.

'What is good about this?' 'Youth Workers'.

But youth workers what? 'Youth workers helping us'? 'Youth workers being there but staying out the way unless we need them'? 'The particular youth workers we know'? 'Any youth workers'?.

The form of answer illicited by the standard flip-chart recording isn't really what we need to make sense of the dialogue and discussion that has taken place. And that is where some sort of 'spine' for consultation responses might come in handy. A mechanism for encouraging statments with enough specificity to be able to feed meaningfully into future decision making.

So what should the spine for a consultation response look like?

When we were looking at the 3D response we discussed preparing a series of cards with a range of verbs on, and then asking those recording on flip-charts to make sure everything written up includes at least one of these cards – but I fear that a manageable set of verb-cards might be too limiting.

Perhaps instead simply providing some sentence starters like:

“We would like it if…”

“We need…”

“Things are better when…”

Would encourage the more complete sorts of responses we need.

I'll certianly be looking to try something along these lines for the next consultation I'm involved in… and I'll be sure to report back with how it works…