Launching a social media experiment and piloting a metrics model: Aiming High for Young People

Aiming High WebsiteIf all goes well, I should be posting this just as we soft-launch the next of The National Youth Agency's* forrays into social media with the Aiming High for Young People: sharing, learning and developing action website. It's a companion website running alongside a series of regional events which are unpacking and exploring the recent Ten Year Strategy (Aiming High for Young People = Ten Year Youth Strategy) for young people launched by the UK Government.

The site, which should evolve and increase in interactivity over the course of the next three months, is based around three core elements:

  • A team blog – offering input from different staff members across The National Youth Agency – designed to capture and share insights and ideas from the events and from work within The National Youth Agency around the ten year strategy.
  • Reporting from regional events – where we will be capturing and sharing video clips, and slide-share powerpoint presentations on the site – and creating space for discussion around these. The video clips will offer narrative insights into examples of best practice. This will start next week after the first regional event (Tuesday 30th October).
  • Discussions space – based around free discussion and, drawing uponDiscuss the Youth Strategy the work of Comment On This, around a copy of the strategy itself. We've chunked the government strategy document up into small sections, each commentable upon – to encourage visitors to offer their commentary, examples of good projects and practice, shared learning and experiences relating directly to parts of the strategy. I'll write a little more about this soon…

As we progress we will hopefully make use of further social media tools. This exploration on social media builds upon earlier work with Youth Summit Live – a live-blogging experiment at a two-day event.

Social Media Metrics

It is, of course, no point making new steps into the social media space, if we don't know what it will achieve. This current launch isn't 'radical social media', but over time, through this project, and others, we will be adding new social media tools to our toolbox, and we need to be able to measure their impact and work out whether they are essential tools for everyone, or ones to 'keep in the cupboard' until we have a project they could really add to.

Beth's post that sparked this postSo to aid thinking around this, I've worked through Beth Kanters proposed model for Social Media Metrics below and filled in under the five headings what the goal, outcomes, metrics, measurement and map for the 'Aiming High for Young People: sharing, learning and developing action' website might be. And of course, in the interests of sharing and learning – I've shared that thinking here:

>>Goal: What is your most important organizational, business, program, or project goal?

The overall mission of The National Youth Agency is: supporting young people to achieve their full potential. The NYA works with organisations and services to improve the life NYA information leafletchances of young people and also works directly with young people themselves to develop their voice and influence in shaping policy and securing social justice.

With respect to this particular project – our aim is to provide information, insights and a space for dialogue that can support individuals and organisations across England in putting recent government policy for young people (Aiming High for Young People) into practice in the most effective ways possible. We're providing information, insights and space for dialogue in order to see services learn from each other, and work together, to provide the best for young people.

>>Outcomes: What changes? What happens? What would success look like to you?

The project and events sub-title provides a good framework for measuring outcomes. We want to see:

  • Sharing – not just from The NYA 'publishing' content on the Aiming High blog, and publishing videos on regional event write-ups. Rather, we want to see sharing from the field. Practioners sharing their own examples of best practice, their stories of challenges and ways they have overcome them, and their insights into steps that can be taken to improve services for young people.

    In that not much of this takes place online at present, encouraging practioners to engage with online spaces for sharing stories, challenges and sollutions will be a change.

    Success in terms of sharing would be to see the majority of content agregated through the Aiming High for Young People website originating not from The National Youth Agency, but from managers and practioners from across the country.

  • Learning – for both The NYA and managers and practioners in the field. Through gathering together stories, insights and discussion we should see new ideas highlighted, and areas for further investigation and work highlighted also.

    We should see a change in terms of how widely the captured learning from regional events is shared. Instead of just being available to delegates at each event, much of the learning should reach a far wider audience.

    Success would be sharing the learning opportunities around Aiming High for Young People with at least double the number of people who are able to attend the physical events.

  • Action developed – that makes a difference to the lives of young people. The sharing and learning that takes place through our social media work around Aiming High for Young People should impact on action.

    Change should be seen in the design and delivery of services for young people. Success would be being able to identify the impact of social media interactions on concrete actions.

As well as sharing, learning and developing action – we also look towards outcomes around networking. Helping develop and maintain further links between The NYA and organisations across the country – links that can be activated and drawn upon when The NYA can offer support to organisations in the future. And creating space for, and fostering, links between front-line organisations who can support each other with ongoing sharing and learning.

>>Metrics: What are the attributes or evidence you will measure?

Google analyticsIt's tempting to look at our metrics in terms of 'Access, Empowerment and Quality' (the three themes in the Ten Year Youth Strategy) – but probably Access, Engagement and Action are more suitable. Here are a few of the metrics we may look to track.

Access – How many people are accessing content provided by this project? Where are they accessing it? How are they coming to find it? What is the demographic of users?

Engagement – How many people are engaging with interactive features (and who?). What percentage of content is user-generated?

Action – What are the anecdotal 'stories of change' and case studies of change that can track some of their inspiration back to this project?

How will you measure? What data? What source?

Below is a quick list of the measurements that we may use to assess against these metrics.

  • How many people are accessing content provided by this project? Where are they accessing it? How are they coming to find it?
    Google Analytics and statistics from the Content Management System (Drupal)
  • What is the demographic of users?
    We can use a combination of user registration information. We should be able to infer some information from working out how many users access accessing from Local Authority Networks / National Government (usually have a reverse DNS for a address). However, we will probably want to look at using some sort of small survey later in the project to gain a deeper insight.
  • How many people are engaging with interactive features (and who?)
    I could look to set up a goal in Google Analytics for users adding comments, and can then track at-a-glance how interactive features are performing.

    Using the Content Management System I should be able to pull out information about the average number of comments each user is adding, and to build up a picture of whether interactive features are being used very actively by a small minority, or more evenly by a larger group (we may need to think more about which of these is more desirable if any).

    It may be possible to use user segmentation features in Google Analytics to track activity from registered users according to their organisation and region (they provide this information at sign-up).

  • What percentage of content is user-generated?
    For this also the data will come from the Content Management System .
  • What are the anecdotal 'stories of change' and case studies of change that can track some of their inspiration back to this project?
    This is more of an evaluation question than an ongoing 'metric' – and finding data to make an assessment in answer to this question will require more than online statistics.

    We will need to look at building in questions about our Aiming High for Young People social media work into future case study gathering processes and in guidance to staff for facilitating sessions around the Ten Year Youth Strategy in future.

    By seeking to build strong ongoing networks on the back of this one-off social media project, we should also be able to open up channels for gaining more informal data about the impact of the project on concrete action.

Map: What social media strategies will you use to reach your goals?

We're started off with blogging, encouraging RSS subscriptions, and allowing comments. We'll be looking at different platforms for video and photo-sharing soon. We will have to see whether we can add social networking to the mix or not, but we're certainly open to trying different tools and strategies responding to metrics, measurements and our developing goals along the way.



Reflections on the draft model
Beth's Social Media Metrics model is at the draft/pilot stage at the moment – and part of my motivation for taking the time to work through it (as above) has been to help try it out and generate some reflections on how it works. So, on reflection:

  • Pyramid of measurementsThere is very much a pyramid at work here. I can write one goal, which leads to a number of expected outcomes, each of which could be assessed against a range of metrics, each of which can be measured by a range of measurements. It could easily become unmanageable at the measurements level if you're not careful higher up. Should I have a pyramid? Or should it be a neat chain of measurements, metrics and outcomes.

    Perhaps it I hadn't been working 'from the top down' I would have ended up with a different story. In my outline above each outcome has it's metrics, each metric has it's measurements – where quite possibly one measurement could tell me something about more than out outcome.

    This said, working from the goal does mean I've not limited myself to easy-to-get measurements only.

  • I'm not clear enough on the difference between metrics and measurements in the model. This is likely down to me not spending enough time reading Beth's post and working on this from the train with very intermittent internet access that has prevented me from following links… but the distinction between 'the attributes I'll measure' and their 'data' and 'source' seems fairly slim in some cases.
  • The model hasn't asked me to think enough about 'when' I'll take measurements. That might be the next step, but working out which of my measurements should be taken regularly and which should be ad-hoc would help me in breaking the bottom level of the pyramid into managageable chunks.

Overall – however – I've certainly been encouraged to do a lot of thinking here… and thinking that, I hope to be able to blog soon, will really add to our ability to make the most of the opportunities for social media to support 'Learning, sharing and developing action' around Aiming High for Young People.

*Context: I work in a consultancy capacity with The National Youth Agency on youth participation and introducing social media into the organisations work.

4 thoughts on “Launching a social media experiment and piloting a metrics model: Aiming High for Young People”

  1. Great stuff here, thanks for a great case study. One thing you might think about is to do some regular relationship measurement based on the Grunig model.The actual survey instrument is available here:
    in the paper called Guidelines for measuring relationships. You’ve got great quantitative data, but I’m thinking that you’re not going to know if you really had any impact on the Youth unless you ask them.

  2. Spot on. The National Youth Agency does a lot of work about measuring the impact of interventions and involvement in decision making on young people and young people’s lived experiences.

    We’re rather fond of the quote from Lord Herman Ousley (paraphrased here): “If the people who are supposed to benefit from change can’t tell you that they have, they probably haven’t”.

    But I’ll certainly check out the Grundig model to see what we should be doing about adding relationship measurement to the mix.

  3. Tim,

    This is fantastic! I’ve been on the road and I’m finalizing the presentation. I’m going to reblog this right now! Thank you for taking the time …

    I think the key to select one or two key metrics to evaluate each outcome. Then, I added another column – “value” you translate into dollars the value of your success.

    Also, don’t forget that you also want to use the metrics to make adjustments that lead to effectiveness in your strategy – as well as learn more about your community.

    Thank you again and I apologize for not being able to write this comment the instant you posted, but been on the road.

  4. This is my favorite quote — from my Father when he was managing editor of Fortune: “If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,
    capricious, motivated by innumerable conflicting interests, and conflicting desires.”
    Ralph Delahaye Paine, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.