Blog Post

Evaluation approaches in community contexts

30.08.17

Sarah Davies on a new resource sharing a range of approaches to evaluating arts projects in community contexts 

Ribbon Wheel - Nicola Winstanley and Sarah Nadin

We don’t have all the answers but we do have lots of questions. Sarah B Davies on how Creative People and Places projects are developing a range of approaches to evaluation.

How many people attended our event? How many of them were new to the arts? What are the key demographic findings and how does this compare with our target? How did our event measure against an identified set of quality indicators? How do we know? What have we learned about our processes for engagement? What do our audiences want to happen next?

These are all good questions, asked generally across the arts, which project evaluations strive to explore. But evaluating coherently across a programme can pose new questions. Local arts programmes might evolve, responding to public need and taste. They might include large-scale outdoor spectacle and intimate creative workshops, and might comprise opportunities for people to interact as audiences, participants and co-creators. How can an evaluation approach apply a consistent set of questions across a broad and varied programme, to coherently capture and hold all the learning? How can it be effective in gathering, storing and analysing the quantitative and qualitative data necessary, in the right ways, at the right times and in the right contexts? How can an evaluation framework nurture these varied programmes that kindle such great local interest?

Within the National Evaluation Programme we organised annual events to facilitate the sharing of learning around evaluation methods between the 21 Places. Our aim for these events was to create a space for Places to share and discuss local evaluation processes and to use this collective learning to strengthen the local and national evaluations.  These sessions were often followed by lively debates and feedback where support, critique and questioning took place within an ethos of peer learning. No one comes with all the answers, indeed they come with many questions, but they can each find a way to share as much as they learn.

Many of the challenges explored during these sessions are not unfamiliar in a wider context. For example:

  • Demographic data gathering and postcode collection - when it is so important to know our audiences better, what mechanisms are manageable and effective, and which methods work well for different types of event?
  • Tracking participation across a broader arts programme – how can we learn more about who is attending our events and coming back for more? How do we measure this?
  • Advocating for our programmes to non-arts partners and funders – how do we explain the value of the investment to our partners in a language these associates understand and appreciate?
  • Defining excellence – while there are so many debates around excellence and quality, how can we navigate this to find the right definition for our programmes and audiences?
  • Collaborative evaluation – can we model our collaborative relationships with partners and participants to create an evaluation approach that is driven by the knowledge held by our collaborators?
  • Creative approaches to evaluation – how can alternative approaches to research aid our evaluation and share our learning in new and accessible ways?

It is the voices of this network of people, making their way through these challenges, that have come together to create Evaluation in Participatory Programmes – a compendium of evaluation approaches, methods and tools being explored within the Creative People and Places programme. The compendium contains a broad range of examples, submitted by CPP teams, including:

  • There are examples of creative evaluation tools aimed at increasing the number of surveys completed and the depth of demographic information gathered
  • There are case studies detailing learning around different approaches to quantitative and qualitative processes
  • And artists, commissioned to work within our evaluation frameworks, share their methodology and creative responses, which includes illustrations, poetry and lithographs amongst other reflective artworks

These examples were put forward in the spirit of sharing our learning with a wider group of arts practitioners, arts programmers and evaluators, to start to open up new conversations within the arts sector. We don’t have all the answers, but by questioning and examining through our evaluations we hope we will continue to understand the nature of participation in our programmes and uncover rich learning about our processes along the way.

You can download Evaluation in Participatory Programmes here.

Sarah B Davies
Sarah managed the first phase of the CPP National Evaluation Programme from 2013-16 on behalf of A New Direction.

Image: The Ribbon Wheel – a creative consultation tool developed by artists Nicola Winstanley and Sarah Nadin for Appetite’s Big Feast festival. 
Image courtesy of Nicola Winstanley and Sarah Nadin.