The report works with the theory of change pdf=328kb developed for Big Local (2013) in that it aims to capture:
The findings in this report are based on:
Table 8: Our Bigger Story: Summary of evaluation activity 2015-16
|Evaluation Activity Year 1 (2015)||Evaluation Activity Year 2 (2016)|
|Review of existing Big Local data – including community profiles and plans as well as materials (such as film, podcasts and photographs) available in other media||Ongoing reviews of Big Local data (e.g. Plan Reviews).|
|102 individual interviews with key stakeholders - including resident partnership members, workers, delivery partners, Locally Trusted Organisations, volunteers and ‘beneficiaries’ (as described by a number of partnerships),||55 individual interviews and conversations, with partners, delivery partners, recipients of small grants and local people accessing Big Local supported projects, and Local Trust|
|Observation sessions at partnership meetings/plan reviews and Big Local events, ten focus groups involving 54 resident partnership members, delivery partners and beneficiaries.||15 facilitated thematic discussions (i.e. all 15 Big Local case study sites), involving 157 participants 13 observation sessions at partnership meetings/plan reviews and Big Local events|
|Four networking workshops in Ramsey, Leeds, Birmingham and Hanwell to explore emerging Big Local themes and outcomes involving 14 areas|
|Focus Groups with Big Local reps (two) and Local Trust staff|
|Resident partner diaries in seven areas|
Our Bigger Story has also built a website which brings together:
This site, therefore, allows for the tracking of change in Big Local areas over the life of the programme and can be searched either by media format (e.g. film/audio etc.) or against a time-line of when things happened.
This report maps progress in, and additional learning from, the evaluation sample of 15 Big Local areas 2015 to 2016. It takes the story of Big Local forwards from the Early Years pdf=2.76mb research undertaken by NCVO during 2014 (NCVO, 2015) and builds on other subsequent evaluation reports such as Community Engagement in Big Local pdf=595kb (NCVO 2016) and People places and health agencies pdf=458kb, (Institute for Voluntary Action Research report 2016).
This report aims to explore:
These are summarised in Table 9, below.
Table 9: Our Bigger Story: Core Evaluation focus
|EVALUATION - PROCESS FOCUS||EVALUATION - OUTCOME FOCUS|
|Resident-led (approach)||Residents better able to identify and prioritise needs (outcomes)|
|Asset-based (approach)||Residents better able to respond to needs (outcomes)|
|Taking place at community pace (approach)||People have increased skills and confidence (outcomes)|
|Opportunities for reflection (approach)||Residents improve Big Local areas in ways that matter to them (outcomes)|
|Taking risks (approach)||A better place to live (outcomes)|
|Local Trust makes funding available in ways that are accessible and user friendly (delivery)||Infrastructure supports resident decision making (impact)|
|Lasting and sustainable change in communities (impact)|
|Government policy facilitates resident-led decision making (influence)|
In addition, in 2016, the evaluation team introduced five key research themes to ‘unpack’ learning around process and outcomes, as agreed with Local Trust. These related to Big Local areas capacity to negotiate change, the extent to which they are a catalyst for change in their areas, the development of community leadership and influence, their experience in dealing with expectations around ‘achievement,’ and understanding about sustainability and legacy. These are explained further in Section 4 of the current report.
Big Local aims to bring about positive change in communities. The aim of Our Bigger Story, as a longitudinal multi-media evaluation, is to record that change, as well as the learning involved, in multiple ways in 15 Big Local areas over time. Considerable attention was given during 2015, the first year of the evaluation, to building the architecture to support what is an ambitious, complex, and long-term study. This has involved:
Selecting and recruiting evaluation sites
The evaluation is being undertaken in and with 15 Big Local areas, representing 10% of all the communities covered by the Big Local programme. These were selected through a short baseline survey of all 150 Big Local areas carried out between February and May 2015. The survey was designed to assess interest in involvement in Our Bigger Story and to identify the range of social media technologies currently being used by areas. It was accompanied by taster workshops at the 2015 Big Local spring events, attracting 51 participants.
In total 35 Big Local areas expressed an interest in participating in the evaluation. These were assessed against criteria agreed with Local Trust which were designed to ensure a balanced sample covering, for example, all regions in England, a mix in terms of urban/rural/coastal areas and differing population sizes. The final group of 15 selected evaluation sites includes:
Inception meetings were held with each of the 15 selected Big Local areas, in order to ensure that each had taken a strategic, collective and informed decision to participate. Big Local areas asked questions around confidentiality, time expectations and the availability of support from the evaluation team, ownership and archiving of evaluation materials. Residents were particularly concerned about the film making aspect of Our Bigger Story as the period of negotiating access to areas coincided with the airing of Benefit’s Street on television and substantial time has been dedicated to :
Web-site and data-base development
Our Bigger Story has started to construct an integrated public facing website which will document multi-media evaluation materials from all 15 Big Local areas over the life of the programme (see www.ourbiggerstory.com). This is searchable by a time-line, by the media used and against Big Local outcomes. It enables the evaluation team and individual Big Local areas to up-load materials (video, audio, photographs and reports) that are for wider circulation and do not include information that could be deemed to be confidential. There are currently 238 ‘posts’7 from the 15 Big Local areas on the website alongside a vimeo channel (The Residents Stories) with 34 films made by OBS over 2015-16.
There have been a number of teething problems with the website. For Big Local areas with slow broadband speeds, uploading films can be extremely time consuming. Internally, the practice of uploading materials has identified a number of layout problems – in particular the matching of text with photographs of different sizing. Both these issues are being addressed by Creative Media through advice to the areas and web-redesign.
Behind the public facing website, systems for secure online data storage and management have been developed as part of building the foundations for the longitudinal study. Given the volume (and file size) of visual materials additional secure storage has been purchased from the University. This has the capacity to preserve a decade’s worth of written, audio and visual data.
Evaluation activities year one (2015)
Evaluation activity over year one consisted of:
The focus in year one was on gaining a baseline understanding of each of the 15 areas: their histories, the patterns of (and learning from) development behind Big Local and what they hoped to achieve. The methods adopted relied largely on more traditional evaluation methods (e.g. taped/transcribed individual and focus group interviews) supplemented by film making and other multi-media activity (e.g. podcasts). This data informed the Interim Summary Report pdf=592kb (with a more detailed paper submitted to Local Trust) published in April 2016.
Evaluation activities year two (2016)
Year one involved substantial learning for the team around the integration of multi-media methods into a longitudinal evaluation. In discussion with Local Trust and the 15 areas involved, a slightly different approach was adopted in 2016 in that Our Bigger Story:
In addition, the focus for data gathering (post base-line) changed. In addition to gathering information on area’s progress against outcomes a more thematic approach was adopted to capture data around the themes of leadership, influence, expectations, legacy and Big Local as a change mechanism.
Research activity has, therefore, focused more on ‘whole partnership’ facilitated workshops (with fewer individual interviews with partners and residents), recording activities supported by Big Local areas (e.g. the use of small grants) and the use of ‘cross partnership’ events to bot stimulate discussion and gather data. Over the year this involved:
In addition, residents in seven areas kept reflective diaries on their experience of being Big Local partners. An edited film version of these diaries is available as ‘Diary Lines’.