- Description of the programme
- The study
- Key findings
- Points to consider
Description of the programme
[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]
Citizens Advice Basingstoke secured funding from the ‘What Works Fund’ to design, test and evaluate an intervention aimed at improving the financial capability of vulnerable young adults. This involved the delivery of a training course of 10 weekly sessions, to three cohorts of between 10 and 11 participants. It was delivered during 2017-2018 in partnership with YMCA Basingstoke - Cohorts One and Two - and Basingstoke College of Technology (BCoT) - Cohort Three.
Key features of the intervention were the co-design and co-delivery of content by peer mentors and use of social media to promote engagement and learning.
The overall aim was to understand to what extent an intervention co-created by vulnerable young adults and supplemented by peer mentoring and social media activity, improved the financial capability of young adults in similar circumstances.
The study
The research questions were:
- Does this model of financial capability education increase the financial attitudes/ understanding of financial concepts/ general mindset of young people?
- Does this model of financial capability education improve financial outcomes and behaviours for young people (e.g. a reduction in rent arrears; increase in savings; etc.)
- How effective is the use of incentives in encouraging young people to participate in the course (and which incentive is most compelling)?
- How effective is the use of a peer education model in financial capability education for young people?
- What are the reactions of young people to creating and sharing content related to financial capability via vlogging and other forms of social media? What are the drivers and barriers to doing so?
- How effective is social media in changing attitudes of young people towards financial capability? E.g. Does it alter the attitudes of those creating and sharing the content? How much is the content shared? What feedback do the creators receive?
The evaluation ran from January 2017 to May 2018 and involved a mix of qualitative and quantitative methods including filmed ethnography, depth interviews, discussion groups and surveys. Administrative records and social media activity were also reviewed. 32 participants completed the pre-stage outcomes survey. Response rates diminished considerably for the post and follow up stages, and the course feedback survey.
Key findings
The following intended outcomes were evidenced for some participants:
- Increased knowledge and awareness of financial concepts, products and services;
- Increased knowledge of, and ability to, save.
Unplanned outcomes for some included increased self-confidence and mental resilience and healthier eating. However, sustained behavioural and attitudinal change appeared limited.
Outcomes for peer mentors included:
- Increased knowledge and understanding of financial concepts;
- Improved behavioural and attitudinal change including capacity to increase income through new employability skills and regular saving.
Process evaluation findings include:
- Significant challenges exist in recruiting and retaining vulnerable young adults. Peer mentors can be instrumental in overcoming these;
- Retention may improve if sessions start after 11am;
- Matched funding schemes can encourage saving and engagement, but eligibility rules need to be clear;
- Maximum engagement is more likely with the use of free food and interactive games, activities and flexible content co-created by peer mentors;
- Young vulnerable adults are cautious about creating and sharing social media material about personal finances;
- Recruiting, retaining and managing peer mentors is challenging, time consuming and emotionally demanding;
- Delivering training in a partner organisation
Points to consider
-
Methodological considerations:
- Challenge of gathering a critical mass of quantitative data due to:
- Small number of participants (32);
- Low number of responses to surveys (as little as four).
- Due to drop outs and fragmented attendance it was not to possible to match data at individual level or compare results over time for a consistent group of participants;
- Outcomes surveys were too long and lacked relevance for some participants;
- Peer mentor involvement was inconsistent making the process evaluation limited; numbers reduced from four to two between Cohorts One and Two, and no peer mentors were involved for Cohort Three;
- Very limited amounts of filmed ethnography due to budget constraints.
-
Other:
- It is recommended the project be viewed as a pair of case studies, focusing first on the YMCA and second on BCoT. Evidence produced meets level two out of five on the Nesta Standards of Evidence scale, meaning findings are indicative rather than generalisable. They provide a useful resource to help develop hypotheses and approaches for future projects of this type.
Full report
CAB Basingstoke What Works project - final report
