Evaluation of MyBnk Money Works

1. Description of the programme
2. The study
3. Key findings
4. Points to consider

• Description of the programme
• The study
• Key findings
• Points to consider
• Full report

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

Money Works is a financial education programme delivered by MyBnk across England and Wales. It aims to improve young people’s knowledge and digital skills in relation to financial matters and their confidence in dealing with money. The programme is targeted at young people broadly categorised as ‘vulnerable’, including individuals not in education, employment or training (NEET) and those identified as financially vulnerable. The expectation is that the programme will help those most in need of support to make informed choices, transition into adulthood and progress towards future learning, employment and financial independence.

The eight-hour programme is delivered by MyBnk’s expert trainers and comprises interactive activities, discussions, the sharing of personal experiences and is flexible to respond to learners’ needs. Between February 2017 and March 2018, 152 sessions were delivered to 1,339 young people through 66 host organisations (e.g. young people charities and Post-16 colleges).

The study

Evaluation and research consultancy, ERS Ltd, undertook an evaluation of Money Works in collaboration with MyBnk between January 2017 and April 2018. A mixed methods approach specifically sought to address the following research question:

What is the effectiveness of delivering digitally enhanced financial education (Money Works) to NEET young people as they transition towards financial independence?

The aim was to strengthen evidence on:

• enhanced self-reported financial capability data with a control group;
• assessing the value of including digital financial learning;
• assessing the social value generated by delivering the programme;
• utilising in-depth interviews to assess deeper impact; and
• geography i.e. effectiveness in a mix of urban and rural settings.

Utilising measures informed by the Money Advice Service (MAS) Outcomes Framework, quantitative data was collected from participants through surveys. These were undertaken at intervals pre- and post- intervention (including a control) in order to capture changes in participants’ financial capability development, attitudes, knowledge and use of digital tools. In total, 2,053 survey responses were collected from 1,243 young people.

Qualitative evidence was gathered through seven focus groups undertaken with participants at different locations across the country, following programme delivery.

In-depth interviews were conducted with representatives of the funders (Money Advice Service, and City of London), the Money Works management team, MyBnk/freelance trainers, host organisations and youth workers.

Analysis examined changes in responses across surveys collectively and individually, with consideration of control data. Social value analysis was also undertaken using the HACT model (which draws upon industry verified questions and methodology from HM Treasury’s Green Book). Alongside this, qualitative data was analysed using Nvivo data analysis software.

Key findings

The evaluation highlights key findings on the programme’s outcomes and processes:

• Money Works has been effective in increasing young people’s financial knowledge, awareness and confidence to tackle financial problems and seek advice.
• There is a heightened awareness among participants of their spending habits, with some indication of behaviour change e.g. steps towards saving and defining goals.
• There was no clear effect on levels of worry about finance and many participants remained suspicious of online resources and fearful of scams.
• There are some differences in the needs and interests of different groups. The flexibility of delivery is enabling these different groups to benefit equally from the course.
• The emphasis on digital tools is important in dispelling myths and concerns. However, its effectiveness is limited by practical issues (e.g. Wi-Fi reliability).
• Key to the effectiveness of delivery are expert trainers, engaging activities, a positive approach in informal learning environments and ensuring content and trainers are up to date.
• Money Works has been assessed to have a substantial positive social impact. It is estimated that every £1 spent on delivery has contributed to £5.57 social value.

Points to consider

• Methodological limitations:
  • In total, across all surveys, only 85 responses were from participants in rural areas and only 6 from Wales, with very few responding in the follow-up surveys. This therefore limited quantitative analysis comparing urban and rural programme delivery and differences across devolved nations.
  • There is a considerable drop off in the volume of responses to the longer-term follow-up surveys. From baseline, through endline to follow-up and follow-up 2, responses fall from 840, 686, 118 to 66. This means that it was not always appropriate to look at longer-term results, where the response to specific questions was low or results were being broken down to analyse trends for specific groups.
  • Survey results informing the evaluation are self-reported by participants and so might be subject to social desirability bias i.e. where young people might feel the need to give what they feel is the ‘right’ (desired) answer, rather than the most accurate.

Full report

Evaluation of MyBnk Money Works - full report