- Description of the programme
- The study
- Key findings
- Points to consider
Description of the programme
[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]
Devised as a short accessible course the Cash Smart Credit Savvy (CSCS) programme offers an introduction to budgeting skills and financial capability as an early intervention. Originally developed by Trent Vineyard Church and Transforming Notts Together and now delivered by Just Finance Foundation (JFF), CSCS has been scaled up using funding from the Money Advice Service’s What Works Fund, to cover five pilot areas: Liverpool, Newcastle, Stoke on Trent and the Black Country, Canterbury and Plymouth. The programme aims to improve financial wellbeing by increasing people’s confidence in talking, and making informed choices about, money, through promoting a stronger savings culture and by encouraging people to share learning from the course with others. CSCS is delivered to two key audiences, Train the Trainers and General course participants. The Train the Trainers course is for those who will go on to deliver the course in their own community or organisational setting, either to groups or one to ones. The programme also runs General courses with individuals (open to all working age adults) who want to improve their own financial capability. CSCS is based on a cascading model which encourages all participants to share the knowledge and skills they attained on the course. The course covers five main topic areas including:
- Savvy spending;
- Building a budget;
- Credit and loans;
- Debt; and,
- Savings.
The study
The evaluation was carried out between January 2017 and February 2018 by the Centre for Economic and Social Research (CRESR) at Sheffield Hallam University. The evaluation addresses a number of key research questions:
- Is CSCS an effective approach to improving financial capability for working age adults through a short-term community-based training intervention?
- Does it improve individuals’ ability to make financial decisions that will benefit them (e.g. appropriate choices of financial providers and building a savings buffer)?
- Does it encourage and enable and prompt participants to share and transfer the knowledge, skills and positive behaviours with others in their community?
- A mixed methods (quantitative and qualitative) approach is adopted to gather data about the course participants’ financial capability and confidence prior to the CSCS intervention, at the end of the course itself, and at an interval of four to six weeks after completing the course. Both process and impact evaluation questions are included in an attempt to identify changes in attitudes, intentions and behaviour of those delivering and attending the course. Outcomes to be measured include whether:
- Course participants increase their confidence in making informed choices about managing money.
- Course participants increase their ability to make and manage a budget.
- Course participants increase their intentions to build a savings buffer.
- Course participants share some learning with others.
The main data collection tools included:
- Telephone interviews and a focus group with JFF (Just Finance Foundation) and CUF (Church Urban Fund) Network Partners delivering courses.
- Pre and post course surveys with Train the Trainer and General course participants completed as part of course delivery.
- Follow-up online surveys and telephone interviews with both types of course participants 4-6 weeks after the course.
- Participant observation of courses (three Train the Trainer and one General Participant course were observed).
Key findings
- CSCS is considered to be an effective approach to improving financial capability for working age adults as participants in the study reported greater confidence and increased ability in managing money and controlling spending. This suggests the aim for course participants to have increased confidence in making informed choices about managing money has been achieved.
- Course participants intended to complete a wider range of actions in order to keep track of their finances and make their money go further through things like using money saving tips and a spending diary, shopping around for food and groceries and avoiding non-essential treats items. This suggests the aim for course participants to have increased ability to make and manage a budget has been achieved.
- CSCS supports individuals to increase their ability to make financial decisions that will benefit them as it had a positive effect on participants’ level of knowledge and ability in undertaking various financial capability tasks such as working out the cost of a loan or credit purchase. A statistically significant increase in participants who plan how they spend their money very or fairly closely and who save money most months since taking part in the course also suggests CSCS played a role in helping participants to build a savings buffer. This suggests the aim for course participants to increase their intentions to build a savings buffer has been achieved.
- CSCS encourages and enables and prompts participants to share and transfer the knowledge, skills and positive behaviours with others in their community. After the course participants felt more able to give guidance to others with their finances. At follow-up four weeks after the course, 72% of participants had shared some of what they had learned about managing money, with the vast majority (95%) feeling confident doing so. This suggests the aim for course participants to share some learning with others has been achieved.
Overall, the course appears to have achieved and sustained the desired outcomes as outlined above, but follow-up results should be treated with a degree of caution; there are limitations in terms of the length of the follow-up period and sample size and ultimately, the evidence base is fairly small.
In terms of the delivery of the course:
- Participants were extremely satisfied with all aspects of the CSCS programme.
- Course materials were described as ‘excellent’ and have continued to be developed and updated during the programme. CSCS resources are accessible and adaptable to a variety of organisational and local settings and the flexibility and adaptability of materials was seen as a major factor in CSCS’s overall success.
- Much of the success of the promotion and delivery of CSCS within such a short delivery timeframe comes down to the strength of CUF Together and Just Finance networks and relationships.
- Translating Train the Trainer courses into General courses was more difficult than anticipated at the outset. The reality of delivery on the ground did not reflect the CSCS model as originally conceived. This led to a shift in expectation of how the course could be delivered with greater emphasis on utilising the materials in a variety of ways including delivering one to ones or to groups of people in community settings.
- The robust nature of materials and Facilitators Handbook guidance provide some assurance of quality control. It is recognised that guidance on what good facilitation looks like and a better indication of the skills and preparation required to deliver CSCS effectively would help improve delivery processes, maintain quality especially if the project is to be replicated.
- CSCS is being embedded into CUF networks and other services and will be scaled up as a result of a successful bid to Nesta’s Inclusive Economy Partnership. Participating in the WWF project and having findings from an external evaluation were seen as helping to give JFF credibility, both in terms of the quality of CSCS and the evidence of it having impact.
Points to consider
-
Methodological limitations:
- The scale and nature of the project and the small sample sizes involved means that it was not feasible to include a comparison group or so called counterfactual in the evaluation.
- The follow-up time period for the evaluation is also limited due to the project timeframe, medium and longer term outcomes may not become apparent within the 4-6 week follow-up period. Due to difficulties in gathering quantitative and qualitative follow-up data the evidence base on whether participants’ intentions to change translated into behaviour change is less than hoped.
- CSCS is a short duration intervention which may have made it less likely that people would participate in follow-up research. As such, it is harder to assess the extent to which CSCS achieved its medium to longer term outcomes as outlined in the project Theory of Change.
Full report
An evaluation of Cash Smart Credit Savvy - full report
