Evaluation of the School of Hard Knocks (SOHK) financial capability project

1. Description of the programme
2. The study
3. Key findings
4. Points to consider

• Description of the programme
• The study
• Key findings
• Points to consider
• Full report

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

With funding and support from the Money Advice Service (MAS), the main research question for this study was to investigate whether prior improvement in non-cognitive skills makes FinCap education more effective for the vulnerable young people the SOHK works with. The aim was to deliver FinCap modules to young people on the project and compare their improvements with other similar cohorts to see whether the combination of FinCap and their sporting intervention produces greater results than FinCap on its own. The SOHK project has three sets of activities: (a) embedded FinCap education including the provision of FinCap sessions to a targeted number of young people engaged in FinCap and sports, (b) the creation of a team, provision of rugby coaching and recognition of achievements and (c) the provision of mentoring and goal setting involving rugby coaching sessions, team cohesion games and one-to-one and goal setting sessions.

Training was provided in two out of the three study cohorts comprised of year eight and nine groups aged 13 and 14 years old in three schools. The cohorts included (a) a SOHK Group receiving sports and FinCap interventions (b) a potential SOHK Group of students from those eligible for a sports intervention receiving FinCap only, drawn from the same schools, and (c) a Comparison Group not receiving sports or FinCap. The SOHK Group and Potential Group were similar in that these pupils are typically at risk of permanent exclusion and many have difficulties controlling aggression. They typically have challenging behaviour, exhibiting occasional use of violence and have issues with authority. There are participants with learning difficulties and mental health disorders and reports of participants engaging in risky behaviour. Some of the Potential SOHK Group from two schools became SOHK participants during the course of the study.

The study

The evaluation sought to understand the effectiveness of SOHK interventions (a process review) and the outcomes for young people (an impact assessment). The former assessed the partnership and engagement process for the project and the latter the impact of FinCap on young people as an integral part of SOHK sporting and mentoring interventions.

The three key project outcomes sought were as follows:

• Improved financial capability.
• Increased self-esteem, self-control and self-efficacy.
• Improved ability to build social bonds.

Key findings

• FinCap headlines were as follows:
  • A generally high awareness of the importance of saving money amongst all groups, although there was a difference between awareness and practice, that is, less young people are actually saving.
  • Whilst there were general improvements to many of the positive aspects of saving, there may be more awareness work to do to encourage saving over spending.
  • There has been an improvement in the number of young people from all groups who could name three ways to pay for things.
  • There is generally a good understanding of price comparison as a means of getting a good deal amongst all groups and there have been some improvements over time in other areas such as buying non-branded goods.
  • The overall picture on keeping track of finances is quite encouraging amongst all groups and there have been some improvements in terms of money planning.
  • Whilst young people’s understanding of online safety varied between groups and various aspects of security, they had a relatively good understanding overall.
• The SOHK model appears to have been more effective at developing ‘softer’ skills with the generally the greatest and most consistent improvements experienced among the SOHK group. Looking at the findings for self-esteem, self-control and self-efficacy the headlines are as follows:
  • SOHK participants showed the most obvious reported improvement in terms of handling their feelings. Approximately three quarters of all pupils were happy to do what was asked of them though many respondents claimed they got easily distracted.
  • All groups appear to feel relatively positive about themselves especially the Potential SOHK and SOHK Groups. Note the fact that some of the potential SOHK Group joined the SOHK cohort halfway through, may have influenced this group’s positive feelings about themselves.
  • All groups are also reasonably sure that their parents or guardians feel positive about them, but have much less teacher confidence, especially the SOHK Group. In terms doing what they set their mind to, all groups are comfortable setting goals. In terms of achieving goals the SOHK Group scored highest and improved. This is an area where the rugby training is likely to have had a noticeable effect (see later).
  • Nearly all SOHK participants now believe they are in control of creating a positive future for themselves which is encouraging from a school’s perspective as well as the SOHK project. It means that those at risk of permanent exclusion can still be encouraged to be optimistic about the future. In terms of social bonds, the SOHK and the Potential SOHK scored higher in terms of maintaining friendships. All groups have declined in terms of ‘having friendships with people who are a good influence’.

Points to consider

• The project had several elements and anticipated outcomes. This combined with the fact there were different schools and cohorts involved made this a somewhat multi-faceted and complex evaluation. For this reason, there are some points to note when interpreting the study findings.
• The FinCap element was delivered as a one-off session as part of a larger intervention aimed at young people that included sports and one-to-one mentoring. This made identifying a definitive relationship between cause and effect in financial capability challenging. The evaluation findings need to be viewed in this context and not in isolation. The findings should be understood within the local context within which they were delivered too. The training was offered to different cohorts within schools with very different circumstances within nine cohorts of ranging between eight and 22 individuals. The limited sample sizes from which to draw conclusions should be noted and these are provided throughout the report and within Annex Three.
• The post survey was delivered a few months after the course, so the evaluation can only really assess the short-term effects of the project. To complicate matters some of the Potential SOHK cohort from two schools subsequently moved into the SOHK Groups so the improvements in performance for some of this group may reflect the additional support they have been receiving.
• The approach did have some limitations. FinCap was delivered as part of a comprehensive package of support measures. The engagement and co-operation of schools has taken some time and schools performance data received was not comprehensive enough to make a clear assessment of distance travelled in terms of behaviour and achievement. There were some lessons in relation to working with young people at risk of exclusion including the use of clear language, the chosen format for the evaluation workshops and in terms of preparing young people for the evaluation sessions. Coaches did help a handful of individuals to complete some questions and joined in the discussions, so a small element of bias could not be discounted. There was some attrition between the two workshop periods with absentees and exclusions, an inevitable consequence of working with the target groups. The data for those not present at the second workshop was eliminated from the ‘before’ records for the SOHK and Potential SOHK Group to ensure we had matched ‘before’ and ‘after’ cohorts.
• SOHK Wales were positive about the intervention’s potential and the training has been adapted and rolled out across Wales for both young people and adults. There were several suggestions made on the potential wider application of the SOHK project including family-based interventions, new elements such as health, alternative provision to schools and applications within other sectors.

Full report

Evaluation of the School of Hard Knocks (SOHK) financial capability project - full report