Independent evaluation of the Community Links’ What Works project

1. Description of the programme
2. The study
3. Key findings
4. Points to consider

• Description of the programme
• The study
• Key findings
• Points to consider
• Full report

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

The Community Links (CL) ‘What Works’ project was a pilot project to evaluate the effectiveness of embedding the delivery of financial capability (FinCap) into existing services. What Works was set up to test the hypothesis that financial capability training is more effective if it is underpinned by an on-going relationship, if there is an element of co-production and if delivered as part of a holistic approach. This simultaneously supports a wider goal at CL to build ‘deep value’ relationships. CL services targeted a variety of groups with different demographics, with the aim of revealing useful additional and attitudinal information. CL delivered training to ten frontline staff and external delivery partners who in turn delivered 12 FinCap training sessions to 87 participants spread across five existing internal programmes and four external programmes in Newham and surrounding London boroughs.

The target groups for CL services include east and north London residents, particularly in Newham, who are not in employment, education or training (NEET); 16 to 24 year olds and working age adults who are unemployed and/or on low incomes, on or about to go on Universal Credit, renting in the private or social sector and/or Black and Minority Ethnic (BAME).

The study

The core research questions CL sought to answer within this project are:

1. Does embedding FinCap into existing, successful services for working age people and NEET young people, produce positive FinCap outcomes?
2. What is the best model for integrating FinCap into existing services?

CL measured the following client outcomes for all participants for pre-intervention and post-intervention, and also for 30% of the participants after a three-month period (in this report, this will be referred to as “medium term”):

• Do participants feel prepared to deal with financial challenges in the future?
• Do participants feel they have improved confidence and skills in managing their current financial situation?
• Do participants feel they have increased willingness to seek, engage with and act on financial advice and guidance?

These outcomes were measured through surveys and focus groups that were done either immediately following the training or one week later. CL also monitored the training and activity to embed the FinCap training within the existing services. The final evaluation was completed over January to April 2018, following the end of the programme.

Key findings

Embedding FinCap into existing, successful services produced positive FinCap outcomes immediately following the training and in the medium term.

• The data suggests that the training increased the majority of participants’ willingness to seek, engage with and act on financial advice and guidance. This effect lasted into the medium term.
• For some measures, such as ‘preparing for and managing life events’ the immediate very positive effect of the FinCap intervention had diminished after three months, and had returned to pre-intervention levels.
• For some indicators, such as ‘feeling that they have improved confidence and skills in managing their current financial situation, particularly ‘dealing with debt’ the immediate positive effect remained with many participants in the medium term.
• Some participants started saving to prepare for future financial shocks, and this remained higher than the pre-intervention proportion at the medium term. On the other hand, however, more participants report not thinking about the future at the medium term point than in the pre-intervention survey.

CL was unable to run comparison groups due to a period of organisational redevelopment during this project (see Section 2 for further details). CL is therefore unable to definitively identify the best model for integrating FinCap, but the process evaluation generated some helpful insights.

• Trainers’ familiarity with the material and personal use of the tools and techniques played a large part in the success of the delivery. Clients responded well to being involved in topic choice, and also to the personal experience of the trusted course leaders.
• Programme staff developed a library of modules for course leaders, who defined the delivery schedule for their programmes. The length of the sessions depended on the programme delivery schedule, client base and confidence of the frontline staff member delivering the training.
• Material outside of the participants’ life experience or mismatched with the delivery style of the programme was less engaging.
• Participants were more likely to attend the FinCap training if it was part of a wider training programme. Participants in more individualised programmes, such as Talent Match, saw the training as optional and were much less likely to attend.

Points to consider

• Methodological limitations:
• The results of this evaluation have indicated that embedding FinCap was successful; however, the following limitations suggest that further research should be done to verify the findings across other settings:
  • Fewer participant sessions took place owing to organisational re-design. The evaluation was therefore completed with 87 participants, instead of the 277 participants CL aimed for.
  • The limited number of participants was in part due to the fact that that Talent Match participants did not attend the FinCap training as it was optional. To build up a reasonably sized evidence base, CL therefore ran additional sessions with other internal projects and with external organisations. This was arguably a less focussed client set, which has led to less coherent statistical analysis than had been envisaged.
  • External delivery partners were used prior to establishing the success of the methods and material and the evaluation process took considerable time for participants with little or no English.
  • CL was not able to run a control group for this project.
  • The quality of the frontline staff training was not assessed.

Full report

Independent evaluation of the Community Links’ What Works project - full report