- Description of the programme
- The study
- Key findings
- Points to consider
Description of the programme
[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]
Focused on supporting vulnerable adults resident within both the urban and rural areas of Tyne & Wear and Northumberland, the aim of the BEAT 2 project was to break the cycle of financial instability and crisis, enabling financially excluded adults to gain financial independence and build resilience to future financial crisis. Delivery was undertaken by a small team of specialist advisors employed by Changing lives and Oasis Aquila Housing. They provided one-to-one financial advice/support on an outreach basis from a wide range of community settings e.g. recovery centres, food banks, or homelessness centres. Project delivery followed a two-stage process of short-term crisis support, followed by long-term development of financial skills.
By the end of March 2018, BEAT 2 had supported 496 adults age 18+ that were either homeless, or exhibited at least one of the characteristics placing them at risk of homelessness (e.g. 20% with rent arrears, 32% with a reduction in income, 56% with substance abuse issues, 66% mental ill health). All clients were within the ‘struggling’ population segment, as defined by MAS. The main support needs of clients were: help to apply for benefits; help to appeal a benefit decision or sanction; support to increase income; debt advice; and help to improve money management skills.
BEAT 2 also provided financial capability training to 134 frontline staff/volunteers working in other services supporting vulnerable adults (e.g. recovery centres, health organisations, employability services). The training equipped staff/volunteers with the skills and knowledge needed to provide basic financial capability support to their clients, and to refer clients on to BEAT Team Advisors for more specialist support.
The study
The overall research question to be answered by the evaluation was: ‘To what extent does the BEAT 2 project help to improve the financial capability of clients most at risk of facing financial exclusion, within both urban and rural settings across Tyne & Wear and Northumberland?’ The evaluation focused primarily on gathering outcomes evidence from project clients, and from frontline staff/volunteers undertaking training, using pre and post-intervention surveys. The research took place between March 2017 and March 2018.
-
Clients: 444 clients (90%) completed a baseline survey on first registering with the project, of which 329 received two or more interventions. 181 of these clients (55%) completed follow-up surveys between one and nine months after receiving support.
-
Staff/volunteers: 134 staff/volunteers completed a baseline survey prior to undertaking training, of which 61 (46%) completed a follow-up survey between one and six months after completing training.
The surveys captured data to measure change against the following outcomes for clients (NB. these project level outcomes were mapped against four MAS outcomes: financial wellbeing; behaviours; ability; mindset).
-
Outcome 1: Clients experience a reduction in benefit sanctions, rent arrears and debt.
-
Outcome 2: Clients have an improved understanding and awareness of their own financial situation, in particular dealing positively with financial difficulties.
-
Outcome 3: Clients feel less anxious about their financial situation.
-
Outcome 4: Clients can keep track of income and spending, and create and stick to a viable budget.
-
Outcome 5: Clients have improved knowledge of financial services and products.
And for staff and volunteers:
-
Outcome 6: Staff and volunteers have a better understanding of the importance of financial capability and its impact on clients.
-
Outcome 7: Staff and volunteers have improved financial capability skills and knowledge.
-
Outcome 8: Staff and volunteers have improved ability to deliver financial capability support to clients.
Key findings
Outcomes evaluation:
-
Financial wellbeing (Ability to live adequately within means) - The project supported clients to resolve benefit problems, restore or increase benefits income, and reduce rent arrears and debt. This improved the ability of clients to live adequately within their means e.g. 51% more clients were resolving benefit problems at the follow-up stage, compared to the baseline.
-
Financial wellbeing (Finances and emotional/mental wellbeing) - The project enabled clients to improve their financial and emotional wellbeing, and reduce their anxiety levels. 37% fewer clients worry a great deal about their financial situation after support.
-
FinCap Behaviours (Managing money well day-to-day) - The project supported clients to improve how they manage their money day-to-day, including taking action to reduce rent arrears and debt. As a result, clients feel they are managing better financially. 75% of clients at the baseline said they were finding it very difficult or quite difficult to manage financially, but by the follow-up stage this had fallen to 43%.
-
FinCap Behaviours (Dealing with financial difficulties) - The project supported clients to take positive steps to deal with financial difficulties, by increasing their income. 28% of clients had applied for new/additional benefits during the past six months, but by the follow-up stage this had risen to 64%.
-
Mindset (Financial attitudes to managing money) - The project improved the mindset of clients, improving their attitudes about managing money and helping them feel more in control financially. At the baseline stage, 43% of clients tended to agree, or agreed strongly, that nothing they do will make much difference to their financial situation, but by the follow-up stage this had fallen to 24% of clients.
-
Mindset (Financial confidence) - The project helped clients become more confident in their ability to manage their finances. 31% of clients reported a significant improvement in their approach to keeping track of their finances (income and spending), between the baseline stage and the follow-up.
-
Ability (Financial knowledge and understanding) - The project helped clients improve their financial knowledge, skills and understanding e.g. % of clients reporting they have a clear idea or some idea about how to create a household budget increased from 56% at the baseline, to 70% by the follow-up stage.
-
Outcomes for staff/volunteers - The training had a positive impact on improving the financial capability, skills and knowledge of staff/volunteers, across a range of topics matched to the needs of clients including benefits, rent arrears, debt, and money management. The percentage of staff/volunteers reporting good or reasonable skills and knowledge in these areas increased by between 10% and 20% from the baseline to the follow-up.
-
Urban and rural impacts - There was no evidence from the evaluation that the financial capability outcomes delivered by the project have differed between clients in urban areas and those in rural settings.
Process evaluation:
_ What worked? _
-
Face-to-face advice provided by skilled and experienced advisors - By building trusting relationships over a period of several months, advisors were able to help clients address a financial crisis (e.g. benefit reduction, debt), then continue to work with them to build financial capability and skills).
-
Working alongside specialists to achieve sustainable change - Financial capability support for this client group should be delivered in conjunction with support from other specialists, such as addiction recovery centres, or mental health counselling. This allows clients to improve their financial capability, at the same time as addressing the underlying causes of their financial situation.
-
Importance of outreach delivery model - Providing financial advice to vulnerable adults where they are (e.g. recovery centres, supported accommodation, food banks) is the key to engaging and retaining vulnerable clients in financial capability support.
-
Using benefits advice as a ‘hook’ to engage clients - Most clients were attracted to the project to get help to resolve a benefits problem. By offering support to resolve benefits problems (and related issues, such as debt and housing), clients can be engaged in a longer-term relationship with an advisor who can then help to develop their underlying financial capability and skills over time.
-
Important role of frontline staff and volunteers - Working in partnership with frontline staff and volunteers in other projects and services (e.g. addiction recovery centres, mental health counselling) is an effective way to identify clients in need, encourage referrals, and allow frontline staff and volunteers to deal with more straightforward financial problems. Financial capability training helps give frontline staff/volunteers the skills and knowledge needed to provide basic support.
Other key findings:
-
Complex needs of client group and significant time required to achieve real change - Adults who are homeless, or at risk of homelessness, face a range of complex issues (e.g. physical and mental health problems, addictions and issues with personal safety), as well as financial exclusion. These underlying problems must be addressed, in parallel with building financial capability skills, if lasting change is to be achieved. This type of change takes longer than the 13 months available for this pilot project.
Points to consider
-
Methodological limitations:
- The mixed methods approach adopted, including outcome, process, and economic evaluation, is considered to have been appropriate and effective. However, there were some limitations.
- The 13 month period available for pilot delivery has been too short for some clients to achieve meaningful change in financial capability, and for this to be measured by the evaluation.
- The outcomes questionnaires, based on the MAS outcomes framework, did not always work well for this group of vulnerable adults. Many clients found the questions confusing and the survey process too long, resulting in some clients becoming stressed and anxious.
- While the evaluation has been able to demonstrate positive changes in the skills and understanding of staff/volunteers undertaking financial capability training, the sample size upon which the analysis is based (61 matching pre and post-intervention survey responses) was smaller than originally planned.
- In line with the MAS outcomes framework, all of the survey data gathered to measure progress against project outcomes is based on self-reporting by clients at the pre and post-intervention stages.
- As is the case with all research based on self-reporting, there is a risk that some respondents may overstate the impacts of the support to show themselves in a good light, or they may understate the true impacts, for example where they lack self-confidence and tend to underestimate their achievements. This should be taken into account when considering the evidence of project outcomes.
- It is important to highlight that the resources available for the evaluation research were insufficient to allow for significance testing to be undertaken. While the outcomes evaluation indicates that clients made positive progress against all of the outcomes defined in the Theory of Change, it is not possible to claim that the outcomes achieved are statistically significant because no significance testing has been undertaken. It is important to consider the evidence of project outcomes in this light.
Full report
BEAT 2 Project - full report
