Citizens Advice Southwark final evaluation report - What Works Fund

1. Description of the programme
2. The study
3. Key findings
4. Points to consider

• Description of the programme
• The study
• Key findings
• Points to consider
• Full report

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

The evaluation considered two research questions: i) whether people already on the path to financial exclusion or debt are able to take on board and act on financial capability messages; and ii) whether utilising community-based financial capability money champions increases awareness and understanding of positive financial behaviours and actions. The project took place over 12 months and two key activities were evaluated: monthly welfare reform events and training of financial capability money champions. 287 people attended the monthly welfare reform events, which took place at a variety of community locations. These were coordinated and led by Citizens Advice Southwark and involved a number of partner agencies in providing financial capability and a variety of other services. 136 money champions were trained by Citizens Advice Southwark and partner agencies and these comprised both volunteers and paid frontline workers.

The study

The outcomes being measured were that people can understand their current financial situation; are aware of the impact of changes to their circumstances and are motivated to take action to plan for these changes; are motivated to take a proactive approach to making their money go further; have the confidence to use their knowledge to take action to prevent problems occurring; understand how to get the best value for money; know when to seek help, and where to get good quality independent advice and guidance.

To achieve this, the project evaluated two target groups: i) social housing tenants living in the London Borough of Southwark and ii) frontline workers providing support to tenants and residents of the London Borough of Southwark. A mixed research methodology was employed. A quantitative pre- and post- event questionnaire was completed by 120 people attending welfare reform events in the first nine months of the project. A qualitative questionnaire was completed by 48 people attending welfare reform events in the final three months of the project. Two focus groups took place comprising 19 social housing tenants who had previously attended a welfare reform event. Financial capability money champions were surveyed on the completion of their training.

Key findings

• Outcomes evaluation findings:
  • 73% of the 120 people who completed a questionnaire described their financial circumstances as unmanageable and 75% of people described themselves as either unemployed, sick or disabled, meaning a majority of those attending welfare reform events were therefore ‘people already on the path to financial exclusion or debt’.
  • One-off financial capability interventions in a welfare reform event setting can improve an individual’s mindset and feelings of wellbeing.
  • Fewer people believe they will run out of money before the end of each month, having attended an event (26% as against 45%).
  • After attending a welfare reform event fewer people believe they will need credit / store cards, overdrafts, payday loans, and loans from a company that collects payments from their home (8% against 33%), or from a friend (25% against 41%), over the next three months than they needed for the previous three months. This demonstrates that financial capability messages at welfare reform events can have a positive impact on people’s intended borrowing behaviours through reducing the amount they believe they will need to borrow.
  • The number of people who have a clear idea or some idea about how to create a household budget increased post-event from 53% to 63%; and the number of people with no idea of how to do this reduced post-event from 30% to 14%.
  • There was an increase in the proportion of people from 8% to 23% who intended to compare electricity or gas deals within the next three months post-event compared with the previous three months.
  • More people said they would save money to put against future falls in household income post-event, compared with pre-event (11% against 4%).
  • More people said they would save money to fund future care and support post-event, compared with pre-event (10% against 4%)
  • Income maximisation is an important part of the financial capability message and should be included at all events.
  • Of greatest help to those attending, in order of importance, was that the events:
    • provided the opportunity to get help with welfare benefits (including Universal Credit)
    • were local to them
    • provided help on rent arrears and council tax issues
    • provided help on bills (including debt)
    • provided good signposting to other sources of help
    • had a range of organisations present
• Process evaluation findings:
  • 136 money champions completed the training and 12 of these completed surveys. All of these believed the training would help them develop their organisation’s financial capability work.
  • With regard to specific ways in which the training has or will benefit the individual’s work most people answered with the following themes:
    • improved ability to support and advise their service user with financial capability messages, which will add social value
    • improved ability to deliver financial capability workshops to service users
    • would enable a new way of building relationships with service users
    • enhanced confidence when giving budgeting advice
• Due to challenges with the money champions’ model, including obtaining evaluation data from partner agencies and ensuring resources were in place to support and supervise large numbers of volunteers, staff consulted recommended considering other models of volunteer engagement in financial capability, such as incorporating volunteers into small teams with paid financial capability staff to better develop skills in a more closely supervised and supported environment.
• Summary:
  • Most findings from the outcomes evaluation relate to intended rather than actual outcomes
  • Evidence obtained suggests that welfare reform events helped to achieve each of the Theory of Change outcomes

Points to consider

• Methodological limitations:
  • Difficulty in tracking the journey of people on the path to financial exclusion or debt once they had attended welfare reform events, meant the findings related to intended rather than actual outcomes; and;
  • Challenges around persuading partners to collect evaluation forms on behalf of their own money champions, and the short time Citizens Advice Southwark volunteer money champions remained with the organisation, which also made it difficult to collect completed evaluation forms.
  • These limitations impact on the project’s success in answering the first research question as the evidence suggests that people are able to take on board financial capability messages, and intend to act on them, but is unable to prove that the action actually takes place subsequently (emphasis added).
• Generalisability:
  • In terms of generalisability and transferability of the evaluation approach, the approach set out in this evaluation could be replicated in other urban areas with high numbers of social tenants.

Full report

Citizens Advice Southwark final evaluation report - full report