Evaluation of My Money Now

1. Description of the programme
2. The study
3. Key findings
4. Points to consider

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

The My Money Now project, delivered by the National Youth Agency was initiated to help young people, aged 16-21 years, to improve their knowledge about financial matters and to help them make good decisions about finances in the future. To ensure that participants could benefit significantly from additional support, the project was designed to cater for young people who had started apprenticeships, had joined or been placed upon employability programmes and/or were still in full-time vocational education in school or college.

The study

When completed, the programme delivered 61 training sessions to 591 young people; 34 sessions were led by peer educators and 27 were delivered by established trainers. Apprentices participated in the programme in 17 of the centres (n=208), in the remaining centres most participants were on employability or vocational training programmes and a very small minority in self-selected ‘open’ sessions. The training was delivered across England. The evaluation had two elements:

• Impact evaluation to examine how well received the programme had been by young people; whether they had garnered the skills and knowledge intended; and, if they felt that the acquisition of knowledge and ideas may change the way they thought about and managed money in the future.
• Process evaluation, to assess whether the NYAs preferred option of using ‘peer educators’ to deliver the My Money Now programme made a tangible difference in terms of: delivery of the curriculum; experience of participants; and the likelihood of changed attitudes and behaviour by participants.

The evaluation was designed to capture qualitative and quantitative data from a range of standpoints to ensure that robust analysis could be undertaken through the triangulation of data. These included:

• Collection of quantitative data using two survey questionnaires which were completed by all participants; at the start of training, and immediately after its completion.
• Assisting in the training of peer educators (and subsequently, 4 peer associates) in reflective practice and observational techniques, and to mentor peer educators before, during the process of undertaking the programme and at its end.
• Undertaking telephone interviews with up to 60 young people three weeks after they had been engaged in the programme so that they had an opportunity to: make a retrospective appraisal of the quality and efficacy of the training.

Key findings

The results from the evaluation, shown here as percentages of participants who agreed or strongly agreed with a series of statements, indicate that the programme has been successful in meeting its objectives.

• 92% of participants agreed or strongly agreed that the training was delivered at a good pace and delivery was clear.
• 84% agreed or strongly agreed that the programme improved awareness of money matters.
• 86% felt that the programme might help them to be able to plan for the future in a more positive way.
• 80% believed that the programme had made participants more aware of the risks of fraud from online transactions and were more confident about anticipating this issue.
• 84% agreed or strongly agreed that the teamwork elements of the programme had helped them to understand issues better.
• 87% agreed that they had better technical understanding about standing orders and direct debits.
• 84% stated that they would be more careful now in the use of credit/debit cards.
• 84% agreed that they now felt it was important to discuss money issues more openly and felt that they would do this in future.

Whilst this form of evaluation does now allow the longer-term impact of an intervention with any certainty, it is possible to produce summary findings, which indicate where the greatest benefit was achieved:

• Participants on apprenticeships were much more likely to value the programme (79%) compared with those young people on employability or vocational courses (60%).
• There was no significant difference in the experience of young people who undertook the programme in a single day session, or where the programme was divided into two three-hour sessions on separate occasions.
• Gender differences in the appraisal of the programme are not visible, it was equally valued by males and females.
• Age differences affected the receptiveness of participants to the programme. 63% of 16-17 year olds rated the programme at its end as ‘really good’ compared with 80% of participants aged over 20.
• Several respondents emphasised the value of the programme by drawing attention to concrete actions they had taken as a consequence of their involvement. Many referred to their greater awareness of their spending patterns and indicated that they were, in the short period of time following the programme, making some changes to the way that they consumed goods and services.
• Others stated that they had taken steps to set up bank accounts or saving accounts so that they could manage and secure their money more effectively.

Points to consider

• Evaluation work which is undertaken while a programme is running is not able to assess the longer-term impact of an intervention with any certainty.
• Whether the programme caused the behavioural changes listed above (or whether they are long lasting) is open to question. Longer-term evaluation would be required to determine if this is the case.
• Terminology surrounding causation also needs to be handled with care. It could be the case that the programme helped participants to consolidate practices they were already engaged in or initiate action that they had been planning to take. Indeed, some of the qualitative interviews suggest that this may be the case.
• The programme delivery structure adopted was tightly defined, meaning that the same material was delivered in the same sequence to all participants irrespective of their age or experience. Qualitative data analysis suggests that such an approach may not always be appropriate – and especially so where more experienced young people require more in-depth or focused attention to specific issues.
• The period of time which lapsed between the start and end of the programme was, however, very short (in some cases, just 6 hours) and therefore cannot be used to determine, with confidence, clear indications of behavioural or attitudinal change. lack of available data on general attitudes of young people about financial issues.
• With no reliable evidence against which to benchmark findings, it is hard to discern whether the target group of this programme is in or out of line with expectations. Investing in a larger scale research study on knowledge, attitudes and behaviours which is not aligned to a specific intervention would be beneficial for future evaluation work in this field.

Full report

Evaluation of My Money Now - full report