- Description of the programme
- The study
- Key findings
- Points to consider
Description of the programme
[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]
This project was undertaken by academics (Dr Declan French, Professor Donal McKillop and Dr Elaine Stewart) from Queen’s University Belfast (Management School). The objective of the project was to determine whether mobile apps designed to enhance financially capable behaviours can help people from disadvantaged communities make better informed decisions about how to tackle debt, manage money day-to-day and prepare for life ahead. Four apps, packaged together under the title ‘Money Matters’, were provided to working age members (16-65 years) of Derry Credit Union (Northern Ireland). The project was undertaken between February 2017 and April 2018.
The study
The evaluation approach was a randomised control trial (RCT). An RCT is a rigorous approach to determine cause-effect relations between two groups by randomly assigning participants to a control group (i.e. those who do not receive the intervention) and a treatment group (i.e. those who receive the intervention). An RCT was chosen as it allows for rigorous randomisation by ensuring that allocation to the comparison groups is unbiased and allows for statistical analysis to efficiently investigate for causality. An initial baseline survey was carried out from June to August 2017 and resulted in 500 completed survey returns. Of the 500 completing the survey, 260 were allocated to the control group and 240 to the treatment group. The latter were provided with the ‘Money Matters’ apps (a loan interest comparison app, an expenditure comparison app, a cash calendar app, and a debt management app). A follow up survey to determine if changes in financial capability had occurred was carried out between February and April 2018. Therefore, in this study any behavioural changes in financial capability is essentially being assessed over a 9-month period.
Key findings
- For the treatment group (i.e. all those who received the apps regardless of whether they chose to use or not use the apps) a statistically significant improvement was identified in ‘personal and financial well-being and confidence’. Receiving the mobile apps increased the probability of being very confident when shown information about financial products such as a loan, credit card or store card.
- For those assigned to the treatment group, statistically significant improvements were identified in measures designed to capture aspects of ‘attitudes to money’.
- Access to the mobile apps encouraged users to think ahead and plan for future life events such as retirement. A more positive attitude was also identified towards the use of technology for day to day financial decision making.
- Those provided with the apps were also identified at the end of the trial as having more favourable attitudes to borrowing. At first glance, this may seem counterintuitive. However, we should not be surprised if mobile apps designed to aid borrowing comparisons and which are shown to improve confidence about loans also reduce antipathy towards borrowing.
- No significant results were found for measures capturing whether money is being managed well on a day-to-day _basis including budgeting and tracking income. _However, receiving the mobile apps generally led to more positive results on these measures and made users aware that their current approach to keeping track of income and expenditures was not working.
- No significant results were found for those measures designed to capture levels of over indebtedness, _or for indicators measuring _keeping up with bills and commitments, _or for measures capturing _product holding and credit use.
- App usage was assessed through usage statistics provided by the app developers as well as through additional questions in the follow up survey. Usage statistics identified the cash calendar as the most used app followed by the expenditure comparison app. Over the course of the RCT the apps were most frequently used at the outset and when two reinforcement exercises were undertaken (a money skills workshop and a money skills competition). The apps were more frequently used by those with higher levels of education. Perceptions around the quality of the apps were encouraging as 58% identified their ease of use as being the ‘most liked’ aspect and 51% stating that they would recommend the apps to a friend. Only 4% indicated a dislike of the apps.
- Infrequent users stated that they would use the mobile apps more often if the information provided by the apps was of greater relevance and if they had greater confidence in being able to understand the information retrieved from the apps.
- 25% of those provided with the apps believed that as a result of using the apps over the period, they now think more about how money advice and guidance could help them, understand the importance of timing in repayments and interest charges and have a greater awareness of their future financial needs and the importance of setting financial goals beyond the day-to-day.
Points to consider
- Incentives were provided to both treatment and control groups to participate in the study. The study sample is perhaps in greater financial need than the population of credit union members.
