Changing behaviour around online transactions

1. Description of the programme
2. The study
3. What are the outcomes?
4. Key findings
5. Points to consider

Description of the programme

[This is an extract from the Executive Summary of the evaluation report. Further amendments may be made to this Summary, pending review by the Evidence Hub partner]

This project was run by Good Things Foundation and Toynbee Hall. It ran between November 2016 and December 2017. The project was delivered to struggling working-age adults. It was delivered by member organisations of the Online Centres Network.

Digital exclusion compounds the poverty premium. Individuals who do not make use of online retail and digital financial services are more subject to inflated prices, less able to compare prices, and unable to take advantage of online-only savings.

Community financial inclusion interventions are impactful but rarely focus resource on digital inclusion. Conversely, the fintech sector creates digital tools for financial inclusion without supporting the skills that are needed to use those tools. This project sought to create evidence around how to embed digital skills in a financial capability intervention.

This project has created new evidence in the sector. The Money Advice Service's Financial Capability Outcomes Framework for Adults showed that there have been no studies of the link between digital literacy and financial capability.

The study

This project was evaluated as a Randomised Control Trial, using a cluster stepped-wedge design. Data was analysed by Michael Campbell, Emeritus Professor of Medical Statistics at the University of Sheffield. This quantitative study was supplemented with semi-structured qualitative interviews of project participants.

The evaluation sought to create evidence in the community financial inclusion sector. Specifically, the evaluation sought to ascertain change in transactional behaviour. The financial capability strategy for the UK shows that financial capability in working-age life is dependent on being able to access and use financial products and services with confidence. As financial inclusion and digital skills become increasingly entangled, access to services is determined by the ability to use the internet.

The project tested the efficacy of an assisted digital transaction . Project participants were supported to conduct a live online transaction. They used their own money and chose their transaction. We measured outcomes against this intervention. This answered the following research question:

The ability to transact online can reduce the poverty premium. Are individuals receiving financial capability support better able to transact online if they are also supported to undertake a live transaction online?

The project answered this question by collecting data against the following outcomes:

• Ability to transact online independently
• Use of online financial products and information
• Ability to exercise judgement in financial decisions
• Ability to maximise income

Data was generated through a question-set using the following indicators:

• Self-assessment of financial capability
• Budgeting
• Consumer behaviour
• Financial outlook
• Self-assessment of financial wellbeing
• Digital inclusion

What are the outcomes?

• Ability to transact online independently
• Use of online financial products and information
• Ability to exercise judgement in financial decisions
• Ability to maximise income

Key findings

Assisted digital transactions improve self-rating of financial capability

• Participation in the assisted digital transaction improved scores across six key indicators. The combined score across these indicators increased by 8.9% (p-value, <0.001) from baseline through to post-intervention. This is a statistically significant effect.

Assisted digital transactions change transactional behaviour

• People who were part of the intervention were 6.5 times (p value, <0.005) more likely to transact again if they were supported to transact through the intervention. This is a statistically significantly effect.

Points to consider

Methodological Considerations

This project was a large-scale Randomised Control Trial, and the first to create evidence in the digital financial capability space. As a trial, it produced outcomes within a tightly controlled delivery context. This context was important to the success of the trial. Two factors were particularly important:

• The assisted digital transaction took place in the context of a financial capability intervention
• Participants were given open-ended support around the transaction. This wasn’t limited to the point of transaction, but extended beyond the transaction to meet any additional support needs. For example, to allay anxiety before an individual’s goods arrived; or to support an individual to return unwanted goods.

Transferability

The context of a financial inclusion transaction was important. It provided a structure which created conversation around financial capability. This created peer-learning opportunities which led to successful transactions. The course also provided ethical surety; we knew that participants had at least basic financial capability before completing a transaction.

Open ended support was guaranteed by the project delivery model. To make the assisted digital transaction transferable, other agencies need to be aware of the additional support needs that the transaction creates.

Even with these limitations, we believe that the central component of the project - the assisted digital transaction - is a highly valuable and transferable component.

Full report

Final Report and Appendix