Get Ready pensions tool – evaluation and dissemination report

1. Description of the programme
2. The study
3. What are the outcomes?
4. Key findings
5. Points to consider

• Description of the programme
• The study
• What are the outcomes?
• Key findings
• Points to consider
• Full report

Description of the programme

The free, interactive Get Ready online learning tool aims to give people better information about how pensions work and to encourage them to prepare for their post-retirement future – all within a UK context. The Skills Academy had separately reported that 75% of survey respondents found understanding pensions difficult and complex, while a Royal London report (2016) found that people would have to increase the standard auto-enrolment pension payment level of 8% to allow for a “gold standard” retirement income.

The study

The study set out to assess whether the Get Ready tool improved users’ understanding of pensions in general; how far it improved their skills, knowledge and attitudes regarding pensions and helpful resources; and whether the tool spurred them to take steps to improve their long-term financial situation. The evaluation used pre and post- engagement phone surveys as well as interviews. 90 people (largely employed in the voluntary or community sector) formed a user group sample with three age-based sub-groups (18-30, 31-50, and 51 and over) who were given access to the Get Ready Tool; a control group of another 90 people with similar characteristics was not. Some members of the user group were also interviewed about their views of using the tool.

What are the outcomes?

• People sign up to their company pensions scheme (if not already)
• People check their pensions statement regularly
• People increase their monthly pensions contributions (if required)
• People understand risk management and how to scale up and down pension risk
• People set up a long-term savings and investment plan
• People keep abreast of changes to pensions policy and the economy
• People seek advice and guidance relating to pensions and their long-term finances (if required)

Key findings

It is important to note that small sample size and attrition make it hard to draw conclusions about the effect of the tool within the user group and through comparison with the control group. Figures given for the user group are based on those who used the tool and responded to the second survey.

• What works outcomes
  • People sign up to their company pensions scheme (if not already): Levels in the user group rose from 89% to 93%, and from 65% to 69 % in the control group.
  • People check their pensions statement regularly: Comparing reported behaviour at the beginning and then within the timescale of the survey, levels rose from 52% to 83% in the control group and from 77% to 80% in the control group.
  • People increase their monthly pensions contributions (if required): Levels fell from 38% to 28% among the user group, but rose from 19% to 28% among the control group.
  • People understand risk management and how to scale up and down pension risk: Levels rose from 19% to 33% in the user group and fell from 29% to 22% in the control group.
  • People set up a long-term savings and investment plan: Levels of those with at least a partial plan rose from 41% to 56% in the user group and from 45% to 50% in the control group.
  • People keep abreast of changes to pensions policy and the economy: Levels rose from 37% to 59% in the user group and fell from 40% to 38% in the control group.
  • People seek advice and guidance relating to pensions and their long-term finances (if required): Levels rose from 22% to 33% in the user group and from 45% to 50% in the control group.
• Use of and views about the tool
  • Only 10 (6%) of the 164 people given access to the tool completed all core modules. The 18-30 age group were more likely to have done so, but had not undertaken the modules called “Close to Retirement” or “At retirement”.
  • 27 participants gave feedback about using the tool. A large majority of respondents were positive about all surveyed aspects. Users aged 51 or over did not rate its effectiveness as a learning tool quite as highly as younger groups.

Points to consider

• Methodological limitations:
  • The report itself highlights a range of methodological issues, primarily the high attrition rate, overall small sample size and the risk of self-selection potentially affecting the generalisability of the findings reported.
  • The design of the study meant that it was not possible to attribute these findings to the tool. the study also overlapped with the implementation of auto-enrolment in pensions.
  • The study does not attempt to address questions of reliability and all reported figures are based on low numbers.
  • Conclusions and recommendations cannot be regarded as statistically robust.
• Relevance:
  • The frank account of the limitations of the methodology highlights difficulties in measuring outcomes and will be of particular interest to those designing evaluations of similar interventions.
• Generalisability/ transferability:
  • Both groups are drawn from a specific professional background that differ from the wider population.

Full report

Get Ready pensions tools evaluation report