Black Box Thinking in financial education: do money lessons really work?

What works in financial education – anything?  Or just some of it? How can we best help young people struggling with debt? Do we talk about maths, or concentrate on the touchy-feely side? Do we train the teachers or get in the experts? Should we be expecting big-budget, top-down programmes with loadsa government investment? Glossy textbooks fronted by millionaire money experts and influencers? Fancy apps that down with da kids? Or grassroots programmes focused on communities, localities – even gender? This week, we went to find out…

A few years ago someone in government had the bright idea of seeing whether policy initiatives assumed to work – like financial education in schools – actually did.

If you have read Matthew Syed’s book Black Box Thinking, you’ll be familiar with the notion that systematic learning from mistakes is only really routine in the airline industry after every single accident (hence the title of the book) and that running rigorous experiments, such as randomised control trials, is only par for the course in the medical world – not public policy.

The risks of letting untested policy initiatives run riot was clearly shown by the failure of the “Scared Straight” programme in the U.S., where young offenders committing petty crimes went into prisons to “hear” from tough jailbirds serving life sentences, ostensibly to scare them out of their criminal ways. It ended up actually increasing the chances of young criminals re-offending – as data proved time and again – but it didn’t stop the programme being rolled out across the Western World and even becoming the subject of an Oscar-winning documentary.

In an era of “fake noos, folks”, it seems more important than ever to trust the data, not a gut feeling, and only get behind those initiatives that actually work. In 2013, the What Works Network was born, aimed at fostering better decisions on a whole range of public services. And this week, the Money Advice Service formally became the eleventh member organisation, as it published findings of a two-year study of 65 of its own projects bankrolled by its own What Works Fund.

It’s the first large scale joined-up approach to understanding the best ways to improve people’s finances, and MAS reckons it has helped 37,000 people. Organisations taking part included Citizens Advice, Age UK, Shelter Scotland and The Children’s Society, and independent evaluation was done by Ipsos MORI, the University of Bristol and Ecorys.

Young Money Blog was at the British Library this week to hear the results. Here’s our verdict…

MAS chief executive Charles Counsell admitted that “millions of pounds were being spent on related financial capability interventions each year, without knowing for sure which interventions were the most effective in helping people manage their money better”.

This was hardly reassuring. Over the past eight or so years, we have been told by strident campaigners and consumer experts that financial education is a must. There have been noisy and ultimately successful endeavours to get financial education onto the curriculum across the country, firstly in Scotland and then in the rest of the UK.  Of course, it seems so rational and intuitive – giving young people information and advice about money would surely help them to manage their own finances more successfully?

And yet…Dr David Halpern, head of the government’s behavioural insights unit, had this to say:

 Education works a bit. Traditional financial education doesn’t look as though it works very well, but ‘rules of thumb’ can be attractive.

That is hardly a ringing endorsement. Does this mean that financial institutions and high-profile consumer websites have been throwing millions of pounds at financial education programmes without knowing whether they actually worked?

Some might say that government funding would have made a difference (not a penny of our state budget has gone into financial education).

Would it? Without data to show what’s actually going on, there’s a very real risk that we would have wasted an awful lot of taxpayers’ money in recent times.

It all begs the question: have “traditional” approaches to financial education – and indeed policy – dominated the debate for too long, given the very real doubts about their effectiveness?

It seems Dr Halpern’s unit – and particularly the Financial Capability Lab – is setting out to prove that hunches, guesswork and conventional thinking just won’t do. Let’s take pensions tax relief. Dr Halpern said: “Our best estimate is that every £1 of tax subsidy leads to 1p of extra savings.” So pensions tax relief is an expensive, poorly understood and ultimately ineffective way to get people saving more for retirement.


The chances of our pension system changing anytime soon to reflect that are next to zilch.

So we persist with costly and complicated failures in financial policy…which brings me back to financial education. At least MAS knew this was a problem, and went out to try and measure success (with £11m in the kitty to help them along).

The 65 projects they supported were UK-wide and covered everything “from pocket money to pensions”.

Counsell said it was a national emergency, because one in five people had less than £100 of savings, one in three couldn’t read a bank statement, and eight million people struggled with debt. He said:

12.9 million adults have no accessible savings to fall back on. We want everyone to save money regularly. 6.4 million adults are having to borrow regularly to buy food or pay bills because they are short of money.

The headline findings?  Young adults are better suited to planning for ‘life right now’, while working-age adults respond best to planning around life events – such as changes to health, employment or relationships – and the ‘teachable moments’ they create.

The most impressive findings? The charity MyBnk had the most tangible results on offer.

It estimated that £874m in social value* could be generated by giving the UK’s most vulnerable young people access to expert-led financial education. Evaluators spent a year examining the impact of ‘survival’ money management workshops on 1000 or so 16-25 year old NEETS (not in education, employment or training) and care leavers, who on average are more likely to be in poverty and have problem debt.

The study showed that each £1 spent on the programme created £5.57 in social value and the impact increased as time went on. The young people’s average debts of £300 dropped 60% to £120 – while ‘control groups’ of their peers saw their average debts grow by 50%.

So what else worked? According to the summary report…

✔️ Learning in group sessions and familiar settings – kids really benefited from setting up things like school savings clubs or individual bank accounts

✔️ Personalised advice on big life moments, like leaving school and starting uni or an apprenticeship

✔️ Digital tools like budgeting apps and online money courses BUT…

❌ Tech is NOT the entire answer. The report found many young people are under-confident and unfamiliar with “fintech”, and will require one-to-one help from actual human beings before they can start to use these tools properly and consistently.

Sarah Porretta, UK Financial Capability Director at the Money Advice Service, said:  

Designing interventions to improve people’s ability to manage money is not an easy thing. Money management touches on people’s core life skills, emotions, culture, mental health and many other aspects of their lives. From childhood to retirement, gaining financial confidence can be a lifelong pursuit. We always knew that a one-size-fits-all approach doesn’t work, but the challenge of finding and clearly defining what does has been faced by every organisation: from charities to banks, businesses, the government and everything in-between.

We’ll be reporting on this again. There is an Evidence Hub where you can find out more –

*Social value is a model created by the Housing Association Charitable Trust. Spending £156m on financial education for  the UK’s 800,000 NEETs and 10,000 annual care leavers would be outweighed nearly six times over – by reducing debts, increasing savings and alleviating financial burdens.

Leave a Reply

20 − ten =

Share on facebook
Share on pinterest
Share on twitter
Share on linkedin
Share on email
Share on whatsapp


Related Posts