Imagine if there were some dramatic developments in the production of nuclear power that made it far easier and less costly to produce. It would provide abundant, cheap energy across the world, but with one small drawback. There was a 10% chance that at some point over the next 50 years it would kill all human life.
I think most of us would assess those trade-offs and suggest we hold off.
Yet claims of existential risks arising from progress in artificial intelligence have become part of normal public debate without provoking anything resembling the reaction we might expect. This tells us a lot about how we think about risk.
In AI safety, the risk of catastrophic outcomes has been a consistent topic – often referred to, in a wonderfully anodyne way, as P(doom) – the probability of an AI-caused existential catastrophe. Recently, Evan Hubinger, a safety researcher at Anthropic, said he believed there was a greater than 10% chance of AI killing all humans within the next decade.
Some people working at the frontier of AI believe their field could have the most disastrous consequences imaginable for humanity, yet development continues at extraordinary speed. This seems odd.
I don’t want to make this post about whether these estimates are in any way reasonable, but rather about why we are prone to underplay some risks and wildly overstate others.
From a behavioural perspective, our sense of risk is heavily influenced by how it makes us feel – the emotions it provokes and the scars it leaves. This is sometimes described as the affect heuristic. We respond strongly to risks that are salient, emotive and easy to recall, and can be remarkably complacent about those that aren’t obvious and do not stir us in any way.
While some people are unconcerned about the cataclysmic risks stemming from AI because they believe the claims are preposterous, there is another issue at work: even high probabilities can fail to elicit much of an emotional response when the risk itself feels abstract and remote.
Now think about the earlier example of nuclear power. Some opposition to nuclear power – in a world where it could help reduce climate risk – is likely shaped by the feelings provoked by the concept of nuclear. The very term brings up images of bombs, devastation and reactor meltdowns. The risk seems both high and consequential because those negative associations are so readily available to us.
By contrast, for many of us AI is benign. For the moment, at least, it can seem like little more than something that helps make our everyday lives a little easier. The most affecting negative marker related to AI might be reserved for those who have watched Terminator 2 recently.
What does this mean?
The use of bland probabilities will likely have relatively little impact on how people evaluate the risk from continued AI development. This might only change if there were a significant negative occurrence related to AI – let’s say a prolonged power outage or material data breach – that affected our lives in a meaningful way. Something like that could radically change how the risk is perceived.
How we feel about AI risk is instructive of where investors often go wrong when dealing with risk. We are constantly responding to whatever is vivid and prominent, while easily losing sight of the things that are more abstract or harder to imagine.
We focus on the things that we see, feel and remember – the pain of the last bear market, severe daily price swings, the latest geopolitical flashpoint. These events either leave deep emotional scars that stay with us or make us overreact to risk in the moment. They shape our behaviour.
Not only can this lead us to make poor choices about risks that are nowhere near as consequential as they seem, but it also means that we ignore or forget those that are not obvious.
This is why a difficult quarter in equity markets can feel far more threatening than, for example, the risk that we fail to stay invested over the long term. One of these is a material risk for investors, and the other is probably not; but our tendency is to get it the wrong way round.
We deal with risk in a very human fashion. We learn from emotive experiences and focus on what is happening right in front of our eyes. We judge risks based on how they make us feel.
—
My first book has been published. The Intelligent Fund Investor explores the beliefs and behaviours that lead investors astray, and shows how we can make better decisions. You can get a copy here (UK) or here (US).