Why Can’t We Stop Making Short-Term Market Forecasts?

Our fascination with forecasts about short-term market movements is a puzzling trait.* Investors continue to make market predictions even though we are consistently wrongfooted. When other people make an incorrect forecast, we don’t disregard what they tell us next but instead say to them: “you were entirely wrong before, what do you think will happen next?”

This is not just a diverting human quirk; it is an incredibly damaging and costly behaviour. To save ourselves we need to stop listening to short-term market forecasts, and certainly not make them.

It is true that even the best investors make plenty of mistakes, but that doesn’t mean we should cut market forecasts some slack. There are four critical aspects, which mean that they should be readily ignored:

1) Most people are terrible at making market forecasts: The track record of market predictions is bleak. From tactical asset allocation to macro forecasting, history is littered with experts and amateurs alike who were undone by developments that they failed to anticipate. If everyone struggles to do it well, why do we pay attention to predictions, or believe that we will be the exception?

2) It is not reasonable to expect people to be good at making market forecasts: Creating accurate forecasts about a noisy, complex system is close to impossible. It is entirely fanciful to believe that anyone can consistently succeed in it.

3) Forecasts can have major consequences: Most market-related forecasts tend to be bold with severe consequences when mistaken. A fund manager being incorrect on a single stock is an entirely different proposition to a stark forecast about an impending market decline. When these predictions are wrong (which they usually are) the impact is typically harsh.

4) Market forecasts encourage poor behaviours: Forecasts about markets encourage the worst behaviours in investors such as short-termism and over-trading.  Also, if we believe that we can forecast markets it diminishes the perceived need for diversification.

Markets constantly remind us that attempting to forecast an inherently complex system is a fool’s errand yet, despite this, we cannot seem to resist it.     

Why do we keep making and listening to forecasts?

Our willingness to ignore the incontrovertible truth about market forecasts is driven by a range of factors from our own behavioural limitations to the structure of the asset management industry:

Overconfidence: Engaging in forecasting is a perfect example of how overconfidence can influence our behaviour. The trait encompasses three distinct forms: overplacement, overestimation and overprecision[i]:

Overplacement – it doesn’t matter if most people are abject at making market predictions, we place ourselves in the upper echelons. We are better than most people.  

Overestimation – we not only think we are superior to others, but we also significantly overstate our own skill and judgement.

Overprecision – we are far too certain that we have the right answer. This can be particularly damaging for investors who are prone to under-diversify and take injudicious decisions based on forecasting prowess.

Hindsight bias: Once something has happened, we cannot help but view it as an inevitable outcome. As everything becomes obvious after the fact, it feels like it was eminently predictable before it. Of course, it is not. What we witness is never inexorable, simply one path taken amongst many other possibilities. 

Incentives: Professional investors are incentivised to forecast market movements. Fees and careers depend on it. Giving clients the belief or sense that market, political and economic developments can be confidently predicted is an incredibly attractive sales pitch (even if it is an empty one).

Comfort: It is difficult living and investing in an uncertain world. Forecasts about the future give us comfort amidst chaos and unpredictability. Even though they are likely to be inaccurate, at the time they are made they make us feel better.

Fooled by randomness / survivorship bias:  If we have a large group of people making market forecasts then inevitably there will be some who are correct; this would be the case even if everyone was making guesses at random. That there are always some people who have called it right makes us believe that it is possible – “somebody got it right, why didn’t we?”. The mistake we make is to assume that some successful forecasts within a group at one point in time (the survivors) means that certain individuals can consistently make good predictions through time.

False Expectations: There is an expectation that investors should have an opinion about ‘markets’ and saying “we don’t know” is incredibly difficult to do (and generally not a good career move). The default is to have a view even when we know that it is likely to be unfounded.  

Despite it undeniably being a loser’s game, it is far easier to make predictions about markets than not. Part of the reason it remains such an ingrained feature of the industry is that most participants are more willing to keep playing the lucrative game, rather than take the more difficult path of realism and education. 

Why Do We Learn the Wrong Lessons?

Given that our forecasts and predictions are consistently mistaken it would seem we have ample opportunity to realise the error of our ways, but we don’t take these. Instead, we learn the wrong lesson over and over again. Whenever we are blindsided by an event or occurrence our tendency is to review what happened and ask – what did we miss? Or – how can we adapt our approach to make better forecasts in the future?

This type of reflection seems sensible but is far from it. The lesson that we should learn (but never do) is that short-term movements in financial markets (and everything related to them) are far too difficult to accurately predict, and we should stop attempting to do it. We need to make our investment decisions on the basis that we cannot forecast most things, not that we can.

But perhaps this statement is too definitive, there are better approaches to forecasting. So, should investors try to improve before abandoning the activity altogether?

Can Investors Superforecast Markets?

Philip Tetlock’s work on superforecasting has risen to prominence in recent years and given credence to the idea that although forecasting is generally done poorly, there are certain individuals that can consistently beat the average and experts. 

Although superforecasting is often framed as a group of particularly skilled and well-calibrated people; the real lessons are in how they make their forecasts and why this leads to better results.[ii] 

The key tenets of superforecasting include an ability to make probabilistic judgements (which allow for the expression of uncertainty), an awareness of base rates (appreciating the odds) and Bayesian updating (adjusting probabilities based on new information). If we are going to indulge in market forecasts, these should be central pillars of our approach. Despite the evidence that forecasting can be improved however, when it comes to markets investors should still avoid it.

Even if we take proven steps to enhance our forecasting abilities the challenge for investors remains too great. Financial market predictions are an exercise in extreme complexity. Not only are we attempting to anticipate how certain issues and events will unfold (known unknowns), but any market forecast will also be implicitly incorporating situations that have not yet occurred (unknown unknowns). To make matters worse, our market predictions are always about second order effects. Projections about how investors in aggregate (the market) will react to other developments. Superforecasting might help a little, but it is nowhere near sufficient to solve something quite so problematic and noisy.

Superforecasting extols the benefits of combining a probabilistic approach with Bayesian updating. This means when we receive new information, the odds should shift based on our assessment of its implications. Although undoubtedly a better approach to forecasting, it is hard to see how investors could realistically implement such an approach. Not only is it a herculean task to correctly judge information and ascribe probabilities, but these would somehow need to be converted into positions and trades. Critically, the inherent uncertainty in financial markets also means that frequent adjustments would be required resulting in high turnover and trading costs. It is simply not feasible.

Predictions about the future are sometimes inescapable and where investors must make them, we can seek to learn from the success of the superforecasters. Unfortunately, however, there is no magic formula to anticipating the short-term fluctuations of financial markets.

One of the problems of market forecasts is that they are so easy to make. That they can glibly roll off the tongue, belies how fiendishly complicated the activity is. When we see or hear one, it is worth taking a step back and thinking through exactly what foresight is being claimed and quite how absurd it is to believe that anyone possesses it.


* It is difficult to precisely define what short-term is, but the shorter the horizon and the greater the specificity in a forecast the more problematic it is likely to be. Forecasting that equity returns will be positive on a twenty-year view, is an entirely different proposition to predicting how the Chinese A Share market might perform over the next six months. Anything inside one year is incontrovertibly very short-term.

I have written about how and when we should make forecasts here.


[i] Moore, D. A., & Schatz, D. (2017). The three faces of overconfidence. Social and Personality Psychology Compass11(8), e12331.

[ii] https://goodjudgment.com/philip-tetlocks-10-commandments-of-superforecasting/