Manchester United, Poor Decision Making and the Problem of Small Sample Sizes

Manchester United are one of the most successful clubs in English football (soccer) history[i] and also the second most valuable sports teams in the world[ii].  It is therefore somewhat puzzling that they made a clear and unambiguous error of judgement in their recent decision to appoint former player Ole Gunnar Solskjaer as their permanent manager*.  This is not a slight on Solskjaer nor his abilities as a manager (upon which I have no strong view), but rather a critique of how such an esteemed and well-resourced organisation could apparently fall victim to statistical naivety and a number of behavioural biases.

Following the historically successful tenure of Alex Ferguson, Manchester United went into a period of relative decline moving through a succession of managers culminating in the sacking of Jose Mourinho midway through an underwhelming 2018/2019 season.  In what appeared to be a prudent course of action; rather than make a rash, reactionary managerial appointment the board decided to appoint a temporary manager and then make a decision on a permanent appointment at the end of the season – thus affording them more time and a likely greater opportunity set of higher calibre managers.

The temporary manager appointed was Ole Gunnar Solskjaer.  Whilst Solskjaer had little managerial pedigree outside of a disappointing spell in the English Premier League and a longer tenure in the minor Norwegian league; he was a cult hero as a Manchester United player – known for his goal scoring ability as a substitute and steeped in the history of one of the club’s most successful periods. As a short-term ameliorative it seemed to be a sensible choice.  Few expected his tenure to run past the end of the season.

The form of the team improved immediately following Solskjaer’s appointment as Manchester United went on to win 14 of his first 19 games in charge.  Despite his experience seeming entirely unsuited to a job of such magnitude, this exceptional run led to an increasing clamour for him to be offered the managerial job on a permanent basis.  With at least 10 games remaining until the end of the season, Manchester United decided they had seen enough and offered him a three year contract.

At face value the move was understandable – Solskjaer’s impact had transformed the fortunes of the team and this had provided compelling evidence of his skill as a manager – but was this really the case?  Were 19 games enough to make such a judgement? And what was the benefit of making a decision before the end of the season?

The underlying inference of Manchester United’s decision to make Solskjaer’s appointment permanent was that there was a causal link between Solskjaer’s skill as a manager and the upturn in the team’s fortunes.  Although this is the instinctive conclusion to draw it is not necessarily the correct one – there are a range of other confounding factors that could have contributed to improved form:

Simple mean reversion: It is likely that a manager will be sacked after an unusually poor run of form – when their recent results are some way below average.  The supposed influence of Solskjaer may just have been reversion to the mean.

A run of easier games: Poor form and a managerial change can often coincide with a run of more difficult games against tougher opponents with a resurgence arriving when the schedule of matches becomes easier.  Mistaking skill for a spell of less demanding games is a major risk if you are working with a small sample.

Players trying harder: A short-term upturn in performance may simply be a reaction from players who were disaffected under the struggling former manager.  They may expend more effort and attempt to impress the new manager – this is a temporary phenomenon and no reflection of managerial skill.

Sheer Luck:  Although hard to accept, when dealing with small samples outcomes may simply be a result of luck and randomness.

The most grievous element of Manchester United’s decision making was their explicit choice to reject the opportunity to observe a larger sample of evidence (more matches), particularly given Solskjaer’s lack of experience at the highest levels of management.  Rather than wait until the end of season they rushed to appoint Solskjaer despite their being at least 10 games remaining, and therefore left themselves beholden to the vagaries of a very small sample of evidence.  A c.50% increase in sample size would not have made the evidence infallible, but the acquisition or more relevant information was both beneficial and costless for Manchester United in this situation – there was no downside to waiting and learning.

The only possible reason for rushed permanent appointment would have been if there was a risk of a Solskjaer accepting a job at another club so they needed to secure his services – however, given his lack of pedigree and affinity with Manchester United this was never a genuine threat.  The club therefore opted to make a decision on a small biased sample despite having a free option to materially increase the amount of available evidence.  It would be unfair to suggest that Manchester United were alone in this thinking; there was seemingly a growing sense from fans and interested observers that Solskjaer’s strong early run had seem him rightfully earn the position on a permanent basis.

Dealing with small samples of evidence allows our ingrained behavioural biases to run amok as we seek to draw meaning from incredibly noisy data.  Not only were Manchester United suffering from a bout of outcome bias and myopia, they also succumbed to the irresistible lure of a compelling narrative.  Solskjaer’s playing career and indeed his words as manager harked back to the glory days of the club that had been somewhat lost in recent years.  The combination of this nostalgic yarn and a short period of strong results simply proved too intoxicating.

With somewhat grim inevitability the form of the team has deteriorated since the permanent appointment was confirmed and it is now doubtful whether the club would have made the same decision had they been in possession of this additional information.  The subsequent performance of Manchester United under the stewardship of Solskjaer is however something of an irrelevance when we come to assess the quality of the decision to make him the manager on a permanent basis.  Even if Solskjaer goes on to achieve great success at the club, it does not change the fact that Manchester United willingly and needlessly made a poor decision based on an inadequate sample of evidence.

I have commented on this blog previously that there is no better place to observe our behavioural foibles in full bloom than financial markets, but I should add sport alongside finance –  it is also provides wonderfully fertile ground for poor decision making and biased judgments from both participants and observers.  Of course, many of the traits exhibited by Manchester United in their managerial appointment will be painfully familiar to investors – the short-term thinking, the confusion of randomness and skill, the danger of small sample sizes and the lure of a good story.  Seeing one of the largest and most prominent sports teams in the world fall victim to such common decision making problems is both worrying and comforting.

* A manager in English football is analogous to a coach in US sports.





Why Are Stories so Important to Investors?

Narratives are human constructs that are mixtures of fact and emotion and human interest and other extraneous detail that form an impression on the human mind.” Robert Shiller [i]

Stories have long been fundamental to the human experience – they are vivid, coherent and memorable – and are crucial to how we interact with the world.  Although the term ‘story’ conjures images of fairy tales and myths, there is little that occurs in our lives around which we do not attempt to weave a narrative.  Stories cater to our need for sense-making [ii] and our desire to observe causality.  In one form or another, they underpin most human decisions.

One particular model of decision-making – the explanation-based theory – emphasises that in certain conditions, individuals start their decision process by developing a causal model to explain the available evidence [iii] – to rationalize it, in other words, and put otherwise potentially abstract data into context.  In this model, the story created informs the ultimate decision as much as the standalone evidence.  Nancy Pennington and Reid Hastie, the academics who developed the theory, argue that trial jurors are more likely to be persuaded by evidence if it is presented in the order of a logical story [iv] than if the same evidence is shown in random order.  What’s more, the story each individual develops around the evidence will be unique and dependent on their own characteristics, beliefs and experience.

The juror example is useful when considering the role of narratives in investment decision making.  Pennington and Hastie note that their explanation-based theory is particularly applicable when decisions are large and complex, as investment decisions often tend to be.  In fact, investment decision-making is a domain in which stories assume a particular importance in driving, informing and justifying conclusions.  There are two key reasons for this:

– Complexity: Narratives aid our comprehension, or, at least, can lead us to believe that we understand something.  As David Tuckett notes about financial markets: “there are few human institutions more alien to our understanding.” [v]  There is intricacy and complication even in the simplest of investments, and if we start to consider the innovation and product proliferation that have come to define the industry, many investments can be fairly considered unfathomable.  Stories are therefore important in allowing investors to both simplify and justify decisions.  This includes the decision to buy inherently complex investment products, as anyone engaged in investment marketing can attest.

– Uncertainty: Financial markets suffer from profound randomness and unpredictability.  Our surest means of coping with the discomfort is to manufacture meaning by forging a relationship between the data and an explanation – a story, in other words – of why the data is what it is and how it got that way– what Taleb refers to as the ‘narrative fallacy.’ [vi] Of course, the problem with this phenomenon is that we are fabricating our understanding – stories make us feel as if we can clearly perceive or even predict a chain of events, when that is far from the case.  Narratives can be effective and efficient in stable, ordered environments where there is a consistently observable cause and effect, but financial markets are anything but this.  Our focus on narratives leads us to hugely understate randomness and chance, and is a major driver of some of our most damaging behaviours.

As discussed above, the perception still holds that news and narratives drive price fluctuations, when the causation is typically the reverse.  We struggle to comprehend that asset prices often move in a random or unpredictable fashion, therefore we must attach some explanation to it.  The more regularly an asset is priced, the more narratives that will be linked to its behaviour – for frequently traded, market-priced assets, most investors have little hope of escaping the swarm of narratives.

There is a close association between how we use narratives and the notion of confirmation bias.  Typically, we think of this phenomenon as our desire to seek out evidence that confirms our pre-existing view and avoid that which might contradict it; however, there is more to it than this. It is not simply that we ignore or certain types of evidence, it is that we can shape a narrative around a particular piece of evidence so it concurs with our own preconceived idea – in this way, two individuals with polar opposite views can use the same evidence to support their conflicting positions.  This type of behaviour is particularly prevalent in financial markets because the level of uncertainty means that there are always different and (seemingly) valid explanations.  As noted in the juror example, narratives are not simply a means of presenting evidence in a coherent manner – they can bestow a particular meaning to the evidence.

As well as creating our own narratives, we are also captivated by the stories of others. Stories are engaging, compelling and persuasive; it is far more interesting to hear yarns of how a fund manager met with the CEO and toured the new factory, rather than discuss probabilities and uncertainty, which are almost always more relevant considerations in an investment decision.  Storytelling in a sales context is particularly effective if the teller holds an information advantage over the listener (as is often the case in an investment context) – as the recipient often has no robust means of assessing the credibility of the story, and therefore is left making judgements based on how convincing or well-delivered it is.

If we return to the opening quotation from Robert Shiller’s work on ‘Narrative Economics,’ he notes that narratives are “mixtures of facts, and human emotion, and human interest and other extraneous detail.”  To that we can add that the types of narrative we create for ourselves will be based on factors such as our character, experience and incentives – not necessarily just the objective facts before us.  The problem is that for many compelling investment stories, facts are often only a small dose in the overall mix of contributory elements.

Most of us won’t make an investment decision without it being supported by some form of story, and that’s understandable; stories are effective and can be very valuable.  However, we must also take the time to consider the credibility of the narrative, the data that underpins it and our own role in shaping it.

[i] Shiller, R. J. (2017). Narrative economics. American Economic Review, 107(4), 967-1004.

[ii] Chater, N., & Loewenstein, G. (2016). The under-appreciated drive for sense-making. Journal of Economic Behavior & Organization126, 137-154.

[iii] Hastie, R., & Pennington, N. (2000). 13 Explanation-Based Decision Making. Judgment and decision making: An interdisciplinary reader, 212.

[iv] Pennington, N., & Hastie, R. (1988). Explanation-based decision making: Effects of memory structure on judgment. Journal of Experimental Psychology: Learning, Memory, and Cognition14(3), 521

[v] Tuckett, D. (2011). Minding the markets: An emotional finance view of financial instability. Springer.

[vi] Taleb, N. (2005). Fooled by randomness: The hidden role of chance in life and in the markets (Vol. 1). Random House Incorporated.

* This is a marginally altered version of a post I wrote for the Essentia Analytics blog, which you can find here.  I will be appearing on a panel at their 2019 Behavioural Alpha Conference (London) on May 15.

Ten Behavioural Advantages Amateur Investors Hold Over Professionals

When discussing the behavioural foibles that impact our investment decision making, it is often stressed that these issues also affect professional investors – seemingly in an effort to allay any notion that expertise insulates them from such issues.  Whilst it is certainly true to state that professionals are not invulnerable, this does not go far enough.  If we think about some of the main challenges encountered by investors around issues such as time horizons, over-trading, overconfidence, misaligned incentives and benchmark obsession, these problems are often exacerbated when investing in a professional context.  Amateur investors* therefore have a number of behavioural advantages:

  • There is no need to check your portfolio on a daily basis: Access and control are optically wonderful developments for investors, but almost certainly come with significant behavioural costs.  They allow us to react to the random, noisy movements of our investments and exhibit our most destructive behavioural tendencies.  However, amateur investors don’t have to engage as frequently as professionals – they can make some sensible long-term decisions at the outset and review their portfolio sparingly; avoiding the emotionally exacting experience of living through your long-term investments on a day to day basis.  Professional investors possess no such edge – they are compelled to constantly monitor their portfolios and must deal with the behavioural issues that stem from this.
  • You can make decisions consistent with your own time horizon: Most people have a reasonable idea of the time horizon for their investments – saving for an expected thirty years to retirement, for example.  Whilst all types of investors struggle to make long-term decisions, this can be particularly pointed for professionals.  In addition to the notion of a reasonable time horizon being somewhat vague, professional investors often have to work to multiple (often implicit) time horizons some of which might seem contrary to sensible investment decision making.  For example: three months for performance reviews, annual for performance fee targets, or three years as the typical minimum assessment period for professional fund investors / consultants.  These types of pressures can foster an ingrained myopia for any professional investor.  Of course, all investors suffer from short-termism, but it is easier for amateur investors to avoid it.
  • Your incentives are perfectly aligned:  Allied closely with the aforementioned time horizon concept, incentives are also problematic.  For amateur investors, the incentive for investing is clear – to meet their specified long-term objective.  For professionals, the incentives can be complicated and often contradict the goals of the strategy that they are responsible for – such as excess rewards for generating abnormal short-term returns (often driven by performance fee structure) or raising assets to levels which maximise revenues, but impair return potential.
  • You can do nothing:  Professional investors have an activity problem – there is too much of it.  There are two major pressures that cause needless and often destructive overtrading for professionals: i) They have constant exposure to incessant newsflow and random market fluctuations, which often compels them to act, ii) It is difficult to justify fees and show expertise by doing less, so the tendency is to do more.  Although not immune to these issues; amateur investors have the wonderful, liberating ability to make sensible, long-term investment decisions and then leave well alone.
  • There is no need to chase performance:  Professional investors are under consistent and significant performance pressure with failure to deliver over short time horizons creating pointed career risk.  This leads to decision making which can often be dominated by the short-term performance imperative at the expense of philosophy and process considerations. It is often viewed as unacceptable for a professional to state that they are doing the same thing after three years of poor performance – even if such a period says remarkably little about the long-term validity of a particular approach.

  • There is no need to window dress your portfolio:  Why is it that professional fund investors are prone to sell their active fund positions after three years of poor performance and replace it with a strategy with a stellar three year track record, despite evidence that, on average, this is poor behaviour?  At least part of the reason is that their portfolios look better (and are more saleable) if they hold long-term winners, even if they have not been holding them for much of the period where they have been successful. Conversely, it is difficult for them to own notable laggard strategies from a perception perspective, even if there has been no fundamental change in the view.
  • You don’t need to make bold forecasts on the economy or market:  If you work as a professional investor there is an implicit (sometimes explicit) assumption that you should have strong views on the near term direction of capital markets and the global economy. Given that this is a skillset few people possess this is a highly problematic situation which results in virtually every professional investor opining and many trading on such views, despite there being scant evidence that they have any particular capability in this area. For professional investors this situation is difficult to avoid because answering ‘I don’t know’ or ‘that is entirely unpredictable’ is not a route to a successful career in the industry. Better to have a bold, well-articulated view and be wrong.
  • There is no requirement to be constrained by arbitrary benchmarks: Benchmarks are seen as the gold standard in assessing the value for money delivered by professional investors, but they are a behavioural disaster. They foster short-termism and create a situation where outcomes dominate process (this is particularly problematic because it is an environment where randomness and uncertainty are pronounced). Most pointedly they overwhelm behaviour – rather than focus on the consistent application of a philosophy or long-term client outcomes, the spectre of short-term benchmark comparisons looms large and inevitably drives decision making.
  • You don’t have to strive to be exceptional:  The investment industry is over-populated and highly competitive, which means to be successful many professional investors believe that they have to generate results that are exceptional.  The problem with this is that it leads them toward making decisions that are injudicious on average.  If you believe that you are exceptional then you can (over)confidently ignore base rates because they don’t apply to you.  Amateur investors suffer no such competitive threat – they can simply follow sensible investment principles and make decisions that are proven to be good on average (which, ironically, will probably lead to exceptional results relative to what other people are trying to achieve).
  • You don’t need to worry about what other people are doing:  Given the incentive structure, professional investors face they are often focused on what is working / performing and what is selling (often one and the same thing) – and can often be diverted from their core competencies.  Whilst amateur investors can easily be swayed by what other people are doing (it can be tough when your neighbour tells you about their biotechnology punt that is up 500%) they don’t need to be.

Of course, amateur investors are not immune from the plethora of behavioural issues that lead to poor investment decision making.  It is, however, important to acknowledge that the investment industry has certain structural features that serve to generate or inflame a particular set of behavioural shortcomings.  In a world of few edges, investors who can avoid them should be sure to do just that.

NB: When I was writing this post I came across an article from Barry Ritholtz that covered similar ground, hopefully the behavioural slant of this article makes it sufficiently distinctive to be interesting. His article is here.

* The term amateur sounds pejorative but this is not meant to be the case – it is simply a reference to those people who hold investments but do not manage them for a living i.e. the majority of individuals who aren’t professional investors.

A Behavioural Finance Toolkit: Six Steps to Better Investment Decision Making

I have written a new paper: ‘A Behavioural Finance Toolkit’, which aims to provide an introduction to the topic and look at how we can apply its lessons.  The paper concludes by offering the following six steps to improve our investment decision making:

+ Do have a long-term investment plan.
+ Do automate your saving.
+ Do rebalance your portfolio.
x Don’t check your portfolio too frequently.
x Don’t make emotional decisions.
x Don’t trade! Make doing nothing the default.

You can find the full paper here.