Many people involved in financial markets are engaged in a perpetual quest to identify skill – that is the attempt to seek out individuals or teams with the expertise to deliver abnormally strong investment returns. Whilst this is an understandable endeavour, it is also incredibly difficult.  In many domains and activities we can simply use outcomes as an effective proxy for skill, but in environments where uncertainty and randomness exert a significant influence, results alone can be woefully misleading.  Whilst an Elo rating may give you a robust guide to an individual’s ability to play chess; a track record of outperformance for an active manager will offer limited guidance on the underlying skill exhibited.

The vast, adaptive and reflexive nature of financial markets means that even if skill exists in certain areas, recognising it is hugely problematic. The complexity of the task means that investors typically default to a simple outcome driven approach – an effective heuristic in many other areas.

Given that focusing on outcomes alone is inadequate in an investment context, how should we approach locating skill in an activity where randomness heavily skews the results?   Rather than focus on one element, I think there are six important inter-related components that need to be considered: Specification, Calibration, Intention, Path, Outcome and Replication.

I will cover each in turn utilising a golfing analogy – although I don’t play the game, it is an activity that does incorporate both luck (less) and skill (more), and will hopefully serve to simplify the idea.

There are two golfers (Golfer A and Golfer B) both have taken one shot at par 3 hole and landed the ball very close to the flag – let’s say 3 feet. How do we determine whether each player has golfing skill?

Outcomes: If we were simply judging outcomes alone we might say that both golfers possess skill, as they have both produced excellent results.

Path: Understanding the path (how a result was achieved) can give us far richer information. Golfer A’s shot went arrow straight at the flag, Golfer B sliced their shot and it rebounded off a tree and onto the green.  Given this new information, we are emboldened in our view that Golfer A has skill, but now we are doubtful that Golfer B does – it looks as if they have just enjoyed a significant amount of luck.

Intention: It is very dangerous to assume that an individual has skill simply from observing an activity – if you don’t understand what they were trying to achieve beforehand.  If we know that Golfer A was attempting to hit his shot at the flag near which they landed their ball then we can have increased confidence that they possess skill.  But what if we knew that Golfer B was actually attempting to hit their ball onto the green after ricocheting off a tree?  Rather than believe that they had just been lucky, we might consider that they have superior skill to Golfer A because they performed a more difficult task.

Replication: Samples of one are never a good guide to skill and the more randomness in an activity the more evidence you require.  Although we might have a strong inclination that Golfer A and B both possess skill – with one example each we are incredibly vulnerable to being fooled by random occurrences.

Specification: When seeking to define skill, we need to be very specific about the activity in which someone possesses it.  Even if we witness both Golfer A and Golfer B repeat the exact same feat on numerous occasions – we can only be confident that they have skill in that precise task – we may infer that they are skilled golfers, but they might be terrible at putting, for example, a particular aspect of the game on which we have no evidence.

Calibration: All activities sit somewhere on the luck and skill continuum, and it is important to have a perspective on how much randomness and complexity there is in an activity before making any judgments about skill.  For example, landing a plane is dominated by skill with a slither of luck involved – if I witness an individual landing a plane successfully it gives me far greater confidence that they have skill in that task, than the confidence I might gain from watching someone hit a single good golf shot.  Trying to correctly calibrate the randomness inherent in an activity helps you to understand how much value there might be in the outcomes alone.

When chance is involved in an activity then we need to rely less on results. As we can see from the golfing example, understanding the different elements of the process can transform our view on whether we are observing skill or randomness.  When we are working to identify skill we should always be able to answer, at least, the following questions:

Specification: What is the precise activity in which we are attempting to identify skill?

Calibration: How much luck or randomness do we think is involved in the activity?

Intention: What is the objective of the activity?

Path: How has the objective been reached?

Outcome: What was the overall result?

Replication: How often has this process led to the same outcome?

In the investment industry we give pre-eminence to outcomes when determining skill. Even when we incorporate other factors, our perspective is often biased by the strong priors we develop after initially observing performance – if we see strong performance; we assume skill must be involved.  We are also prone to assume that apparent skill in one specific aspect translates across the entire spectrum of investment activities – someone is often considered a ‘good / great investor’- good at what, exactly?

Although skewed incentives and our obsession with outcomes make it incredibly demanding, the only way to even attempt to successfully identify skill is to understand not what the outcomes were, but precisely how they have been achieved.

If you ask a fund manager why they believe that their investment philosophy can generate excess returns, they will almost inevitably state that they are seeking to exploit the behavioural biases exhibited by other investors that create pricing inefficiencies.  It is somewhat puzzling therefore that if you question the same fund manager about how they seek to address their own biases the response is either entirely unconvincing or evasive.

There is a stark contradiction in acknowledging an awareness of the influence of behavioural biases (to such an extent that the perceived viability of your investment strategy is founded upon it) but then willfully ignoring your own susceptibility.  Although there are competing explanations for this phenomenon the most compelling is the ‘blind spot bias’ – we see bias in others, but not in ourselves.  This concept was developed by Emily Pronin, Daniel Lin and Lee Ross, and reported in 2002[i] – the main conclusion of their research was as follows.

“We propose that people recognize the existence, and the impact of most of the biases that social and cognitive psychologists have described over the past few decades. What they lack recognition of…is the role that those same biases play in governing their own judgements and inferences”

In the study, participants were asked to rate their own susceptibility to various biases compared to the average American – these included: the halo effect, dissonance reduction and biased assimilation of new information.  Although the magnitude of the results varied, the direction was consistent – they considered themselves to be less vulnerable to bias than other people:

One obvious confounder for these results was that they may have reflected the participants’ perceived general superiority over the ‘average American’ – they were all Stanford University students; however, the same pattern emerged when they were asked to compare their capabilities to fellow students.

Our broad tendency to assume we are better than average was also contradicted in the same study as participants stated that they were actually worse than others when it came to certain personal limitations, such as procrastination and fear of public speaking.  It was only when specifically considering biases that the ‘blind spot’ appeared.

Given the issues with failed replications in psychological research, it would be easy to question the results of this work – particularly given that it was lab based, with a small student sample; reassuringly, however, the bias blind spot outcomes were repeated in a recent replication study[ii] (not yet peer reviewed). This was carried out across a far larger sample, which suggests a level of robustness in the main conclusion of the original[iii].

The bias blind spot theory certainly provides a convincing explanation as to why many investors seem to actively seek to exploit behavioural bias in others, whilst at the same time being reticent to acknowledge and address the issue from a personal perspective. But why do investors underestimate their own vulnerability to bias?  Below are five (of many) possible explanations:

Overconfidence: Our belief that we are better than others is probably the most obvious explanation; this issue is exacerbated for professional investors as there is undoubtedly a selection bias into fund management roles toward those with exaggerated levels of confidence in their own capabilities.  Inherent in the role of an active investor is the presumption that you are more skilled than other market participants – this might stretch to believing you are less subject to behavioural biases.

Cognitive dissonance: Whilst the overconfidence explanation focuses on how we perceive ourselves relative to others, cognitive dissonance is focused on how we judge ourselves internally.  There is an inherent friction in considering yourself a highly capable investor but also being susceptible to a range of behavioural biases – particularly given that these biases are often irrational and simplistic.  One way to alleviate this dissonance is to assume that you are above such biases.

Too complex / too difficult: It may simply be a case that dealing with personal biases is too difficult.  The list of biases is extensive, definitions are sometimes vague and they can often be contradictory.  For example, I recently attended a conference where there was a discussion of the endowment effect and how it can lead to fund managers maintaining winning positions for too long; whilst this is a valid explanation, it is in sharp contrast to the disposition effect, wherein investors are prone to sell their winners too rapidly.  Here we have two credible biases supported by academic research, but where their effects are directly contradictory.  Trying to incorporate such disparate information into our own decision making can be fiendishly difficult.

Personal narratives: When objectively and dispassionately observing another person’s investment decisions it is often easy to identify the potential biases that are likely to be influencing their judgement; however, it is far more challenging to adopt a similar approach for our own choices.  Instead, we are likely to create narratives around our own decisions that diminish the role of any bias – whilst our desire to sell a failing position might be due to outcome bias or myopic loss aversion; we will convince ourselves that there is a ‘pure’ investment rationale informing our view.

The sales message: Perhaps the reticence of professional investors to engage with their own bias is related to a general reluctance to acknowledge mistakes.  The sales message around active managers is about high conviction insights and the presentation of ‘hero’ stock performance charts – in at the bottom and out at the top.  Pitching to clients and discussing all of the irrational decisions you have made is not the route to convincing them of your skill.

None of these potential justifications are a reasonable excuse for understating our own behavioural limitations or failing to actively mitigate them.  Given how few genuine edges are available in the investment industry, it is baffling that this one remains widely neglected.

—

[i] Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28(3), 369-381.

[ii]  Replication study of Bias Blind Spot

[iii] https://digest.bps.org.uk/2019/03/27/good-news-for-science-bad-news-for-humanity-the-bias-blind-spot-just-replicated-everyone-else-is-more-biased-than-me/