“Nothing in life is as important as you think it is while you are thinking about it.” Daniel Kahneman

Which major investment issue were you thinking about on March 11^th 2016?* You probably can’t remember even though, at the time, it seemed incredibly important.  Whilst most of us should be investing for the long-term, markets conspire against us; lurching from one obsession to the next, drawing our gaze and enticing us to take action.

As the news about a particular event or development takes centre stage, and experts hold forth about the potential outcomes, the notion of doing nothing can seem ridiculous.  The issues are, by definition, salient, recent and available; all factors which make it very difficult for us to have any reasonable perspective on their long-term significance.

It is not that matters such as war on the Korean Peninsula / peace on the Korean Peninsula or Italy leaving the EU are not meaningful (to take recent fascinations), but, from an investment perspective, there is very little most investors can do to benefit.  Each year there is a succession of topics that we become excessively diverted by, where the temptation to act is strong; but before we do, we should try to answer the following three questions:

1) Does it matter to returns?  Given that we tend to hugely overstate the importance of the issue that we are currently focused on, we are likely to assume far more things matter to long-term returns than actually do. It is vital to remember that the historic long-term returns of major asset classes inevitably include periods of tumult at least as significant as the one that currently has our rapt attention.

2) What is going to happen? Even if we are confident in the view that returns will be impacted by a given issue, we then have to predict what the outcome will be.  I think enough has already been said about our ability to forecast such things.

3) How will it impact markets? In the unlikely event that we have identified an event that will materially alter asset class returns and successfully envisaged the outcome, we then need to understand how markets will react.  Markets are reflexive and unpredictable. Do we really know how other investors will behave or what’s in the price?

In simple terms, such activity is incredibly difficult to get right, particularly on a consistent basis.  Even if we have the foresight to identify which events truly matter amidst the clamour – we then need to forecast a particular outcome and the response of markets.  Whilst some professional investors specialise in this activity, most of us should avoid such heroics.

This is easier said than done.  Financial markets create a cacophony of noise and a flow of narratives that we find irresistible; a vicious circle forms where even if we want to disregard an issue, we cannot because it is considered unacceptable to do so.  How can you be ignoring something that is so prominent and material?  Action and opinion are incredibly highly valued, even if their true worth is often negative.  Thus, we end up in a situation where investors spend the vast majority of their time on things that don’t matter and not enough on the things that do.  I imagine the breakdown of investor attention as being something akin to this (entirely unscientific):  

Having an asset allocation that is suitable for your requirements, considering valuations and thinking about how best to control your behaviour is the surest way to deliver solid results, whilst avoiding the most common investment mistakes.  Taking a long-term approach doesn’t mean you should set once and forget, rather think carefully about your time horizon when making decisions and don’t check your portfolio too regularly.  Doing nothing should be a strong default.

Investing for the long-term seems easy until you understand that it is comprised of many days and many more temptations.  Financial markets will do their utmost to lure you toward the rocks, be sure to tie yourself to the mast.

* This is a random date, I have no idea what particular issues I was being distracted by at the time.    

As noted by Jason Collins in his excellent behavioural economics blog, Daniel Kahneman’s next book is expected to focus on the concept of ‘noise’ and how it impacts our judgements.  Although often conflated with behavioural biases, noise is a distinct phenomenon that relates to the random variability in our decision making.  Whilst biases exhibit a consistency of effect (at least in direction, if not magnitude), noise is defined by the absence of consistency.  A watch that loses time each day is biased; a watch that can either gain or lose time during any given day is noisy.

In an article in Harvard Business Review, Kahneman, alongside his collaborators, discussed how individual choice is “strongly influenced by irrelevant factors” and gave examples of how professionals are prone to contradict their own previous conclusions.  The problem of attempting to grasp the idea of noise is that it is so amorphous – whilst we can at least develop a (limited) framework for defining and understanding biases, by definition noise is hard to isolate and anticipate. Noise can stem from entirely spurious factors – such as mood, weather or hunger – or variables that we perceive to be meaningful, but are in fact meaningless.  It is certainly possible to test whether noise exists in any given scenario – by observing decision making consistency – but this is only the start of understanding the issue.

Noise has profound implications for investors, but is often ignored or, at least underappreciated. It can be difficult to accept that our judgements can be shaped by erroneous, often farcically minor, factors.  Furthermore, we are often uncertain about the key variables that define any given problem.

In the realm of investment decision making, we can define two separate forms of noise:

• When given the same objective data and relevant variables we are unlikely to make the same decision. This is consistent with Kahneman et al.’s article – even if we hold the meaningful factors constant, other irrelevant issues will lead to inconsistent choices.

• We don’t know what the relevant information is and therefore make decisions based on what we perceive to be ‘signal’ but is in fact noise. This is such a major problem – one which the industry perpetuates – that it is difficult to know where to begin.

We could crudely define these as unconscious noise and conscious noise.  In the first case there are many factors that impact our decision making over which we have no real awareness and we would be reticent to acknowledge had any influence over us.  In the second case, the issue is uncertainty about what constitutes relevant information and what is superfluous noise – this will vary by context and discipline, but it is difficult to think of an industry with a greater ratio of noise to signal than asset management. Conscious noise is the oxygen on which the industry, in its current form, exists.

Second by second coverage of random market movements (with accompanying narratives), heroic forecasts (usually wrong), luck masquerading as skill, complex products and every decreasing time horizons are just a few of the factors that contribute to the maelstrom of noise that investors are forced to navigate.  Of course, this is good for the industry – simplicity and inaction are not typically an aid for revenue generation – but it fosters a situation where decision consistency becomes close to impossible for most investors.

Kahneman et al. proceed to argue that a “radical solution” for the problem of noise is the replacement of human judgement with algorithms, or a structured set of decision rules. They also acknowledge, however, that such processes are less effective in environments where uncertainty is high or where consistency is difficult to attain.

Can algorithms be effective  in muting the incessant noise in investment markets, and even exploit it, to improve decision making?  To a certain degree.  One effective and humble decision rule / heuristic, is portfolio rebalancing.  A structured and consistent approach to rebalancing a portfolio back to target weights is proven to be effective and cancels out a great deal of market noise.  It ensures both that your portfolio doesn’t stray too markedly from its desired allocation, and that you consistently sell assets that have become more expensive and reinvest in those that have become cheaper.  Whilst this might seem a simple course of action, rebalancing into assets that have struggled (amidst the prevailing negative market narrative that will inevitably accompany the poor performance) can be difficult without a formal / systematic decision rule.

The oft-mooted remedy to the problem of noise and inconsistency in human-led investment decisions is the movement to full automation and the use of complex algorithms / machine learning.  It deals, at least in part, with the aforementioned ‘unconscious noise’ angle as the feelings of the decision maker are no longer a direct issue, however, at some point human judgement will inevitably exert an influence – for example, in the decision to initially invest in a strategy or to redeem, thus it does not provide full immunity.

More importantly, algorithms do not necessarily resolve the issue over noise in regard to the use of irrelevant information.  Many an ETF has been created based on factors with no empirical credibility; therefore although the decisions within the process can be dispassionate, the very existence of such strategies is reliant on the fact that there is noise in the market.  Furthermore, even the most sophisticated systematic approaches are vulnerable to trading based on patterns than are simply a consequence of random market movements, with no structural, technical, economic or behavioural reason to exist or persist.

Even without full automation, there are means of dulling the noise in human-led investment decision making, such as checklists.  Whilst checklists are particularly effective in areas such as aviation and surgery where many of the checks can be simple and objective –  whether the correct leg is being operated on, for example – they can still improve discipline and focus around decisions with inherently more subjectivity. Formally reviewing a list of your key criteria prior to making an investment can serve to highlight noise driven deviations from your core process and also acts as a useful record for reviewing historic decisions.  Of course, using checklists when answers are subjective means the potential for manipulation is ripe, so it pays to be as rigid as possible when defining questions; however, even if this is not possible, checklists remain a useful means of reaffirming your investment principles amidst the market noise.

Noise is an inescapable feature of human judgement.  In random and uncertain investment markets its influence is profound. Although it is impossible to eradicate, acknowledging its presence and taking steps to simplify and systematise certain decisions can be an effective way of turning down the volume.

Key Reading:

Kahneman, D., Rosenfield, A. M., Gandhi, L., & Blaser, T. (2016). Noise: How to overcome the high, hidden cost of inconsistent decision making. Harvard Business Review, 94(10), 38-46.

Please note all views expressed in this article are my own and are not necessarily shared by my employer.

“It infuriates me to be wrong, when I know I am right.” (Moliere)

In a previous job, I worked for an investment consultant who employed a psychologist to assist in their fund manager research process.  I vividly recall her making a particularly astute point about how we constantly hear about the failings of active fund managers, but rarely anything about how they can improve.  Whilst inevitably there are exceptions to this view, it is certainly reasonable to suggest that evidence of fund manager learning and development is scant.  I do not mean that there is a lack of generic, box ticking courses with no specific purpose, rather an apparent absence of weaknesses being openly identified and addressed.

This may seem odd as active management should be a ripe environment for constant improvement – the difficulty level is high, there is continual feedback (though not necessarily the right sort) and sufficient latitude in the role to change behaviours.  There is, however, a major problem; a key aspect of learning is the ability to identify areas of limitation and, in particular, admit mistakes.  This is something that fund managers are typically not very good at.  Although it is unfair to imply that the profession is unique in this regard – it is a constant struggle for us all – the environment in which a fund manager operates makes it particularly difficult to be open about failings, and therefore seek remedies.

“When an expert is wrong the centrepiece of his or her professional identity is threatened” (Tavris & Aronson, 2008)

The primary driver of our inability to admit errors and failings is cognitive dissonance – the battle to deal with situations where we hold contradictory beliefs, attitudes or feelings.  Such instances are deeply uncomfortable and we seek to resolve these inconsistencies as rapidly as possible, in order to preserve a coherent sense of self.  The most common ameliorative is some form of simple justification: “I am a safe and considerate driver, but the message on my phone could have been urgent, and the road was clear”.  This sort of internal monologue is a crucial part of how we navigate everyday life and look to validate our own behaviour.

Whilst cognitive dissonance is focused on self-perception, a similar dynamic is apparent in how we seek to present ourselves. In The Presentation of Self in Everyday Life (1959) sociologist Erving Goffman describes how individuals seek to manage and control the information that we offer others, and are consistently engaged in some form of performance, where we try to shape the impression that we leave.  Thus, we are often behaving in a way that we perceive to be consistent with what we believe are the key traits of the particular ‘role’ we are hoping to play – am I meeting the criteria expected of this type of person?

We are constantly faced with the challenge of attempting to manage how we perceive ourselves and how others perceive us.  To develop their career, and sell their own capabilities, fund managers are required to present themselves as assured experts with an ability to opine on virtually any investment issue and present their ideas with little hint of doubt. The expectations of fund managers almost entirely contradicts the reality – adding value in markets is fiendishly difficult and a hit rate north of 60% should be deemed heroic; furthermore, any edge possessed by a particular manager will likely be focused and specific, not all encompassing. Markets are chaotic and unpredictable, even for the most talented manager, mistakes will occur with alarming regularity.

Yet it is hard to sell your fund to clients if you talk about failed positions, and approach your investment ideas with circumspection rather than unabashed conviction. Fund managers need to convince themselves and others that they have expertise, and a level of expertise that is superior to peers. Experts don’t make mistakes…

The other major problem for fund managers is one of commitment. A crucial part of their job involves convincing themselves, colleagues and clients of their opinion, and the more this opinion is repeated the more committed they become to it. In the face of overwhelming disconfirming evidence, fund managers often persist with a view or indeed become even more emboldened in it – rather that than lose face and recant on their previously high conviction ideas.  On countless occasions I have witnessed fund managers take such forthright positions that it becomes obvious that even if they are proven wrong it would be entirely unconscionable for them to perform an about face.  Often an investor’s entire professional identity will be forged on one particularly bold viewpoint.

In addition to simply ignoring mistakes, there are a number of tactics fund managers can employ to protect their ego.  These were first raised by Phillip Tetlock (1999) when he analysed how forecasters dealt with having particularly poor records in forecasting, and also highlighted by James Montier in his Seven Sins of Fund Management. My favourites include:

–  “If only”: If only the company hadn’t decided to engage in that expensive acquisition, my view would have been correct…

– “Ceteris paribus”:  Some external factor, which was impossible to foresee, impacted my otherwise robust analysis.

– “Not happened yet”:  A favourite of the dogmatists and those with commitment bias issues – I am right; it has just not been reflected in markets, yet

The problem is not that such excuses are always without merit, but that they subvert any need for introspection. As financial markets exist in a constant state of flux, it is all too easy to move forward with minimal consideration of the decisions that have come before.

The mistake I can recall most readily from my own career was a decision to sell a high yield bond fund early in 2009; if this was not at the ultimate peak of credit spreads, then it was impressively close.  I have, of course, made many errors since, but this is one that resonates sharply, I think because it was particularly stupid. It was not a mistake because junk bonds generated strong performance following the sale (you cannot decide the quality of a decision based solely on noisy market outcomes), but because, based on the evidence available, there was a (very) high probability of subsequent returns being significantly above the historical average.  This was a clear behavioural error driven by the sheer tumult of the period, leading to an ignorance (wilful or otherwise) of the data and historical precedent. I actually remember very little about the decision, except that it felt good for a few days and then bad for a few years.

This sort of mistake can be considered simple, in that there is the combination of a bad process followed by a bad outcome – these are the easiest to acknowledge and most difficult to ignore.  Investment markets, however, aren’t often as straightforward; their randomness and unpredictability often mean that sensible decisions are regarded as mistakes, whilst poor judgements can masquerade as investment acumen. Given this, it is naive to review a fund manager’s underperforming positions and suggest that they all constitute mistakes.  Relative stock price performance is far too capricious to use as a reliable arbiter of decision quality.

Identifying and understanding a mistake requires knowledge of the initial rationale for a particular course of action and then some comparison with the outcome.  This does not mean making forecasts, but simply comprehending the key drivers behind a decision.  For example, if part of my rationale for investing in a particular company is its strong balance sheet and it subsequently struggles with its sizable operating lease commitments, that is a specific error of judgment. Approaching mistakes in a process / outcome manner is the only way to address shortcomings and seek to make improvements.

We can think about most mistakes made by fund managers as either analytical or behavioural.  In the former, specific areas of flawed analysis can be pinpointed – either we looked at the wrong evidence or misinterpreted it, as in the example on lease commitments in the previous paragraph.  For behavioural mistakes, we make poor decisions despite having the appropriate evidence available to support a decision; my mistake of selling high yield at the most inopportune time was made in spite of their being sufficient information for me to know ex-ante that this was likely to be an error.  It was simply that a swathe of behavioural factors (most likely recency, availability, probability neglect and affect-led decision making) became overwhelming.  Of course, these two broad categories of mistake are not mutually exclusive; many analytical mistakes will be driven by behavioural flaws, but spending some time at least attempting to decipher how mistakes were made, and perhaps identifying patterns, can be a valuable process.

The concept of learning from mistakes can sound horribly trite, but it is actually an integral element of improved decision making.  In many ultra-competitive professions, such as fund management, where expectations about performance are significantly above any semblance of reality, being open about lapses and faults can damage ego, harm career prospects and lose clients. However, choosing to ignore or obfuscate rather than engage with things that have gone wrong is undoubtedly of detriment to future behaviour.  Taking steps to better engage with mistakes is not technically difficult, and there are a number of simple steps that can make a material difference, such as:

– Documenting decision making appropriately:  Make sure that prior to committing to a decision you detail the underlying rationale and key risks. It is impossible to assess a mistake after the event if you have not explicitly noted your thinking at the time a decision is made.  As much as you might think you can recall the reasons behind a past course of action – your memory is too fickle to be a reliable source.

– Performing a regular mistake audit: On an annual or semi-annual basis you and your colleagues should write down two / three mistakes they have made over the period.  They have to be genuine mistakes, with no caveats or excuses allowed.  This sort of process can go at least some distance toward creating a more favourable climate for embracing the times when we err.

These steps, however, can be fruitless (or indeed counterproductive) if you are operating in an environment that continues to stigmatise mistakes. In such organisations and teams, mistakes are ignored, excuses created and responsibility apportioned elsewhere.  Although this may enhance the esteem of the team (internally and externally); it is a terrible waste of a great resource to improve future decision making.  As the structural pressures on active fund managers increase, they need to grasp every available edge.

Key Reading:

Goffman, E. (2002). The presentation of self in everyday life. 1959. Garden City, NY.

Montier, J. (2005). Seven sins of fund management.

Tavris, C., & Aronson, E. (2008). Mistakes were made (but not by me): Why we justify foolish beliefs, bad decisions, and hurtful acts. Houghton Mifflin Harcourt.

Tetlock, P. E. (1999). Theory-driven reasoning about plausible pasts and probable futures in world politics: are we prisoners of our preconceptions?. American Journal of Political Science, 335-366.

Please note all views expressed in this article are my own and are not necessarily shared by my employer.

The idea of loss aversion – that losses ‘loom’ larger than gains – is one of the most established and prominent findings in behavioural economics, and could be considered a foundation stone for the entire discipline.  Recent research, however, has questioned the validity and robustness of the supporting evidence, suggesting that it is at worse a false concept and at best overstated (in particular see Yechiam 2018, Gal & Rucker 2017).  Given the issues surrounding p-hacking and failed replications, placing such widely-accepted beliefs under scrutiny should be applauded yet, in this instance, some of the claims appear exaggerated.

One prominent criticism of loss aversion has been that its presence is reliant on the stakes involved being of a sufficient magnitude –  the evidence evaporates if the potential loss or gain is not meaningful.  Harinck et al. (2007) argue that the loss aversion phenomenon actually reverses when the amounts of money involved in a decision are small.  This makes intuitive sense – if an individual were gambling a minor amount of ‘play money’ on a slot machine during a visit to a casino, they are unlikely to be particularly loss averse, indeed the ‘utility’ from winning could easily outstrip any pain from losing.  Conversely, if a risky decision has the potential to incur a material cost, then the classical features of loss aversion should take hold.

The extent to which a loss is considered meaningful or an amount of money considered ‘small’ will also be heavily dependent on its size relative to wealth.  The same monetary amount at stake could be viewed as inconsequential to an affluent individual, yet incredibly valuable to another with less resources – the ‘marginal utility’ of the lost money would be far greater in the latter instance.

The influence of bet size and relative materiality are two reasons why it is difficult to create general rules around the concept of loss aversion; however, more vital, and certainly less prominent, is the importance of reference points.  Reference points dictate what is considered a loss or gain – we can think of them as a break even point. Although in certain circumstances defining a loss may seem simple, this is far from the case – losses are subjective rather than objective. Understanding reference points is crucial in ascertaining how and where loss aversion may occur.

The major problem is that reference points are not fixed, but subject to a multitude of behavioural biases and heavily dependent on individual differences, environment and decision context.  Let’s take a simple investment example to illustrate the point:

There are three investors, A, B and C, each have been invested in the same portfolio for the previous 12 months.  Over this period, the portfolio has fallen in value 10% and its benchmark has lost 15%.  From the perspective of loss aversion and reference points, how does each investor feel about this outcome?

– Investor A is satisfied with the performance relative to the benchmark and considers it to be a ‘gain’.

– Investor B is disappointed with the absolute loss suffered by the portfolio over the period.

– Investor C is pleased with the returns as he just spoke with his friend and their portfolio lost 22% over the same period.

This is a heavily stylised example, but aims to emphasise the point that reference points can vary between individuals and within individuals, and in many cases it is impossible to know what that reference point is.  Absent that information, it becomes difficult to precisely understand or predict behaviour consistent with loss aversion in all situations.  Of course, in certain circumstances the reference point might be obvious or we might be able to decipher it from analysing individual behaviour, but they should still be considered highly variable and vulnerable to manipulation.

Given that the genesis of behavioural economics was, at least in part, a reaction to the rigidity of classical economics and the ultra-rational assumptions made about individual / collective behaviour, it seems nonsensical to criticise loss aversion for not being universally applicable.  It is, however, useful to be reminded that circumspection is required when making broad claims about any research findings.

Will loss aversion appear in a consistent fashion, irrespective of context or individual difference?  No, but in some ways that it the point.

Key Reading:

Gal, D., & Rucker, D. (2017). The Loss of Loss Aversion: Will It Loom Larger Than Its Gain?.

Harinck, F., Van Dijk, E., Van Beest, I., & Mersmann, P. (2007). When gains loom larger than losses: Reversed loss aversion for small amounts of money. Psychological science, 18(12), 1099-1105.

Yechiam, E. (2018). Acceptable losses: the debatable origins of loss aversion. Psychological research, 1-13.

Please note all views expressed in this article are my own and are not necessarily shared by my employer.

Although increasingly aware of the concept of behavioural biases; we remain poor at accepting that they directly relate to our own decision making, largely because we struggle with the negative inferences.  Take professional fund investors (a group I consider myself to be part of), we are vulnerable to a host of established biases but many of these are rarely acknowledged or discussed.  What can we infer from behavioural science research about the judgments made by buyers of mutual funds?

– We will favour fund management groups that have provided gifts and hospitality, no matter the price.  Small gestures may even be more influential than those of greater value.

Whilst undoubtedly the most difficult bias to acknowledge, the evidence on reciprocity – our desire or obligation to return a good deed – across all aspects of life is overwhelming, and professional  fund investors are not immune.  The fund industry (certainly in the UK) has sought to restrict the value of gifts and hospitality by asset managers seeking to sell their wares to prospective investors; however, the notion that limiting the monetary value prevents reciprocal behaviour is entirely spurious (see Katz, Caplan and Merx, 2003). Indeed, small gifts can be more influential – individuals provided with lavish gifts and hospitality may not invest in a fund managed by the giver for fear of the perception that their decision has been influenced, but as small gifts and hospitality are considered ‘inconsequential’, they can accept them without any questions raised about impartiality.  It is crucial to highlight here that reciprocity is an ingrained and often unconscious act, in most cases we will be blithely unaware of how our behaviour is being impacted, and indeed we are likely to rail against the very notion that we can be influenced in such a fashion.

– We are inclined toward investing with fund managers / sales people that we like.

Another of Robert Cialdini’s key pillars of influence (in addition to reciprocity) is ‘liking’; that is we are more readily persuaded by people that we like. This might be because they are similar to us, because they offer us compliments: “that was a great meeting, you really understand our fund”, or because we find them physically attractive.

– The further into an in-depth due diligence process we are for a fund; the less likely we are to identify a major problem with the strategy.

Detailed due diligence is crucial for any credible fund selector, but the time and effort of such a process can leaves us vulnerable to sunk costs – after multiple meetings and reams of analysis the prospect of abandoning research can become unpalatable.  More work makes us more committed (Garland 1990).

– The moment we see the performance track record of a fund our assessment of the quality of the philosophy, process and fund manager will change.

The grip of outcome bias is difficult to escape and even harder to see in ourselves.  In Baron and Hershey’s (1988) study on medical decisions and monetary gambles they found consistent evidence that participants rated the quality of a decision as better and the decision maker as more competent when the outcome were favourable rather than unfavourable (all other things being equal).  Given the amount of luck and randomness in the returns delivered by active funds, outcome bias is a particular problem for fund investors.

– We will recognise worrisome process related issues with underperforming fund managers that we would never have ‘identified’ had the performance been strong.

Outcome bias is so important to fund manager selection that I will mention it twice.  A classic case of the issue is evidenced by mutual fund investors propensity to sell losers and run winners (the reverse disposition effect), which is driven, in part, by our assumption that poor outcomes (disappointing performance) must be linked to problems with the process.  Thus, team changes, capacity issues or style drift are often cited as reasons for the sale of an underperforming fund; even if results can better be attributed to the general wax and wane of financial markets.

– We will blame the fund manager for poor performance and accept the credit for outperformance.

That we tend to internalise success and externalise failure has been well-documented in psychological research with a variety of causes, such as the desire to present ourselves in a favourable light or to enhance or confirm our own self-worth (Shepperd, Malone and Sweeny, 2008).  As fund investors we are, in essence, delegating investment decisions to a third party, and the opportunity to succumb to self-serving bias is great; as are the potential problems stemming from it – most notably an inability to learn from mistakes.

– We will erroneously associate strong presentation / public speaking with the possession of fund management skill.

The halo effect is a situation where we take one prominent and strong trait for an individual (or group) and extrapolate our positive view across of all of their characteristics (Nisbett & Wilson, 1977).  There is no proven correlation between communication skills and fund management aptitude, but we can easily become captivated by compelling presenters and discard those lacking the ability to converse in a convincing manner. It is also important to note that there is inevitably a selection bias in the pool of available active fund managers, with confident and persuasive individuals more likely to ascend to the upper echelons of the industry.

Key Reading:

Baron, J., & Hershey, J. C. (1988). Outcome bias in decision evaluation. Journal of personality and social psychology, 54(4), 569.

Cialdini, R. B., & Cialdini, R. B. (2007). Influence: The psychology of persuasion New York: Collins.

Garland, H. (1990). Throwing good money after bad: The effect of sunk costs on the decision to escalate commitment to an ongoing project. Journal of Applied Psychology, 75(6), 728.

Katz, D., Caplan, A. L., & Merz, J. F. (2003). All gifts large and small. American Journal of Bioethics, 3(3), 39-46.

Nisbett, R. E., & Wilson, T. D. (1977). The halo effect: Evidence for unconscious alteration of judgments. Journal of personality and social psychology, 35(4), 250.

Shepperd, J., Malone, W., & Sweeny, K. (2008). Exploring causes of the self‐serving bias. Social and Personality Psychology Compass, 2(2), 895-908.

Please note all views expressed in this article are my own and are not necessarily shared by my employer.

When talking to active managers, fund investors can focus on the wrong things – we are heavily biased toward the short-term, and obsess over issues that are recent and salient.  We also drastically overvalue confidence as a characteristic, whilst punishing circumspection, realism and humility.  Given this, it is unsurprising that conversations with active managers are often shaped in a manner that is entirely at odds with the capricious and unpredictable nature of financial markets, and do little to help identify skill.

Active managers inevitably attempt to present themselves in the manner that they believe most appeals to investors and there are, therefore, many things that they should say, but rarely do.  Here are some examples:

“It was a genuine mistake – our analysis was incorrect, but I will make sure I learn from this for future decisions.”

“Whilst I believe in this investment, given that I will get at least 40% of my decisions wrong (assuming I do a very good job), this could be such an occasion.”

“It is important to highlight the pronounced style tailwind that I have enjoyed during this period of outperformance”.

“I can’t confidently predict the outcome of this event and, even if I could, it would be difficult to gauge how markets would react”.

“Although the trade was profitable, the situation did not develop as I had imagined and its success was actually just a dose of good fortune”

“Whilst the event is material and may have significant ramifications for markets I just don’t know what these will be with any certainty, so have decided to do nothing”.

“I appreciate that recent market volatility feels significant, but I don’t want to focus on it because, on a ten year view, it is likely to seem meaningless”.

“I am happy to give my perspective on the macro environment, but I have no particular expertise in the area, certainly relative to the thousands of others opining on the same subject”

“Of course I can review three month performance, but the results are almost entirely random market noise”

“I appreciate that I previously held a high level of confidence in this view, but, after careful analysis of new evidence, I realised that I was wrong”.

“I have absolutely no idea what equity markets will do over the next 12 months, but the odds are in favour of them going up”.

“I am fully aware of our propensity to make behavioural mistakes, and this is how I aim to minimise them…”

“On the balance of probabilities…”

There is persuasive evidence that a select group of risk factors in equity investment provide investors with a return premium – that is they offer a long-term performance advantage relative to the broad market, on a risk-adjusted basis.  The most established and robust are value, momentum and size.  All three of these meet the crucial criteria of being present over an extended time horizon, pervasive across markets and supported by a behavioural / economic foundation.

Based on this, I consider it prudent to have exposure to these factors in a portfolio. However, this acceptance raises an immediate question – if I believe in the efficacy of these factors now, is this decision irrevocable? If I take a view about there being certain structural risk premia in equity markets, is it possible to change my mind? What could the rationale be for such a shift of view?

It is important to make a distinction here – I am not referring to the cyclicality of returns from a factor, I fully acknowledge that the performance of all equity risk factors will wax and wane, often over long cycles.  Rather, my concern is a situation where the long-term expected risk premium for a particular factor evaporates because of some change in the drivers of the anomaly.  I do not worry that the size factor will fail to deliver for sustained periods (for example), but that the premium no longer exists.

Whilst such an occurrence may be considered to hold a low probability, it surely cannot be impossible. Therefore a risk exists, yet one that is difficult to identify and quantify for a variety of reasons:

–  The pillars of all credible equity risk premia are economic and behavioural rationale. Structural excess returns cannot simply be present in the data; there must be a reason for their existence – I have written previously about behavioural drivers for value and momentum.  The problem is that these are only suppositions, in most cases it is impossible to precisely and unequivocally identify the causes of certain risk premia. It therefore follows that we cannot confidently observe when these alter.

–  Long term horizons (large sample sizes) are a prerequisite to build confidence in the robustness of equity risk premia. If we require multiple decades of data (at the very minimum) to support our behavioural / economic hypothesis, we need similar periods of time to ‘disprove’ them.  By the time there is sufficient evidence to support the contention that a risk factor no longer effective, you are likely to be enjoying retirement.

– The problem is exacerbated by the fact that even if a currently sound factor stopped working it would still behave, at times, as if it did.  Let’s assume that from this day on there was no value premium in equity markets – even in such a scenario there would continue to be extended periods when value outperformed.  Expensive stocks can (and do) generate prolonged periods of excess returns, despite there being no evidence of a structural premium.

–  In the event that you are willing to declare the demise of a previously robust equity risk premium, you are likely to be wrong, and at a particularly inopportune time – the sounding of the death knell for value investing will almost inevitably be prelude to resurgent performance from the factor.  It is easy to mistake the cyclical for the structural, and abandon a style of investing when it is depressed and the valuation dispersion wide.

– It is simply not in the interests of the majority of groups working within the burgeoning factor investment sub-industry to state that an prominent risk premium has lost its efficacy – informing clients that the factor they have invested billions in is unlikely to work in the future seems to be an unwise and unlikely business decision.  There are undoubtedly investors who would seek to exploit the change in factor dynamics, but these would be the exception rather than the rule.  It is not an industry renowned for parsimony.

As with all investment judgement, the consideration of factor viability it is about uncertainty and subjective probability. On the weight of evidence, it is likely that established equity risk premia will continue to deliver over the long term, however, there is a low probability that they will not, and it is important to incorporate this possibility into your decision making.  It is also prudent to assess how rich or cheap a particular factor is relative to its own history, which may provide some protection in the event that the presumed structural risk premium is no longer present. Neither of these options directly addresses the central question posed in this article, simply because there are no obvious answers.

One of the many oddities of the investment industry is that whilst it operates in a constant state of uncertainty and flux, its participants talk in the language of certainty.  Confident forecasts and dogmatic opinions are valued far more highly than circumspection and caveats.  This bias stems from the mistaken belief that expertise is related to conviction – that an expert must know the ‘right answer’.  Whilst this might hold in certain situations where the environment is stable and skill dominates outcomes; the reverse is true in random and unpredictable domains, such as financial markets. Here, expertise is more evident in humility, a willingness to revise views and to deal in probabilities.

There is certainly truth to the view that we are poor when it comes to thinking in terms of probabilities, and there are multiple examples of our irrationalities in this area – such as the tendency to neglect probabilities when considering extreme scenarios, and our propensity to overweight or ignore small probabilities.  The notion, however, that we should not talk about probability because we have limitations in this regard is entirely spurious; we cannot make a decision without taking some view on the likelihood of potential outcomes, even if we are not explicit about it.  If we bring our probability assessments into the open; it materially improves our ability to understand our biases, receive feedback on our judgements and learn.

Aside from our general struggle with thinking in such a manner, there are a range of other issues that limit the use of probabilities when making investment decisions and forecasts:

–  Lack of expertise: As soon as probabilities are expressed around a variety of potential outcomes, there is an acknowledgement of uncertainty, which is often erroneously viewed as a lack of expertise and akin to the cardinal sin of stating “I don’t know”.  This issue is exacerbated by the fact that there will be others making bold, singular predictions – surely, they must know better?

– Spurious accuracy: In uncertain environments views on probabilities are necessarily based on subjective judgements. Applying specific probabilities to a range of scenarios can appear overly scientific – what does it mean to believe the likelihood of an event is 17%?  This view, however, misses the value of applying probabilities.  Its use is not as a precise figure but as a measure of confidence and a means to monitor how our views evolve through time.

– Implementation challenges: It is far easier to translate a strident, narrow view into an investment decision, than to attempt to reflect uncertainty and a variety of potential outcomes.

– The forty percent problem:  Probabilities can be misused when making forecasts – the classic case is an individual assigning a 40% probability to an outlier event (a recession, typically). The 40% level means that if the event doesn’t occur their forecast was correct (on balance), but if the scenario does transpire it is of a high enough likelihood for them to be feted for the prediction.  This is an example of probabilities being utilised in a strategic, unhelpful fashion.

– Changing minds:  When we frame outcomes in terms of probabilities, we allow ourselves the freedom to alter our view.  Whilst this should be considered positive and a reflection of realistic pragmatism; it is more often stigmatised as ‘sitting on the fence’ or providing a pre-prepared excuse for ‘being wrong’.

None of these arguments are particularly compelling and they are outweighed by the manifold benefits of thinking explicitly in terms of probabilities when making investment decisions:

– Humility:  Simply by framing potential outcomes in terms of probabilities, we immediately acknowledge the uncertainty and unpredictability of any given judgement. We are divorcing ourselves from hubristic, narrow forecasts from the outset.

– Reduce confirmation and commitment bias: When we espouse high conviction and overly precise views we come to be defined by them. Instead of refining or rejecting our beliefs in light of contrary evidence, we instead seek to preserve our ego by becoming increasingly committed and ignoring countervailing information.  By ascribing probabilities to a range of different scenarios we are more likely to remain open-minded and diminish the influence of being committed to any particular outcome.

– Understand strength of view: It is often difficult to gauge the strength of someone’s view – this is particularly the case for binary situations where a ‘yes’ decision could either represent a marginal call or be a display of resounding confidence.  For example, if we had to predict the result of two coin tosses one, which was biased 55% in favour or tails and the other 95% towards tails – our ultimate view on the most likely outcome should be the same – but the level of confidence wildly different. Given that the investment industry tends to favour those making binary, conviction calls; such nuances are often lost and marked uncertainty can exit beneath a veneer of confidence – employing probabilities can help to reveal this.

– Consider alternative scenarios: The use of probabilities forces us to consider alternative (often negative) scenarios; an individual with a high level of confidence in a specific outcome is still likely to ascribe some probability to other less favourable results.  Even if the chance of these occurring is regarded as minimal, there is an undoubted benefit from considering them, rather than ignoring them entirely and focusing solely on the primary case.  The best example of this would be in assigning a probability of a bullish case for a particular stock – the simple act of not giving the positive scenario 100% likelihood, compels consideration of other potential outcomes, and hopefully encourages debate.

– Incorporate new information: A crucial element of successful decision making is the ability and willingness to revise views in the light of new information. If we make binary decisions (such as buy or sell) we can easily avoid incorporating fresh evidence by claiming our view has not altered (“it is still a buy”); however, by specifying probabilities it becomes increasingly difficult to ignore developments that are likely to impact your level of confidence.

For example, imagine you buy an active mutual fund and hold a 70% confidence that it will outperform its benchmark over three years; after one year the fund has generated significant excess returns – do you revise your view? If it is a simple, vague buy decision you are likely to continue to hold; however, if you have to update your confidence level there is likely to be an impact. Given the evidence of mean reversion in active manager returns, can you really still can you really continue to ascribe the same probability of outperformance over the next three years, or should it be revised lower?  Being explicit with probabilities forces us, and others, to challenge and update our thinking.

– Evaluate past decisions: Given the scourge of hindsight bias it is often impossible to assess the quality of our historic decisions, as we cannot accurately recall our thought processes and feelings at the particular time. If, however, we consistently review and document our decision / forecast confidence level, we are in a far better position to understand how we came to a particular viewpoint and the manner in which we reacted to new information.  This can become a vital tool in learning from past behaviours.

Ascribing likelihoods to potential outcomes is not easy, as with all our behaviours it will be blighted by noise and bias; however, even if we are not openly expressing opinion in probabilistic terms, they are still deeply embedded in the views we hold.  As highlighted in Philip Tetlock’s Superforecasters and Annie Duke’s Thinking in Bets, being explicit about probabilities when making judgements or forecasts allows us to embrace uncertainty, affords us the freedom to revise our opinion as new information arrives and fosters the ability to learn from previous decisions.  In complex and ambiguous financial markets such features are invaluable.

Evidence on the effectiveness of momentum investing is overwhelming – it has delivered a return premium across markets and asset classes, and there is over two centuries’ worth of data (see Asness et al. 2014).  Whilst the performance advantage delivered is difficult to refute, it does present something of a puzzle – why should a strategy that is both technically simple to apply and behaviourally attractive deliver excess returns? Before exploring this question, it is worth clarifying these two claims:

Momentum is Easy:  A standard time series, price momentum strategy is relatively easy to construct and operate. Although there are a vast range of iterations and nuances (this is the investment industry, after all) – the basic premise is to go long assets rising in value, and short those falling. The technical barriers to employing this strategy in some form are limited.

Momentum is Behaviourally Attractive: Most of us are wired to be momentum investors. Being part of the trend plays to our desire to be ‘right’, to be a member of the herd and to conform to the compelling narratives that price momentum inevitably generates.  Participating in a momentum trade is psychologically comfortable.

Yet whilst most of us are momentum investors of some variety (often indirectly), rather than obtain the documented return premium – we help to create it.  Our behavioural limitations mean that our attempts at discretionary momentum investing (driven by human decision making) are deeply flawed and incur a cost, which can be exploited by systematic approaches.  It is important to remember that the evidence upon which the case for excess returns to momentum is built necessarily relates to systematic / rules based strategies.

In his book ‘Following the Trend’ Andreas Clenow described a traditional managed futures approach – a basic trend following strategy – as such:

“A statistical game with a slight tilt in your favour and that you just have to keep throwing the dice long enough to get the law of big numbers on your side”

This description bodes ill for the ability of humans to capture a momentum return.  Victor Haghani and Richard Dewey produced a study where financially literate individuals (including some investment professionals) had to bet on the result of a coin flip, which was loaded in their favour 60/40. They started with $25, could bet stakes as small as $0.01 and were given 30 minutes. Despite this favourable setup, only 21% reached the maximum payout of $250 and 28% went bust. Even more curiously, 48% of players bet on tails (which had the 40% probability) more than 5 times. If our behaviour can be so irrational and erratic in such a dispassionate setting, it is no wonder that investors make consistently poor decisions in stimulating and emotion-laden financial markets.

To understand how momentum investing driven by human judgement serves to create a fecund environment for rules based momentum approaches, it is helpful to consider the central tenets of a typical systematic process and understand how this compares to the features of discretionary (or human) decisions:

Momentum	Systematic	Discretionary
Winners Purchased	Earlier	Later
Winners Sold	Later	Earlier
Losers Sold	Earlier	Later
Diversification	High	Low
Rules	Clear	Vague
Persistence	High	Low

The characteristic approach of a discretionary, ‘human’ investor described above is driven and shaped by a range of behavioural factors:

Buying Winners Late:  Investors tend to underreact to news that fundamentally alters the value of a security.  Whilst this phenomenon may be caused simply by the pace of information dissemination; it is more likely a result of commitment bias (our reticence to recant prior views) and also the gradual development of a new narrative – one piece of newsflow or data is unlikely to shift the prevailing market story, but if this persists the story surrounding it will build strength, drawing in investors.  Also, as highlighted by Mark Granovetter (1978), we all have a different threshold for joining the riot (or herd) simply based on how many other people are participating. Momentum begets momentum.

Selling Winners Early:  As detailed by Shefrin (2010), the disposition effect – the tendency to cut winners and run losers – has a material impact on asset prices and the momentum effect.  In the case of relinquishing winning positions, investor willingness to sell on good news (and capture gains) serves to slow the adjustment of an asset price to its new fundamental value, thereby helping to foster momentum.

Selling Losers Late:  Whilst arriving conspicuously late to the party, discretionary investors often overstay their welcome in loss making positions.  Investors are committed to their viewpoint and suffer from confirmation bias, so may simply ignore new information that runs contrary to the view implicit in their positioning. This situation is exacerbated by cognitive dissonance, which makes them unwilling to crystallise losses and acknowledge error.

Lack of Diversification: Spreading risk across a variety of positions and asset classes is crucial to all sound investment approaches, but is particularly relevant (essential) for momentum-driven investing.  The hit rate for individual positons can be low and predicting where momentum will arise (and be sustained) before the event is difficult. There is also the constant threat of sharp reversals, which can rapidly destroy gains made in previously successful positions.  The danger for discretionary investors is that they become overconfident in their ability to identify the best trades and also overweight the most recent market activity.  This can lead to exceptionally narrow portfolios focused on securities that have the strongest current momentum, rather than holding a diverse range of positions with varying levels of price strength.  Whilst systematic momentum approaches can become concentrated, they consistently maintain a broad opportunity set and (should) have prudent risk controls.

Vague Rules: The best defence against the impulses of emotion or narrative-led decision making is a set of decision rules, which dictate investment behaviour.  Systematic momentum approaches are founded on such principles, and actively divorce decision making from all intrusive elements except the key variable – which is typically price, but can include fundamentals, such as earnings.  Discretionary human decisions are rarely driven by a binding set of rules and, absent strict guidelines, our actions are impacted by a range of factors, most of which have limited relevance to the judgement at hand. These might include: how we feel at the time of making a decision (our emotional state), our prior experience with a particular investment, the most recent news we have read, our belief in a particular narrative, or even how hungry we are (Danziger,  Levav,  & Avnaim-Pesso, 2011), and the weather (Hirshleifer & Shumway, 2003).

Being behaviourally consistent without a set of rules to adhere to is close to impossible; this was highlighted by Daniel Kahnemann (alongside Rosenfield, Gandhi and Blaser) in an article published in Harvard Business Review in 2016.  The authors discussed how the influence of irrelevant factors can lead to huge variability in decision making; unlike biases, which tend to be consistent and persistent, the impact of noise is random and erratic – and therefore more difficult to mitigate.  Even with the best intentions, the ability of a human decision maker to remain disciplined is severely limited.

Low Persistence: Perseverance is undoubtedly a key requirement for successfully capturing the momentum premium, and its return patterns make this an exacting challenge. The hit rate is unlikely to be high – therefore you will be ‘wrong’ frequently; furthermore, there will be many false dawns where momentum appears and rapidly evaporates.  There will also be prolonged fallow periods in choppy markets where returns to momentum are poor, and short-term shifts in markets that see prevailing trends whipsaw and sharp losses incurred.  Whilst a systematic, rules based strategy can be agnostic on such a performance profile (and indeed specifically designed to withstand it); the foibles of momentum are hugely problematic for discretionary investors, for whom it would be a herculean task to remain sufficiently controlled amidst the welter of behavioural impediments.

Whilst most of us are invariably attracted to momentum investing and carry it out in some form (even if it is merely implicit in our actions, rather than an express choice), few of us do it well.  Our decision making is blighted by our behavioural shortcomings and the substantial influence of irrelevant ‘noise’.  Human decision makers are willing but inferior momentum investors, creating the opportunity for systematic approaches to capture a premium.

—

Key reading:   

Asness, C. S., Frazzini, A., Israel, R., & Moskowitz, T. J. (2014). Fact, fiction and momentum investing.

Clenow, A. F. (2012). Following the trend: diversified managed futures trading. John Wiley & Sons.

Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011). Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences, 108(17), 6889-6892.

Granovetter, M. (1978). Threshold models of collective behavior. American journal of sociology, 83(6), 1420-1443.

Haghani, V., & Dewey, R. (2016). Rational Decision-Making Under Uncertainty: Observed Betting Patterns on a Biased Coin.

Hirshleifer, D., & Shumway, T. (2003). Good day sunshine: Stock returns and the weather. The Journal of Finance, 58(3), 1009-1032.

Hurst, B., Ooi, Y. H., & Pedersen, L. H. (2017). A century of evidence on trend-following investing.

Kahneman, D., Rosenfield, A. M., Gandhi, L., & Blaser, T. (2016). Noise: How to overcome the high, hidden cost of inconsistent decision making. Harvard business review, 94(10), 38-46.

Shefrin, H. (2010). How the disposition effect and momentum impact investment professionals.

Heuristics, or what we might call ‘rules of thumb’, have become somewhat maligned as a method of decision making in recent years. They are often erroneously conflated with cognitive biases and suffer their negative connotations.  Furthermore, the scepticism regarding the effectiveness of heuristics has been exacerbated by the ubiquitous system one (fast) / system two (slow) thinking construct (popularised by Daniel Kahneman), which can frame system one as the domain of rash, unthinking errors and system two the home of cool, rational and correct calculation.  System one and system two is frequently used as shorthand for ‘bad’ and ‘good’ thinking respectively, with the mental shortcuts that define heuristics firmly in the former grouping.

Even without exploring the limitations of the ‘two systems’ theory (see Osman, 2004); it is clear that heuristics do not fit within such a binary framework.  They can be either instinctive or deliberate, and often deliver results that are more robust than more complex methods of judgement.  For example, see Serwe and Frings (2006) on predicting the result of Wimbledon tennis matches, and McCammon and Hageli (2007) on avalanche risk.  Andrew Haldane’s speech around this topic is also excellent.

Unsurprisingly, the best definition of heuristics comes from Gerd Gigerenzer, who has been the pre-eminent voice on this subject, and the most vocal critic of the work of Kahneman.  He defines the heuristic approach as such:

“A strategy that ignores part of the information, with the goal of making decisions more quickly, frugally, and/or accurately than more complex methods” (Gigerenzer & Gaissmaier, 2011).

Although heuristics do feature in the world of investment there is an undoubted preference for complexity.  This is understandable – financial markets are intricate, unpredictable and often unfathomable; therefore we struggle to believe that simple solutions can be effective – a complex problem must be untangled by a complex process.  Furthermore, it is difficult to justify fees (and our own existence in the industry) by recommending quick and easy solutions.

This is not to suggest that everything in investment can be improved through simplification; rather we should not pour scorn on approaches that seek to streamline decision making in the midst of high uncertainty or difficulty. Sensible heuristics should be viewed as a valuable tool for investors, and below I have suggested five simple rules.

– Jam Yesterday Heuristic:  If an investment has produced strong returns in the recent past, reduce your expectations for future performance.

Whilst this is self-explanatory, it often seems to run contrary to the behaviour of most investors. The gains registered by an investment before you decided to purchase, are returns you will not be enjoying in the future!

– Upside Down Heuristic:  If an investment has outperformed its benchmark by a significant margin on the upside, it can also do so on the downside by a similar magnitude. If this is not acceptable, do not invest.

The lure of funds that have generated material excess returns can be irresistible, but if they have taken sufficient active risk for their performance to diverge markedly on the upside, they could generate similarly divergent returns in a less favourable fashion. Whilst index +20% is attractive over three years, only invest if you are willing to withstand the reverse 

– Keep it Simple Heuristic:  If presented with similar investment choices that differ in terms of complexity, select the simpler option.  

Simple investments are certainly no panacea, nor should they always be the preferred choice.  However, in situations where it is difficult to differentiate between alternatives in terms of quality, the preference should be towards that which is most easily understood.  Having clear and well-calibrated expectations is crucial for any investor.  If an investment strategy is straightforward, it is less likely to deliver negative surprises and you are more likely to stay invested for the long-term.

– Hedge Your Bets Heuristic: If you have any uncertainty over the timing of an investment, phase it into the market in equal tranches.

Phasing investment decisions is an effective behavioural trick, not only does it reduce the potential reference point impact of a single price on entry or exit; it also allows for the positive framing of your decision. If the price of your targeted asset rises as you stagger your investment you can feel content that you initiated the purchase when you did, whereas if it falls you can be glad that you decided not to invest in one hit.

– Double for Drawdown Heuristic:  Your absolute minimum expectation for drawdown in any diversified investment (outside of cash) should be double its expected long-term volatility.  If this is not palatable, do no further work.

This rule is not a comment on the appropriateness on volatility as a measure of risk, nor the expected distribution of asset class returns (which are often skewed); rather it seeks to translate the most commonly employed investment risk metric (volatility) into a simple yes / no decision based on a willingness to bear (temporary) losses.  Of course, volatility is a woefully inadequate measure of risk for illiquid investments with artificially low volatility, and both undiversified or ‘penny in front of steamroller’ strategies, which suffer from sharp tail risks.

None of these heuristics is likely to be revelatory, but amidst the myriad options that investors face as they seek to meet their financial objectives, it is easy to lose sight of crucial principles that can lead to improved investment behaviour and better long-term outcomes.  Although lacking the intellectual cachet of more sophisticated approaches; easy to remember, simple to apply rules of thumb can be remarkably effective way to navigate a complex investment landscape.

—

Key Reading:

Gigerenzer, G., Todd, P. M., & ABC Research Group, T. (1999). Simple heuristics that make us smart. Oxford University Press.

Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual review of psychology, 62, 451-482.

McCammon, I., & Hägeli, P. (2007). An evaluation of rule-based decision tools for travel in avalanche terrain. Cold Regions Science and Technology, 47(1-2), 193-206.

Osman, M. (2004). An evaluation of dual-process theories of reasoning. Psychonomic bulletin & review, 11(6), 988-1010.

Serwe, S., & Frings, C. (2006). Who will win Wimbledon? The recognition heuristic in predicting sports events. Journal of Behavioral Decision Making, 19(4), 321-332.