invest(igations)

How to Be a Great Investor, Part Five: Think Probabilistically

This article is the fifth in a ten-part series loosely based on Michael J. Mauboussin’s white paper “Thirty Years: Reflections on the Ten Attributes of Great Investors.” See “Part One: Be Numerate,” “Part Two: Understand Value,” “Part Three: Properly Assess Strategy,” and “Part Four: Compare Effectively” for previous installments. And please keep in mind that although I’m basing my work on Mauboussin’s, I am departing from his ideas on occasion.

Thinking Probabilistically

Whenever you come up with a new investment idea—whether it’s a new security to buy, a new factor to consider, or a new strategy to implement—you naturally ask yourself whether this new idea will increase your portfolio returns or cause you to lose money (and, of course, how much). Thinking probabilistically involves assessing the probabilities and coming up with a reasoned answer. In this article I'll tackle strategies and factors first before dealing with individual securities.

Assessing the Odds of a New Factor or Strategy

If you’re assessing the new factor or strategy on the basis of past performance, this involves three steps: assessing how well it would have worked in the past; assessing how correlative past results are to future (out-of-sample) results in general; and assessing whether the new factor or strategy will have better odds than the factor or strategy it is replacing (assuming you have to take the money from one in order to implement another).

If, on the other hand, you think that past performance is irrelevant to assessing future success, or that the correlation between past performance and future performance is unknowable, negative, or minimal, then you have to come up with another way of assessing future performance. But even in this case, you should also factor in your estimate of the correlation between your ideas and their future performance.

So let’s take a concrete example. Let’s say I want to replace a microcap-only strategy with a similar strategy that invests in stocks of all sizes, and that I believe that backtesting its past performance will give me some relevant results.

The first step is to test using robust methods. Test both systems on subsets of the universes you’re going to be investing in, and on subsets of the time period you’re backtesting. Try to make the tests as similar as possible to each other.

So, for this example, I’m going to backtest my strategies using Portfolio123. I’m going to make the two strategies as similar to each other as possible except that one will emphasize microcaps and the other won’t. In addition, I’ll program my slippage costs for the microcap model to be more than double my costs for the other.

It turns out that in 14 out of 16 tests, my new strategy (all caps) gives worse results than my old one (microcaps). That gives me only a 12.5% chance that my new strategy is an improvement.

Now I estimate that the correlation between past performance and out-of-sample performance of stock strategies is around 0.2 (using Kendall’s tau; 0.3 using Pearson’s r). To get at that number I took fifty strategies that I had not backtested that were loosely based on screening rules that were written by others. I did so in order to minimize the chance of using factors that I had already developed or tested, since those might contaminate the experiment. I tested and compared these strategies’ performance over various time periods.

This 0.2 correlation translates to a 40% chance that the past relative performance of any two strategies will reverse. (Here’s the math, if you’re interested; if not skip to the next paragraph. Kendall’s tau is calculated by looking at every possible pair in the two series and classifying them as concordant or discordant, depending on whether they’re in the same order or not. You then take the number of concordant pairs, subtract the number of discordant pairs, and divide by the total number of pairs. For example, let’s calculate the correlation between 1 2 3 4 5 and 1 3 5 2 4. Each series has ten pairs. 1 2 is concordant, 1 3 is concordant, 1 4 is concordant, 1 5 is concordant, 2 3 is discordant, 2 4 is concordant, 2 5 is discordant, 3 4 is concordant, 3 5 is concordant, and 4 5 is discordant. The correlation is thus (7 – 3)/10 = 0.4, and the chance that the relative position of a pair will reverse—a discordant pair, in other words—is three out of ten, or 30%. If the correlation had been 0.2, that would have been the result of (6 – 4)/10, and the chance of a discordant pair would be 40%.)

To calculate the chance of an event occurring with a certain level of confidence (or a certain correlation), you multiply the chance that the event will occur by the chance that you’re right, multiply the chance that it won’t occur by the chance that you’re wrong, and add them up. (The weird corollary is that if you’re right precisely half the time—a correlation or confidence level of zero—the probability of an event occurring will always be 50%, no matter how likely or unlikely it is.)

The probability that my new system will work better than my old one, then, is actually 42.5%. (The formula is 12.5%*60%+87.5%*40%.) This is still not high enough to bank on . . .

Now let’s say you consider past performance a completely unreliable measure of the success of a strategy. And let’s say you think your new strategy has a 70% chance of performing better than your old strategy. Well, the other number to take into account is the correlation between your predictions and what actually happens in the future.

In other words, how good a strategy designer are you? Let’s say you design ten strategies, and you judge one of those to be better than another. What is the chance that if you invest in both those strategies you’ll be right? Try to be as objective as you can here.

So let’s say that you think that this chance is about 60% (again, this represents a correlation of 0.2). Do the math again (70%*60%+30%*40%). It turns out you have a 54% chance of outperforming, not a 70% chance. So you might want to balance your investment between the two strategies, or combine them into one somehow.

So far, we’ve been discussing new strategies; but we can apply the same line of thinking to new factors. With factors, we have additional tools at our disposal because we can combine them using ranking and weighting. (We can combine strategies too, of course, but it’s not as easy to do.)

There are lots of ways to assess the effectiveness of a factor. The most traditional is to rank stocks in terms of the factor and see how well each quantile performed in the past. Many scholars simulate shorting the bottom quintile or decile and going long the top quintile or decile, but with some factors the middle quantiles may have performed the best.

Let’s look, once again, at a concrete example. I have a four-factor ranking system and I’m considering adding a fifth factor to it. The decile returns for my four-factor system over the universe of stocks that I invest in, with a one-month holding period, over the last ten years, looks like this:

If I add my new factor, I get a chart like this:

Clearly, the second system, with five factors, works better than the first.

But if you look at probability, there’s a problem with this way of thinking. You could keep adding factors and varying their weights ad infinitum until you had a system that was perfectly optimized according to the tests that you run on it. Your system will look something like this:

Or, taking a more granular view with thirty quantiles rather than ten:

Your system will be optimized precisely for the time period that you are testing it for.

What is the probability that a system optimized for a specific period of time will outperform a non-optimized system in a different time period?

I’m not sure. But I want to offer an analogy.

Let’s say you’re designing a robot to play poker. In order to do so, you want the robot to play against real poker players so that it can learn from practicing the game. So you collect the five best poker players in your hometown and the robot plays poker with them. It soon gets so good that it can beat them at any game anytime. Now what would happen if you sat it down with five different poker players? It would probably fail miserably. Why? Because it would have learned how to beat the first five players based on their reactions to the cards and to each other, on their tells and on their strategies. On the other hand, if you had simply taught the robot the rules of playing poker and had fed it a variety of different examples and made it think about things more abstractly, it probably wouldn’t have won nearly as handily against the first five players, but would probably have performed better against the second five.

The analogy is imperfect, I admit. Poker doesn’t change as much as the stock market. A stock-market strategy or factor that worked really well in the 1960s may not have as much relevance in the 2020s as one that worked well in the 2010s.

At any rate, I recently ran an experiment. Remember those fifty strategies I told you about earlier? I took the ones that had performed best over a certain time period and tested them over a subsequent time period. Some of them did well, others did not. When I combined the ones that had done best in the first period into one comprehensive system, it did quite well in the second—better than most of those that had performed best in the first. When I optimized the one that had done best in the first period, fiddling with the factor weights until its performance was greatly improved, it didn’t do as well as the combined system in the second period. When I optimized the combined system to improve its performance in the first period, its performance in the second period got worse. Now this is a very small and limited experiment, and my results are hardly representative. It would take me many months to optimize dozens of systems over different time periods and test them all. But this does indicate that optimization may not be the best idea.

The other reason to be wary of optimization is the fact that with low correlations, what outperforms in one period is likely to suffer from mean regression and underperform in another period. In my previous article, “How to Bet,” I published the following correlation table.

Let’s say seven players play a tournament and player 1 wins, player 2 comes in second, and so on. The top row lists Kendall’s correlation; the other rows list the chance of each player winning in a second tournament. (I created the table by taking every possible order of seven players and grouping them according to their correlation with the first order.) You’ll see that with a correlation of 0.333 or higher, the top player has the best chance of winning; with a correlation of 0.238, the top two players have an equal chance of winning; with a correlation of 0.143, the top player has a lower chance of winning than the second- and third-place winners; and with a correlation of 0.048, the top player has an equal chance of winning as the sixth player, with the second, third, fourth, and fifth all having a better chance. This is due to the fact that absent a perfect correlation, there will always be some element of mean regression.

Assessing the Odds of a Security’s Future Price Increase

Let me start out by saying forthrightly that I believe that there is no way to do this. If there were, stock picking would be simple and profitable. And it’s definitely not. Instead, your best bet is to think about your strategy’s odds rather than the odds of an individual stock or ETF.

Mauboussin has a few pointed words about this. “When probability plays a large role in outcomes, it makes sense to focus on the process of making decisions rather than the outcome alone. The reason is that a particular outcome may not be indicative of the quality of the decision. Good decisions sometimes result in bad outcomes and bad decisions lead to good outcomes. Over the long haul, however, good decisions portend favorable outcomes even if you will be wrong from time to time. . . . Learning to focus on process and accept the periodic and inevitable bad outcomes is crucial.”

Allow me to add a few words about stock-return odds in general.

I recently took tens of thousands of random samples of 1-year returns of stocks over the last twenty years (with a minimum market cap of $50 million and a minimum price of $1.00, and with no survivor bias). For such stocks, here are the decile median returns:

In other words, a randomly chosen stock out of this group will have a ten percent chance of a 93% gain, a ten percent chance of a 78% loss, and a ten percent chance of each of the other numbers in-between.

Here is a graph of the returns distribution (the rightmost bar represents stocks with a return greater than 250%).

The median stock gets a 5.35% return and the average stock gets an 8.12% return. If you were to randomly choose twenty stocks a year and then do so again and again thousands of times, your compounded annualized return would likely be between 3.9% and 8.7%. On the other hand, if you were to randomly choose just one stock a year and then do so again and again thousands of times, your compounded annualized return would likely be a 100% loss, because at some point you’re going to choose a stock that goes bankrupt. (This is the key to understanding why some small amount of diversification is essential: you need to minimize the probability of losing all your money. Because of this risk, I would advise you to never put all your money into fewer than four stocks.)

The best you can hope for when you choose a stock is a returns distribution that is somewhat better than that of the average stock. Some writers, when they talk about choosing stocks, discuss “expected return.” But doing so without considering the entirety of the returns distribution can lead to gross errors such as the one Harry Markowitz made at the outset of his seminal 1952 paper “Portfolio Selection,” the foundation of Modern Portfolio Theory, in which he incorrectly “proved” that to maximize the expected return of a portfolio you should put all your money into the one security with the highest expected return. Markowitz ignored the laws of probability, as does MPT in general.

But that will be the subject of a future article . . .

My CAGR since 1/1/16: 32%.

My top ten holdings right now: KBAL, TTEC, LAWS, CURO, NRC, ELMD, CO, ACU, LIVE, ABBV.

How to Bet: Correlation, Mean Regression, Expected Returns, and Position Sizing

Understanding the rules of probability—how to bet—can be a valuable asset for investors. Every day I come across investors who think that regression to the mean has nothing to do with correlation, who don’t bother to think about odds, and who take wild stabs at position sizing. Until relatively recently, I was such an investor myself. In this article I’m going to attempt to impart my understanding of these things and give some general hints as to how they can be applied to investing strategy; in my next article, which is part of my How to Be a Great Investor series, I’m going to apply my conclusions specifically to the art of investing.

A word of warning: I am approaching this problem very differently from the traditional approach, which builds on Harry Markowitz’s seminal 1952 paper “Portfolio Selection.” Markowitz began his discussion with a rejection of the rule that an investor should maximize expected returns. I don’t reject that rule at all. Markowitz proves mathematically that if we accept this rule, then “the investor places all his funds in the security with the greatest discounted value” and does not diversify at all. But his mathematical proof looks only at the expected return and not at the range of possible returns. Another way to put it is that his proof is perfectly valid for a small, one-time investment, but cannot apply to compounded and repeated large investments, for when every investment carries the risk, however small, of a 100% loss, the investor should never place “all his funds in the security with the greatest discounted value.” Following his rejection of the maximize-expected-returns hypothesis, Markowitz concludes that an alternative rule must apply: that the investor should “consider expected return a desirable thing and variance of return an undesirable thing,” and the bulk of his paper is concerned with finding an efficient balance between those two. I come at the subject from an entirely different perspective, since I view risk as the probability of permanent loss and view variance of returns as the natural result of low correlation between expected and actual returns.

A Hypothetical Problem

There are seven poker players. They just completed a tournament in Las Vegas, and we know in which order they were eliminated. Now, a week later, they’re about to play a different tournament in Atlantic City, and we are trying to predict which of them will outlast the others. Besides the order of elimination in Las Vegas, the only other fact we know is that in similar situations, there is usually a modest correlation between the positions of the players in the first tournament and those in the second. (Correlation, in this context, means sequential similarity. A perfect rank correlation is one in which the values are in the same order of size. A perfectly negative rank correlation is one in which the values are in the opposite order of size.)

You are required to bet on which poker player will win. You can make one and only one bet.

You have two conflicting impulses. On the one hand, obviously, some players are, in general, better than others, and this is proven by the modest correlation. Poker is not entirely a game of luck. Therefore, you should place your bets on the winner of the last tournament. On the other hand, obviously, there is some regression to the mean. A player that does extremely well in one tournament will probably not play quite as well in a second. Therefore, you should avoid placing your bets on the winner of the first tournament and instead place your bets on one who performed closer to average. 

Which is the best strategy for placing your bets?

The Solution

We have been told that there is usually some correlation between the positions of the players in the first tournament and those in the second, and we know this from the results of previous poker tournaments.

Regression to the mean is the precise opposite of positive correlation. Think of it this way. If the poker players were eliminated in exactly the same order in Las Vegas and Atlantic City, the correlation would be perfect (1.00) and there would be no regression at all. And if the poker players were eliminated in exactly the opposite order, regression to the mean would be perfect (the worst would do best and the best would do worst) and there would be a perfectly negative correlation (–1.00).

If we were to maximize our potential for winning, should we place our bets on the winner of the Las Vegas tournament, the runner-up, the third-place winner, or someone else? After all, with an imperfect correlation (less than 1.00), there is some regression going on, right?

Let’s say you were to generate an actual permutation table (and these can be absolutely huge: the number of possible permutations of ten players is 10!, or 3,628,800), measure the correlations of each permutation to the original order, and look at who has the best chance with each correlation to win. Here's the correlation table for seven players, with the most likely winners in bold (click on it to enlarge):

There are two surprising things about this result. First, a known correlation of zero is fundamentally different from an unknown correlation. If the correlation is unknown—you have no idea if it’s positive or negative—then every player has an equal chance of winning. But if the correlation is zero, then middling players are your best bets. Second, one might have thought that with a very small but positive correlation, the top player would remain the best bet; but that’s not actually the case.

The correlation table shows that you should place your bets on the top player when the correlation is high (above, say, 0.25), the worst player when the correlation is low (below, say, -0.25), the middle player when the correlation is 0, and some in-between player when the correlation is between 0.25 and -0.25.

For those who want to get into the weeds of measuring correlations, I’m now going to explain the mathematics of Kendall’s tau; feel free to skip this paragraph if you’re not interested. I use Kendall’s tau as my correlation measure because it’s less susceptible to outliers, easier to interpret, and more intuitive than the other standard measures of correlation, Pearson’s r and Spearman’s rho; in addition, with Kendall’s tau, you don’t have to deal with covariance or standard deviations. Take a pair of data points in the two series and classify it as concordant or discordant depending on whether the points are in the same order or the opposite order in the two series. Kendall’s tau is the number of concordant pairs minus the number of discordant pairs, divided by the number of all pairs. It ranges from 1 for a perfect correlation—all the pairs are in the same order—to -1 for a perfectly negative correlation—all the pairs are in the exact opposite order. If Kendall’s tau is 0.2, then there is a difference of one-fifth between the number of concordant pairs and the number of discordant pairs, and the only way you can split a whole into two parts with a one-fifth difference is three parts to two parts, so the odds of one player beating the other if she beat her in the first tournament is three to two. Similarly, if Kendall’s tau is 0.333, her odds will be two to one, and if Kendall’s tau is 0.5, her odds will be three to one. Obviously, if you know that one player was in first place and the other was in last place, the odds will be higher, and if you know that they were in third and fourth place, the odds will be lower. Kendall’s tau tends to be significantly closer to zero in most cases than Pearson’s r or Spearman’s rho. A Kendall’s tau of 0.25 might be a Pearson’s r of 0.35, for example.

Applying Correlation to Investment Strategies

In the above example, we’re talking about the correlation of the performance of players between different tournaments. We can now apply this lesson to the performance of investment strategies between different time periods or the correlation between expected and actual returns.

Let’s look at time periods first. The natural impulse of investors is to “chase performance.” Investors want to put their money into strategies that have performed well in the recent past. This makes perfect sense if the correlation between past performance and future performance is high. It makes no sense at all if the correlation is negative or close to zero.

So it’s worthwhile investigating exactly what that correlation might be. And that depends on two things: how different the strategies are and what time periods you’re looking at.

In my own experimentation, I have found that longer time periods are generally more correlative when it comes to performance. The most correlative look-back period for future three-year performance seems to be the previous ten to twelve years; I would guess that the most correlative look-back period for future ten-year performance would be the previous thirty to forty years (though I have not done the testing to establish this). Correlation is probably lowest when looking at a short period—five years or less—which is precisely what performance chasers usually look at.

If your investment strategies are limited to long-only equities, the performance correlation is going to be quite low—in the range of 0 to 0.3. Using alpha as a performance measure improves the correlation a little bit. If the strategies include fixed-income and long-short strategies, the correlation will likely be higher. If they’re limited to very diversified large-cap equity strategies, the correlation may well be negative. Actively managed mutual funds, for example, have a slightly negative correlation between performance over long periods.

One doesn’t have to look at past performance at all, though, in order to apply the lessons of correlation. One can use correlation as a proxy for confidence. Let’s say you can think of seven different investment strategies and can assign an expected return to each one based purely on your theory of how they will perform. Don’t stop there. Consider this: if someone you knew, with precisely your experience and depth of thought, were to assign expected returns to seven strategies, what do you think the correlation between present expected returns and future performance will be? Use that number as your correlation estimate.

My advice: With low correlation, spread your bets. Don’t chase short-term performance. Don’t optimize your strategy over a specific period. Don’t expect your estimation of expected returns to correlate perfectly with future performance, and take action accordingly.

This advice goes against my own impulses—and my own past practices. But I’m working to change those.

Spreading Your Bets

Let’s say you had the opportunity to bet $100 and you could put as much of that money on any player as you wanted, and let’s say the typical correlation between tournaments is 0.33. Would it be better to bet the entire $100 on the first-place winner of the Las Vegas tournament, or to spread some of your money around on the others?

That depends entirely on whether this is a one-time bet or a habitual bet, whether you’re making small bets or putting all your money on the line, and whether the payout is all-or-nothing or graduated.

Let’s look at the one-time, small-bet, all-or-nothing scenario first. (This is essentially the scenario that Markowitz examined at the outset of his seminal paper.)

In order to place bets wisely, it’s important to know what the expected return of each bet is. There are many ways to estimate odds, and one of them is to look at past performance. It is also, of course, important to take into account the payout schedule. For an all-or-nothing bet, the correlation table can be easily converted into a table of expected returns by taking each value, multiplying it by the number of players, and subtracting 1. Notice that the sum of each column is 0.

So if it’s an all-or-nothing bet and the correlation is 0.33, your expected return if you put all your money on the number-one player is 97%, and your expected return if you put all your money on the number-seven player is -90%. If you put a third of your money on the number-one, number-two, and number-three players, your expected return will be 70%. There is basically no way to spread your bets so that you can have a higher expected return than putting all your money on the number-one player.

However, if you are making a practice of placing bets like these and the bets are large (i.e., after each bet you put your entire returns on the line), diversifying and compounding will make a huge difference. In that case, you want to maximize the chance of not losing the entirety of your stake by spreading your bets proportional to your expected return.

Let’s say you spread your bets evenly, putting one-seventh of your money on each position. Because the total payout is just as much as the amount you bet, you’ll break even every time you bet.

Let’s say you put one-fifth of your stake on the first five positions and nothing on the others. Then every time one of the two losers wins, you’ll lose 100% of your stake.

But if you place your bets proportionally to your odds—if you spread your $100 so that you put $28 on the first position and only $1 on the seventh, and divide your next bets in the same proportion—you’ll do extremely well. If the odds hold out, you’ll average 30% per bet. (The actual optimal bet size by position is $28, $25, $20, $14, $8, $4, and $1.)

Now all of the above applies only to all-or-nothing bets. With an all-or-nothing payout, you want to spread your bets proportionally to the expected return (in other words, add 100% to each expected return and divide by the number of positions) to maximize your take. The mathematical demonstration of this is pretty nifty, if you’re into that kind of thing.

What About Lower-Risk Payouts?

The above scenario is inappropriate for an activity like investing, where the payouts are rarely such high-risk. The all-or-nothing scenario is like investing in seven companies, one of which will pay off 600%, and the other six of which will go bankrupt, and then doing it again and again. High-risk payouts are a good starting point for computation of probability, correlation, and position sizing. So let’s take what we’ve learned and apply the lessons to lower-risk payouts.

If the payout is graduated, as it is with investing, then the calculation of expected return is far more complicated, as you’d have to take into account not just the chance of each player winning, but also each player placing second or third or fourth, etc. So, for example, let’s take the following payout schedule. If you pick the #1 winner, you get a 40% return on your bet; if you pick the #2 winner, you get a 30% return; #3 pays 20%; #4 pays 10%; #5 and #6 pay nothing; and if you bet on #7, you lose 100% of your money. Now the odds are much more similar to what you might get by investing in seven different stocks with a one-out-of-seven chance that they’ll go bankrupt.

Let’s say the estimated correlation between past and future performance (or expected and actual returns) is 0.238 (per the above table). Then the expected returns on the seven players are as follows:

Now, if your bets are long only, your optimal bets would be around 40% on player 1, 27% on player 2, 18% on player 3, 8% on player 4, 5% on player 5, 1% on player 6, and nothing on player 7. (The calculation of these optimal bet sizes can no longer be expressed with a simple formula, but is extremely complicated.)

Notice that you’re not placing all your money on the past winners, which is what we have a tendency to do. But you’re also not spreading your bets as thinly as you would if you were risking all your money on six out of the seven players. Now there’s a large difference between the expected return and the optimal position sizing.

The scenario changes significantly if you’re allowed to use unlimited margin and go short. In that case (assuming no interest charges), the optimal bet sizes are very approximately 150%, 120%, 105%, 50%, -50%, -65%, and -190%.

Now what if you’re a very conservative investor and would prefer to keep half your money in reserve? (This situation is equivalent to judging that none of the stocks you invest in will ever go bankrupt and that your downside is limited to a 50% loss.) In that case, your bet sizes need to double (proportionally, that is) in order to make the same return. Take the original optimal bet sizes, multiply by the number of positions, and subtract 100%. Now double everything. Then reverse the process, adding 100% and then dividing by the number of positions. If you can’t go short, you’ll have no way to replicate the optimal return of the above scenario; if you can, your bet sizes will increase substantially.

Conclusions

Here are, perhaps, the most important general points from this understanding of probability, correlation, and position sizing.

1. Mean regression and correlation are precise opposites.
2. If correlation is high, the winner of one series is going to be the most likely winner of the next. If correlation is well below zero, the loser of one series is going to be the most likely winner of the next. If correlation is between -0.25 and 0.25, middling players are most likely to win.
3. When making small, infrequent bets, always bet on the position most likely to win. When making large, frequent bets, calculate optimal position sizing if you can, based on the expected payout and your estimate of the correlation between past/expected returns and future returns.
4. As your risk is reduced—either because of graduated payouts or because you’re putting less of your money at risk—your position sizing should be more aggressive, even to the point of shorting the positions you have the least confidence in.

All of these points should make intuitive sense. Perhaps it’s good to know that they’re backed by solid mathematical foundations.

In my next article—How to Be a Great Investor, Part Five: Think Probabilistically—I’ll be applying what we’ve learned here to some more practical investing scenarios.

CAGR since 1/1/16: 32%.

My top ten holdings right now: TTEC, RMNI, ABBV, RHHBY, BIIB, CELG, PCOM, JCS, KBAL, BLDR.