invest(igations)

How to Design a Fundamentals-Based Strategy that Really Works, Part Four: Strategy Implementation

This is the fourth and last article in a series; here is the first (on factor design); here is the second (on designing ranking systems); and here is the third (on principles of backtesting).

I’ve been using a fundamentals- and ranking-based strategy for stock picking since 2015, and since then my compound average growth rate is 47%. So I can attest that a fundamentals-based strategy can really work. In this series of articles, I’ve been advocating creating ranking systems as a good alternative or addition to traditional screening. I use a subscription-based website called Portfolio123 for this; ranking can also be done with spreadsheets or other data management tools, but that involves a huge amount of work and is beyond the scope of these articles.

Putting Factors to Work

You’ve created some factors and put them into a ranking system. You’ve done some backtesting and you’re pleased with the results. How do you actually make money?

You must develop a strategy that you can stick with through thick and thin. You can’t just buy the top-ranked stocks willy-nilly and decide when to sell them later. You have to be able to manage your cash, decide your position size, set limits on when to buy and sell, and so on.

Some Behavioral Rules

• Never make spur-of-the-moment decisions. Always follow a system. Modify the system slowly.
• Never panic. Always remind yourself that if something bad happens, something good might happen next.
• Accept volatility.
• Every time you make a modification to your system, keep the old version. You might want to revisit it.
• Only use several systems at once if you feel completely unsure as to which one is better. Otherwise either combine the several systems into one, or go with the better one.
• Don’t make discretionary exceptions to your system. Modify it instead.
• If your main focus is avoiding losses, you’ll never
  • a) make any big gains;
  • b) learn from your mistakes.
• Never use stop losses. Remember that the future price of a stock has absolutely no relationship to what you paid for it.
• Don’t set target prices. Sell a stock when you need the money to buy another whose probability of future gains is higher.
• Often, rebalancing involves adding to losing positions and trimming your winners. Be comfortable with that.
• Don’t worry about things being too complicated. The stock market is intensely complicated; there’s no reason why your strategy shouldn’t be too.
• Don’t follow trends.

Learning from Trading

Every time you make a trade, questions arise. For example:

• A stock appeared on my screen even though it shouldn’t have. Why?
• A stock I bought was obviously a poor investment because of something that the company did that my rules didn’t cover.
• My system is making way too many trades and I keep buying and selling the same stocks.

All of these require investigation and possibly rewriting your portfolio management rules.

Number of Positions

Diversification lowers volatility but can depress returns. A portfolio with too many holdings will be vulnerable to:

• decreased overall returns from holding too many middling stocks,
• management difficulties and too much time spent trading and placing orders, and
• oversight lapses.

A portfolio with too few holdings will be especially vulnerable to

• market impact costs,
• sudden downturns,
• missed opportunities, and
• high turnover.

I've found that the ideal portfolio size is between eight and thirty-two positions unless you're using multiple strategies at once, in which case you go higher.

Position Weighting

It’s best to put more money in positions that inspire your greatest confidence (algorithmically). Mathematically, you’d want to put the most money in the position with the highest expected returns and the least in the position with the lowest. However, if you’re concerned about portfolio volatility, you’ll also want to look at the correlation of the positions and their volatility. And don’t forget to adjust for transaction costs. A stock with a very high expected return might also have serious liquidity issues.

I suggest rebalancing a position if it has radically grown or shrunk or if your estimation of the stock has drastically risen or fallen.

Trading Frequency

Trading daily, two or three times a week, weekly, or monthly is largely a matter of personal choice. Trading daily can increase your returns because you can respond quickly to price changes, earnings reports, and other news; and because it’s easier to ease in and out of positions trading frequently. But trading less frequently can increase your returns because you can drastically lower your transaction costs.

Holding Period

I would not advise you to have a fixed holding period unless you’re absolutely allergic to taxes. Some stocks you might want to hold for only a few weeks, others for more than a year.

But you might want to set yourself a minimum holding period to avoid excess churn.

Entry and Exit Strategies

Do you buy/sell your entire position at once? Or do you ease in and out of positions?

Because factors and prices are unstable, easing in and out of positions over several days may provide you with more stability than simply trusting one day’s output and buying an entire position on that day. In addition, this approach can dramatically lessen market impact.

On the other hand, buying/selling all at once allows you to fully take advantage of sudden price moves and to move quickly on earnings reports.

Hedging

Personally, I don’t hedge. I keep my volatility low by diversifying, buying low-volatility stocks, paying attention to industry trends, and buying a few put options now and then if they’re not too pricey. But if you do want to hedge, try to figure out if you want to use a

• temporary hedge—placing it only under certain market conditions;
• permanent hedge, such as a 5% or 10% short position—this can smooth your returns, but also diminish them; or
• variable hedge, such as using one of the CBOE's volatility indexes to adjust the amount you hedge.

Margin

Margin should be used in small amounts and very carefully. It has to be closely watched. You must understand how margin works, how it turns small losses into large ones and small gains into large ones. It’s also helpful to understand the mathematical relationship between your expected drawdown, your margin debt, the market value of your securities, and your house surplus. And you should try to fathom how Reg T works and the difference between a Reg T and a portfolio margin account.

Margin interest can often be lessened by talking to your broker and threatening to move your account to a different one.

Lastly, I recommend reducing your use of margin when volatility is rising.

Cash

Lots of investors like to keep a reserve of cash on hand in case they encounter major bargains. I don’t. If I encounter a major bargain, I simply sell some other securities in order to buy it. Cash is costly: it’s one of the few asset classes that always loses value, every day, every hour (unless you’re in a deflationary environment). Stocks are much better, even if there are some trading costs involved with selling them in a hurry.

Of course, cash is necessary for paying bills, buying goods, and emergencies. I suggest putting a certain amount aside and not touching it, adding to it as need be.

Trading Techniques

The object of smart trading is to get good fills: to buy or sell exactly the number of shares you want at the very best price.

There are four major impediments to doing so:

• The bid-ask spread might be high.
• The depth might be insufficient.
• The market impact might be large.
• Your timing might be off.

For large caps and mid caps, only the very last of these (timing) is a consideration. For small caps and microcaps, however, all four are important.

Interactive Brokers has a large number of trading algorithms, as well as access to dark pools, which may be useful to reduce trading costs or to get good fills fast.

If you use another broker, you should consider how to best place limit orders.

Trading Microcaps While Minimizing Transaction Costs

First ascertain how tradable the stock is. What’s the median bid-ask spread as a percentage of the stock’s price? If it’s more than 2.5%, it’s going to be hard to trade profitably. What’s the median daily dollar volume? I sometimes trade more than 100% of a stock’s median volume in a day, but that can get quite expensive.

Never use market orders to trade microcaps. Stick to limit orders. If they don’t get filled, you can replace them or place them again another day.

If you’re after a large number of shares, it’s best to place several small orders over the course of a day or a week instead of one large one.

Don’t commit yourself to getting a complete fill every time. If the price moves strongly against you, you might be able to buy more tomorrow at a better price.

The simplest way to trade microcaps is simply to place a limit order before market open at the previous day’s close. Then forget about it. If it doesn’t get completely filled, try the same thing the next day.

I use a much more time-consuming method, involving placing orders somewhat far from the previous day’s close and then adjusting or abandoning them depending on my fills, the bid-ask spread, and how far the price has moved against me.

Portfolio Management Tools

Lastly, and unfortunately, most brokers don’t offer very good portfolio management tools. But some websites do. Portfolio123, for example, has a free “Manage” platform that offers portfolio tracking and analysis. It allows you to import transactions from any broker, send automated or manual orders to Interactive Brokers, divide your accounts into different strategies, and work with watchlists and notes. I find most of these tools very valuable in making my strategies work for me.

 

My CAGR since 1/1/2016: 49%.

My top ten holdings right now: CTG, VHI, RCMT, NATR, SAMG, HMDPF, SUPGF, PTRUF, PATI, ISTR.

How to Design a Fundamentals-Based Strategy that Really Works, Part Three: Principles of Backtesting

This is the third article in a series; here is the first and here is the second.

I've been using a fundamentals- and ranking-based strategy for stock picking since 2015, and since then my compound average growth rate is 48%. So I can attest that a fundamentals-based strategy can really work. My previous articles in this series discussed a) designing factors for multifactor strategies; and b) applying rules and creating ranking systems. In these articles, I’ve been advocating creating ranking systems as a good alternative or addition to traditional screening. I use a subscription-based website called Portfolio123 for this; ranking can also be done with spreadsheets or other data management tools, but that involves a huge amount of work and is beyond the scope of this series of articles.

Past Performance and Future Results

We always say that “past performance is no guarantee of future results”—or, even worse, “past performance is not indicative of future results,” which implies that there’s no relationship at all between them. If this were true, backtesting would be a complete waste of time. The whole idea behind backtesting is that there is a relationship between past performance and future results.

Laws and Theories

If you throw a ball into the air a hundred times and it always falls back down, you can assume that it will continue to do so every time you throw it into the air. Why? Because there’s a law behind it: the law of gravity. Similarly, there are absolutely reliable laws that govern portfolio performance, laws derived from statistics and mathematics.

Portfolio performance is governed not only by laws, but by theories. There are thousands of those, and they can be tested. But they’re not immutable or completely reliable. They could pertain to certain time periods and not to others. It’s important to recognize the difference.

Laws of Portfolio Performance

There are many of these, but the ones that I’ve found most useful are the laws of diversification, regression to the mean, outliers, and alpha and beta.

• The law of diversification can be stated as follows: The standard deviation of a portfolio’s returns is more likely to decrease than to increase as more imperfectly correlated assets are added to it. (This law can be derived from the law of large numbers, which states that the average of a sample converges in probability toward an expected value as the sample gets larger.) In plain English, the greater the number of stocks you hold, the less fluctuation you’ll see in your returns, so long as the stocks are relatively uncorrelated.
• The law of regression to the mean can be stated as follows: As long as there is a meaningful average of values, an extreme value will be more likely to become less extreme over time than to continue to be extreme. For example, if a particular industry has outperformed or underperformed other industries, it will be more likely to have average performance in the future than to continue to outperform or underperform. The same is true for factors, stocks, price returns, and so on. Regression to the mean can be overcome by certain tendencies, as one sees with stock momentum. But the law still pertains. It explains why, for instance, companies with very low free cash flow yields are more likely to have higher free cash flow yields in the future, while companies with very high yields are more likely to have lower yields in the future.
• The law of outliers can be stated as follows: The impact of extremely large and small numbers upon results is likely to make them difficult to replicate. This law applies to averages, standard deviation, compounding, and regressions. For example, the average of 2, 3, 4, 5, 6, 7, 8, and 100 is close to 17, which is not at all representative of the sample. Similarly, a strategy that has a high return due to one or two outperforming stocks or periods is less likely to have its return replicated in another time period than one whose return is relatively unaffected by such stocks or periods. This law does not apply to medians, percentiles, and similar measures. Measures that are relatively impervious to outliers are called robust.
• The law of alpha and beta is one I came up with myself, and proved it. It can be stated as follows: alpha and beta are inversely correlated so long as the market return tends to be positive. (There’s a corollary: alpha and beta are positively correlated in the rare case that the market return tends to be negative, and are uncorrelated when the market return is neutral.) This only applies when you calculate alpha and beta using a benchmark that constitutes the average returns of the securities among which you’re choosing from (as is usually the case). This is a mathematical law that applies to all asset classes. However, this inverse correlation, while provable mathematically and testable empirically, is relatively weak. The takeaway, in plain English, is that low-beta portfolios are likely to outperform high-beta portfolios.

Theories of Portfolio Performance

There are legions of these. A few examples that come quickly to mind are:

• Stocks with low prices relative to their fundamentals tend to outperform stocks with high prices.
• The stock prices of small companies tend to be more volatile than those of large companies.
• Companies with growing sales and shrinking inventory tend to outperform companies with shrinking sales and growing inventory.
• Companies that pay dividends regularly are likely to continue to pay dividends regularly.

All of these, and many thousands of others, are ready for testing to see if they can be backed by empirical data.

The Correlation Between Past Performance and Future Results

In order to figure out how best to backtest, you have to answer the following question to your satisfaction: Under what conditions does past (in-sample) performance best correlate with future (out-of-sample) results?

Further questions that this question raises include:

• How many years should be backtested?
• How large a portfolio should be backtested (i.e. should a portfolio consisting of the same number of stocks that will constitute your out-of-sample portfolio be backtested, or would a more diverse one be better?)
• What performance measures should I use?
• Should backtested results be adjusted for outliers?
• How will optimization affect the correlation?

The best way to backtest is the way that maximizes the correlation between past performance and future results.

Some people think that there is no correlation between them, and that backtesting just gives you false hope. People warn about data mining and misleading backtests. That’s why it’s so important to establish a correlation baseline.

For example, let’s say an academic study shows that a certain strategy gets a very good Sharpe ratio over the past fifty years, and it involves buying the top quintile of stocks in the S&P 500 and shorting the bottom quintile.

What the study does not answer is this question: if one buys the top quintile of stocks in the S&P 500 and shorts the bottom quintile for fifty years according to, say, one hundred different strategies, does the rank of these different strategies according to their Sharpe ratio correlate with the out-of-sample Sharpe ratio of exactly the same strategies, using the same methods, over the next, say, twenty years? Of course, this would have to be tested over various fifty-year periods and the following twenty years.

I have no answer to this question. I honestly have no idea if the answer is “Yes” or “No.” And if the answer is that there is no correlation, then all these academic studies are useless.

Furthermore, there is no standard for testing. Some academic studies use the top decile minus the bottom decile, some the top tercile minus the bottom tercile, some ignore the bottom quantile altogether. Some test over fifty years and some over less than twenty. They rely on a huge variety of different universes. Some use the Sharpe ratio as a performance measure, other use CAGR, others use risk-adjusted alpha.

It astonishes me that I have not seen any such correlation study pertaining to academic methods such as these. They have been the standard for academic backtesting over the past fifty years, yet nobody has actually tried to see if the results are meaningful in a general way. In my opinion, prior to using a testing method, one must determine if that testing method will generally yield meaningful results.

Running a Correlation Study

Running a correlation study is a huge amount of work. One has to create a library of strategies, run equity curves, and measure their correlation. Doing this may not be worthwhile for most investors. So in this section of the article I’ve summarized my own conclusions.

Here’s the way I have run correlation studies—with a view not for writing academic papers but for coming up with a strategy that will work for the way I buy and sell stocks.

1. Create fifty to a hundred different portfolio strategies. These should, ideally, have never been backtested. Get them from somewhere else, not from your own research. It’s especially important that they have never been optimized over a specific time period. On the other hand, the strategies should not be random. They should make sense, at least to someone. They should ideally be strategies that someone might use. The strategies should be flexible enough that you can decrease or increase the portfolio size. For example, you could use the various screening strategies that AAII has developed over the last few decades.
2. Run all the strategies over as long a period as possible. Then rerun them with varying portfolio sizes.
3. Compare in-sample and out-of-sample results. How long do you want your own strategy to last? Do you anticipate changing your strategy every six months, every three years, every ten years? Take your answer to that question and make that your “out-of-sample” period. Then measure the correlation between the total return of the out-of-sample period for your fifty or one hundred strategies and the return of the preceding in-sample periods: one year, two years, etc. up to twelve or fifteen years. (For example, a good correlation will show that the same strategies perform poorly in both periods and the same strategies perform well in both periods.) Ideally there should be a year between the end of the in-sample and the beginning of the out-of-sample period in order to avoid data leakage. Do this over and over again, as much as your total time period allows. Considering that mass of data, what is the length of the in-sample period that best correlates with the out-of-sample results? If the in-sample period is too long, the ranks of the strategies will cluster together and be almost indistinguishable; if it’s too short, the ranks may show very little correlation since regression to the mean is more prevalent than persistence over short periods. My own results are as follows: for a three-year out-of-sample period, a ten-year in-sample period is the most correlative, though almost anything longer than ten years will still correlate decently well. A five-year in-sample period is the worst—the correlations are often negative.
4. Vary your portfolio size. Now that you’ve determined your ideal in-sample time period, measure the correlation of those in-sample time periods with various portfolio sizes to the out-of-sample periods with the portfolio size you’re actually going to use. If you double, triple, quadruple, or quintuple the number of positions in your in-sample period, does the correlation of the strategy ranks with the out-of-sample period improve? In my experience, about two to five times the number of positions gives me better results than using the same number of positions.
5. Determine your portfolio backtesting method. You might find that a top-quintile vs bottom-quintile approach correlates best with your out-of-sample portfolio simulation; or you might find that a rolling backtest correlates best, or a simple simulation. (For me, the latter has always worked well.)
6. Examine performance measures. If you’re interested in getting the highest return in the out-of-sample period, should you follow the strategy with the highest in-sample returns? Or is better to use a different measure? What if you’re interested in getting the highest Sharpe ratio? Do Sharpe ratios in different periods have a strong correlation? Or is there another method to use in the in-sample period that will correlate better with an out-of-sample Sharpe ratio? Personally, I have found that the performance measure with the highest correlation to out-of-sample CAGR is in-sample alpha after trimming outliers.

Optimizing Your Strategy

One of the warnings I always hear is don’t optimize your strategy too much. Simply run a few backtests to see if something works or doesn’t.

The logic of this is pretty clear. The more you optimize your strategy, the closer it will hew to the data that’s available. You will then end up with a strategy that is tailored to work beautifully for a very specific set of stocks during a very specific time period. Such a strategy may be less likely to work in a subsequent time period, and you will end up with illusory results.

On the other hand, if there is a correlation, even a slight one, between in-sample and out-of-sample returns, the laws of probability say that you will be more likely to do well out-of-sample with a strategy that performs well in-sample than one that performs badly.

One solution to this conundrum may be to test over various different time periods and various different groups of stocks. I break up my universe of stocks into five equal parts and optimize different strategies for each one. I then combine those five strategies into one. I also change the beginning and end dates of my backtests; some people (notably James O’Shaughnessy) subdivide their in-sample testing period into discrete testing periods and only pick strategies that work in most of them.

There are a lot of things to optimize for when backtesting. You might want to optimize the rules that govern your universe of stocks. You might want to optimize the various elements of your portfolio management: how many stocks to hold at a time, how often you buy and sell them, what weights you give them, and so on. If you use a ranking system, like I do, you might want to optimize the weights of your various factors.

There are some things that backtesting may not tell you. Will a backtest ever give you really realistic slippage? I suggest setting your liquidity limits without backtesting. Instead, use your experience and look at the actual slippage you’re paying by comparing your transaction records for stocks with varying liquidity.

Your backtesting method may end up favoring a certain industry or sector that outperformed during your in-sample period. Beware of this. Different industries and sectors perform well at different times, and certain factors can help you concentrate on those. But don’t favor a method that will always choose the same industry/sector simply based on past results. The best investment strategies will rotate and/or diversify between industries and sectors so that you’re always on top. And you certainly don’t want to exclude certain industries just because they performed poorly during your in-sample period, or because doing so improves your in-sample backtests. Those may end up being the best industries in the near future.

Optimizing Portfolio Parameters

In my opinion, the best way to optimize your portfolio parameters is to use a factor-based stockpicking strategy that you or someone else developed a while ago and see what portfolio parameters would have worked best with it in the years since then. The worst way is to use a strategy that you’ve already backtested over the same period that you’re now optimizing.

Here are some questions you should ask:

• How many stocks should be held?
• What buy rules and sell rules should you use?
• What weights should the stocks have?
• Should you have a minimum holding period to reduce slippage?
• Should you set a maximum number of stocks per industry or sector?
• Should you set a maximum weight for a single position?

Optimizing Factors

If you’re using ranking, some factors might benefit from an approach that doesn’t favor the highest or the lowest values, but instead chooses something in-between. For example, my ranking system works very well for microcaps and small caps but doesn’t work very well for nanocaps, mid-caps, and large caps. So I don’t want my size factors to be simply smaller better, because that would emphasize nanocaps.

One way to optimize factors like these is to take your ranking system, exclude the factor in question, and test it on subsets of your universe that are split up according to the factor. So, for example, I could test my ranking system on stocks of various sizes and see which size stocks perform best and which perform worst. Or, if I were optimizing a growth factor, I could test my ranking system on stocks with various growth rates.

Optimizing a Ranking System

Two questions that will immediately come up are:

• which factors should you use?
• what weights should they be assigned?

Fortunately, those questions can be easily answered at the same time by experimenting with a long list of factors and allowing yourself to assign 0% weights to many of them.

As I wrote above, I subdivide the universe of stocks I’m willing to invest in into five more-or-less random subuniverses and run my tests on each one. I vary my factor weights until I get the ranking systems that perform best in each of my universes—and the ones that perform best in all of them—and then I average the weights of those outperforming systems.

A tip: don’t vary your factor weights by less than 2%. It’s a waste of time. One way to do this is to have all factor weights divisible by 2%, 2.5%, or 4%.

The amount of backtesting time this process takes can be extreme. Expect to spend weeks on it, or automate the process somehow. However, you really don’t have to be perfect in your optimization. A very rough approximation is good enough. You’ll find that the highest-ranked stocks are pretty much the same after a certain point.

Common Backtesting Mistakes

• Optimizing only on the entire universe from which you’ll be choosing stocks.
• Optimizing on a short and fixed time period.
• Optimizing using a small number of positions.
• Believing that an optimized result is achievable out-of-sample. (Remember that the word optimize implies that the result is closely tied to a specific sample of stocks in a specific time period.)
• Basing your portfolio parameter backtests on a ranking system already optimized over the same time period.
• Basing your optimization only on CAGR rather than on more robust performance measures.
• Taking into account only overall performance rather than looking at performance during various discrete time periods. (Some robust performance measures can handle this for you.)

Stress-Testing Your Systems

As I’ve written before, it’s important not only to build a good strategy but also to try to destroy it. A strategy that succeeds over a variety of stress tests is ready for the long haul. Backtesting for failure can be just as important as backtesting for success.

What’s Next?

My last article in this series will be devoted to implementing your strategy. I hope to publish it soon.

-------------

My CAGR since 1/1/2016: 50%.

My top ten holdings right now: Valhi (VHI), Computer Task Group (CTG), Silvercrest Asset Management (SAMG), First United (FUNC), RCM Technologies (RCMT), Medical Facilities (MFCSF), Nature's Sunshine Products (NATR), Superior Gold (SUPGF), Hammond Power Solutions (HMDPF), Petrus Resources (PTRUF).