In ETF Investing, 'Backtest' should be synonymous with 'Research'

Dec 22, 2015 in Backtest

In one of the best poker books ever written The Theory Of Poker, the author makes a point quite relevant to investing.   He writes,

“Beginning poker players sometimes ask, ‘What do you do in this particular situation?’”   

The problem is that it is simply the wrong question and indicates overly-simplistic mode of thought.   The right question is:

“What do you consider in this particular situation before determining what to do?”

Likewise, the term ‘backtest’ is sometimes viewed critically by those who are beginners.   Some make simplistic assumption that just by applying a longer timeframe to a given backtest then validates a strategy and that therefore they can just follow a simple model and don’t have to burden themselves anymore with any critical thinking.

But ETF backtesting isn’t anything more (or less) than information, and in the investment business,  information is research.  Research is what leads you to make informed decisions.    Often times you run backtests that seem to conflict with each other in terms of what to do at a given point in time.  A quarterly updating backtest with a 12-month lookback might have good results and indicates to hold on until Dec 31 while a monthly backtest with a 3-month lookback might be indicating a switch is called for immediately.  Some argue that whichever one has a higher return over the longest timeframe available must be correct. But frankly, your strategy backtested to the year 1900 might not be nearly as relevant as you think (see Intro to Regime Change ETF Backtesting). It may even be an awful strategy for the present-day environment.

At the end of the day it is sometimes going to be a judgment call based on the weight of the evidence and the conclusions you reach based on your overall research/information.       

No matter whether you stand pat and do nothing or make some trades due to perceived increased risk, you might end up wrong in the short-run.   This is why you focus on the decision-making process and not short-term results because any single decision can lead to a poor result.    But performance over time is the culmination of many decisions.

Some might say to this…. ‘well I don’t want to play’ and I do have the choice to ‘just buy index funds.’  But ask anyone maintaining a buy and hold global portfolio how those ‘just buy index funds’ portfolios have done?   They are loaded in international stocks and emerging markets and have portions allocated to gold and other commodities.   Have you seen how poorly these segments have done?   Moreover, do you think the S&P 500 is immune to similar bouts of very poor relative performance?  Think again.

The point is that any judgment you make can be right or wrong, so don’t think anyone is removed from the process just because they buy index funds.   They aren’t.   You didn’t see buy & hold investors do much marketing in 2001-02 during a bear market.  Nor in 2008-09.   Buy and hold global advisors of today can’t market their returns because they have losses, not returns.  

Summary:  Focus on what to ‘consider’ and then make an informed decision.  Good investors know how to control their drawdowns while still exposing themselves on the long side as a general framework.   Backtesting is very much a part of this broader decision-making process because be it buy hold portfolio allocation backtesting or tactical backtesting, backtesting is simply about research.

Follow ETFreplay on