In response to feedback we have improved the suite of Relative Strength backtesting applications. The "Use Next to Last Day Pick(s)" option, which some users found confusing, has been replaced by an option to "Invest on first day of next period".
This change has two benefits:
A) Consistency throughout the site - because the "first day" option still uses the Screener’s end of period picks, the ETFs chosen will always match those shown on the Screener Ranks and RS Reader pages
B) Because the "first day" option invests in exactly the same ETFs, the performance is generally much closer to that of the standard backtest
To be clear, ETFreplay.com does not use opening prices in any of its backtesting apps. This is because opening prices can often be skewed quite significantly. Closing prices cannot be manipulated nearly as much because the final 30 mins is a very volume-intensive time and anyone trying to set a price could easily get 'run-over' trying to do it. By contrast, opening prices are somewhat guesswork. The underlying securities of an ETF haven't opened either - so it is impossible in some cases to precisely know how all the various gaps will play out in terms of the overall index. Moreover, market makers can sometimes manipulate the opening price to fill existing market orders and then let the price go back to its natural point. This is especially true on options expiration. Sometimes the opening price might print a small number of shares and then move quickly in the other direction to a more natural price. Consequently we think it's best, as a rule of thumb, to avoid trading the open.
In addition, it should be clear that if you trade on the first close of a month, then the comparison index return should also be set from the first close of the month. You should not compare an entry on the first close to an index return that uses a prior day -- it is an unfair comparison. You want to go for consistency and match the time periods.
Let’s go through one example using the 'Sample' Portfolio. This portfolio was added to member’s lists to show an example of a few different ideas -- everyone got the same portfolio since we launched this idea in the Spring of 2010.
First note not only the raw return -- but the difference between the return and the index. +12.0% vs +6.2% for a spread of +5.8%:

Then view using the first close to track performance for both the strategy return as well as the benchmark:

And then finally, take note of the fact that the dates are offset by 1 day. We go deep into the issues of dates throughout our site as there are many problems with dates in most financial databases. If you 'miss' dates, your output will be garbage. We make sure there are no missing dates for any ETF through the use of SQL database queries.

If we leave the "Invest on first day of next period" option unchecked, then the back test will invest in the chosen ETFs on the close of the last day of the period. If we use the checkbox, two things happen:
1) the 'update' period will now be the first closing price of the new period
2) the comparison index will be tracked from the matching date.
Since the picks are the same regardless of method, this tightens up the performance greatly. The only difference is the performance on the first day of the period and the performance of the extra closing day on the update date. These will affect the overall result -- but they treat both the chosen ETF and the comparison index the same.
Note the slight difference in year-to-date returns between the standard (unchecked) backtest and the first day option backtest. It will be better sometimes and worse sometimes and over time, some of the differences will offset each other. To the extent the first day of the month goes up, you will be long the pick from the previous month and will benefit from this. To the extent it goes down, so too will the comparison index. We think this consistency will clarify some of the issues we've been having in email.
Follow Us On Twitter:
http://twitter.com/etfreplay