• S-Network Global Indexes
  • Posted September 8, 2017

Not All Backtests Are Created Equal

The efficacy of the Smart Beta backtesting process has come under criticism from some corners of the market recently. Some of this criticism is valid, but much of it depends on gross generalizations.
 
A well-constructed backtest provides investors with important information about the performance characteristics of a specific methodology. That said, the fact remains that not all Smart Beta backtests are created equal.
 
Here are some key factors to look for in evaluating backtested results:
 
Transparency
Having a clearly stated and transparent methodology is at the core of any sound backtest. If the index rule set can be applied to fully replicate the index, then you know the back test is grounded in reality and will be valid. Complete transparency is a hallmark of all S-Network branded Smart Beta indexes; all of our rules are published on public-facing websites. In fact, we encourage regulators, gatekeepers and sophisticated investors to replicate our indexes independently.
 
The Universe
It’s critical to select the Smart Beta stocks or re-weight the Smart Beta stocks from an appropriate universe of stocks. At S-Network, we've developed a family of benchmark indexes that provide daily index files starting on December 31, 1999. Data from inception has been fully cleansed for these benchmarks, which have been relived on a daily basis from inception, reflecting bankruptcies, delistings, mergers, acquisitions, spinoffs and other types of corporate events. By using a well-developed universe, the practice of "garbage in/garbage out" is largely eliminated.
 
Investability 
Although trading stocks in recent years has become increasingly more frictionless, pitfalls remain in defining implementation costs. To begin with, all financial products based on an index will underperform the index by the amount of the total expense ratio of the financial product.  Beyond that, trading costs can compromise real life performance compared to backtested performance. We take steps to avoid some common pitfalls that may compromise the liquidity of an index. For example, an index’s investability is constrained by the liquidity of its least liquid stock. Accordingly, we avoid mixing large cap and small cap stocks or developed market and emerging market stocks because of potentially sharp divergences in liquidity characteristics. By the same token, we always apply liquidity screens to our eligibility rules.
 
By examining these three key considerations carefully, one can be reasonably assured of the efficacy of a backtest.  A well-conceived and implemented backtest covering an extended period provides an astute investor with critical information that would ordinarily be unavailable. At S-Network, most of our backtests originate on December 31, 1999 and therefore demonstrate performance characteristics in both good and bad markets. For example, it is revelatory to see how an index performed over the 2001 tech crash or the 2008 real estate crash.
 
Finally, it is important to note that a Smart Beta Index will seldom outperform the market in all conditions. At the risk of using a hackneyed expression, "Past Performance Is No Guarantee of Future Results!"  So far this year, most Smart Beta Indexes have underperformed the market, regardless of whether their histories are live or backtested. The reason being Apple, Microsoft, Amazon, Alphabet and Facebook have accounted for a disproportionate percentage of the stock market rise.  If these stocks did not dominate the Smart Beta index, as they do in most capitalization-weighted large cap indexes like the S&P 500, the Smart Beta Index will have underperformed the market.
 
A lot of garbage can make its way into the market based on flawed backtests. However, S-Network branded indexes as well as many of those branded by other major providers are based on sound investment principles, specific diversification policies and are screened to assure liquidity up to a certain AuM threshold.

Ultimately, it pays to evaluate backtests carefully and be able to take advantage of the myriad benefits they provide.

Latest News

Dec 13, 2017

S-Network appointed as calculation agent for two indexes at Milliman Financial Risk Management

Dec 7, 2017

10 Additions and Seven Deletions in S-Network Disruptive Technology Index Quarterly Reconstitution

Dec 7, 2017

One Addition in S-Network Mid-Cap Dividend Index Annual Reconstitution

Oct 11, 2017

S-Network Launches Real Oil Index

Sep 27, 2017

S-Network Launches US Micro-Cap Benchmark Index

Sep 8, 2017

Not All Backtests Are Created Equal

Sep 7, 2017

One Addition in S-Network Developed International Dividend Index Quarterly Rebalancing

Sep 1, 2017

A Rising Tide Lifts These Boats in Particular

Aug 3, 2017

A Different Take on Socially Responsible Investing (SRI)

Jun 8, 2017

One Addition in S-Network Mid-Cap Sector Dividend Dogs Index Quarterly Rebalancing

Jun 8, 2017

Two Additions and Two Deletions in S-Network Global Automotive Index Semi-Annual Reconstitution

Jun 8, 2017

51 Additions and 50 Deletions in S-Network Developed International Dividend Index Annual Reconstitution

Jun 8, 2017

25 Additions and 25 Deletions in S-Network ADR Dividend Index Annual Reconstitution

Jun 8, 2017

21 Additions and 21 Deletions in S-Network Mid-Cap Dividend Index Annual Reconstitution

Mar 8, 2017

Three Additions and Three Deletions in S-Network ADR Dividend Index Quarterly Rebalancing

Dec 8, 2016

43 Additions and 43 Deletions in S-Network Large-Cap Sharpe Ratio Index Quarterly Reconstitution

Dec 7, 2016

Rules Amended for Power Dividend Index Family

Oct 12, 2016

S-Network Announces Suite of Benchmark Indexes

Oct 5, 2016

Skytop Strategies Presents The Future of Corporate Reporting: Driving Brand Value and Attracting Investors

Sep 16, 2016

Three Additions/Two Deletions in S-Network ADR Dividend Index Quarterly Rebalancing

Sep 15, 2016

Stephen H. Bitteker Joins S-Network as Director of Sales

Sep 13, 2016

Two Additions/One Deletion in S-Network International Dividend Index Quarterly Rebalancing

Sep 7, 2016

One Addition in S-Network International Dividend Index Quarterly Rebalancing

Sep 7, 2016

Three Additions/Two Deletions in S-Network Weak Dollar Equity Index Quarterly Rebalancing

Sep 7, 2016

One Addition in S-Network Strong Dollar Equity Index Quarterly Rebalancing

Sep 7, 2016

Forty-two Additions/Forty-one Deletions in S-Network Large Cap Sharpe Ratio Index Quarterly Reconstitution

Sep 7, 2016

One Addition in S-Network CDP Carbon Footprint Index Quarterly Rebalancing

Jun 8, 2016

Seven Additions/Seven Deletions in S-Network CDP Carbon Footprint Index Annual Reconstitution

Jun 8, 2016

Forty-four Additions/Forty-two Deletions in S-Network Large Cap Sharpe Ratio Index Quarterly Reconstitution

Jun 8, 2016

Eighty-One Additions/Eighty-One Deletions in S-Network Durable Dividends Developed International Index Annual Reconstitution

Jun 8, 2016

Sixty-One Additions/Sixty-One Deletions in S-Network US Large-Cap Durable Dividends Index Annual Reconstitution

Jun 8, 2016

Forty-Six Additions/Forty-Six Deletions in S-Network International Dividend Index Annual Reconstitution

Jun 8, 2016

Twenty-Four Additions/Twenty-Three Deletions in S-Network Mid-Cap Dividend Index Annual Reconstitution

Jun 8, 2016

Twenty-six Additions/Twenty-sixDeletions in S-Network ADR Dividend Index Quarterly Reconstitution

Jun 8, 2016

Thirty-Five Additions/Thirty-Four Deletions in S-Network US Mid-Cap Durable Dividends Index Annual Reconstitution

Mar 15, 2016

Rule Change for S-Network ADR Dividend Index

Mar 9, 2016

One Addition in S-Network Weak Dollar Index Quarterly Rebalancing

Mar 9, 2016

Forty Additions/Thirty-nine Deletions in S-Network Large Cap Sharpe Ratio Index Quarterly Reconstitution

Mar 9, 2016

One Addition in S-Network Durable Dividends Developed International Index Quarterly Rebalancing

Mar 9, 2016

Two Additions in S-Network International Dividend Index Quarterly Rebalancing

Mar 9, 2016

One Addition in S-Network Mid-Cap Dividend Index Quarterly Rebalancing

Feb 23, 2016

Clarification of Maintenance Methodology for All S-Network Equal Dollar Weighted Indexes

Feb 9, 2016

Calvert Investments Launches Water Research Index

Dec 16, 2015

Five Additions/Six Deletions in Stowe Global Coal Index Quarterly Rebalancing

Dec 11, 2015

Seven Additions/Seven Deletions in Thomson Reuters CRB Commodity Producers Index Quarterly Rebalancing

Dec 10, 2015

One Addition/One Deletion for Ardour Global Alternative Energy Index Extra Liquid in Quarterly Rebalancing

Dec 10, 2015

One Addition/One Deletion in S-Network ITG Agriculture Index Quarterly Rebalancing

Dec 7, 2015

S-Network Mid-Cap Dividend Index Constituent Change

Sep 10, 2015

One Addition/One Deletion in S-Network ITG Agriculture Index Quarterly Reconstitution

Sep 10, 2015

S-Network Emerging Sector Dividend Dogs Index Constituent Change

Sep 10, 2015

Correction to S-Network Emerging Sector Dividend Dogs Index Constituent Change Press Release

Sep 10, 2015

Four Additions/Seven Deletions in Rogers-Van Eck Natural Resources Index Quarterly Reconstitution

Sep 10, 2015

Ten Additions/Nine Deletions in S-Network Sector Low Volatility Index Quarterly Reconstitution

Sep 10, 2015

One Addition in Ardour Global Alternative Energy Index Extra Liquid Quarterly Reconstitution

Sep 10, 2015

Ten Additions/Ten Deletions in S-Network Sector Leaders Index Quarterly Reconstitution

Sep 10, 2015

Thirteen Additions/Four Deletions in S-Network Tech Targets Index Quarterly Reconstitution

Jul 1, 2015

S-Network Launches Sector Leaders Index

Jul 1, 2015

S-Network Launches Sector Low Volatility Index

Jun 30, 2015

S-Network Launches Suite of Responsible Indexes for Calvert Investments