|
|
|
|
Bias in the Benchmarks
by Craig L. Israelsen
University of Missouri-Columbia
From Financial Planning Magazine, February 2001. Reprinted with permission.
enchmarking the stock market has never been more popular, primarily due to a surge in the demand for index funds over the past decade. And with over 100 equity market indexes, there are plenty of benchmarks to choose from. The Morningstar Principia Pro database primarily utilizes four specific equity benchmarks: the S&P 500, S&P 400 Midcap, the Wilshire 4500, and Russell 2000 (see descriptions below).
This article examines two separate but correlated issues relative to these four specific equity benchmarks. First, how does the historical performance of these four indexes relate to the two broad categories of the equity market, namely growth and value? Second, how well do these four indexes correlate to aggregate mutual fund risk/return performance as categorized by Morningstar=s popular equity style box? The style box is, very simply, a 9 square matrix which segregates stocks on the basis of market capitalization (large, mid, small) and style (growth, blend, value). Blend is a mixture of growth and value. For example, the S&P 500 Index is typically categorized in the large cap blend box.
Figure 1. Morningstar's Best-Fit Equity Indexes
|
U.S. Equities Index
|
Benchmark Market Cap
|
Morningstar Definition
|
|
Standard
& Poor=s
500
|
Large
Cap Stocks
|
A
market capitalization‑weighted index of 500 widely held stocks
often used as a proxy for the stock market. It measures the movement of
the largest issues. Standard and Poor's chooses the member companies for
the 500 based on market size, liquidity and industry group
representation. Included are the stocks of industrial, financial,
utility, and transportation companies. Since mid 1989, this composition
has been more flexible and the number of issues in each sector has
varied.
|
|
Standard
& Poor=s
Midcap 400
|
Mid
Cap Stocks
|
Includes
approximately 10% of the capitalization of U.S. equity securities. These
are comprised of stocks in the middle capitalization range. At the
original time of screening, this was a $200 million to $5 billion market
value range. (As of November, 2000 the midcap range was $1.5 to $10
billion.) Any midcap stocks
already included in the S&P 500 are excluded from this index, which
started on December 31, 1990.
|
|
Wilshire
4500
|
Mid
& Small Cap Stocks
|
Listed
for small‑company funds, measures the performance of all U.S.
common equity securities excluding the stocks in the S&P 500.
|
|
Russell
2000
|
Small
Cap Stocks
|
Consists
of the smallest 2000 companies in the Russell 3000 Index, representing
approximately 7% of the Russell 3000 total market capitalization.
|
Before addressing the first question, a bit of background information might be helpful. Equity mutual funds, within the Morningstar Principia Pro database, are assigned a Abest-fit@ equity index. The term Abest-fit@ makes reference to the statistical correlation between the performance of a benchmark index and a particular fund. Therefore, the past performance for each equity mutual fund in the Morningstar database is compared to the performance of these four indexes. The index with the highest correlation is assigned as that fund=s Abest-fit@index. (There are actually two additional equity indexes used by Morningstar, JSE Gold and Wilshire REIT. Relatively few funds end up being assigned to either of these two, hence they were omitted from this study). The term best-fit refers to a statistical measure known as R-squared. For a review of the major components of modern portfolio theory, please refer to the article entitled AAlpha Beta Soup@ in last month=s issue of Financial Planning.
As already mentioned, there are dozens of equity market indexes. Clearly, the U.S. equity market is too broad to be tracked by one index. A single benchmark index would be comparable to using one temperature reading per state -- and calling that a good indicator of the state=s overall temperature. Maybe that would work in North Dakota in the dead of winter, but it sure wouldn=t work in Texas where the temperature in the Brownsville is often very different than in Lubbock. The range of temperatures in Texas is simply too broad to reduce down to one reading. The stock market is no different. So, just as Texas is divided into broad climate zones, the equity market is also. At the very least, the U.S. equity market can be divided into three size categories (large cap, mid cap, small cap) and two style groups (value and growth). Value is a term assigned to stocks (and stock portfolios) with low price to earnings ratios, whereas growth stocks have high p/e multiples. As no single index can possibly incorporate all of these distinct equity market segments many have been created over the years (see Figure 2).
FYI, mass media sources have historically focused on the Dow Jones Industrial Average despite the fact that it is a relatively poor barometer of the market given that it only includes 30 extremely large companies. In recent years, the Standard & Poor=s 500 Index has gained popularity among mutual fund companies as Athe@ index to mimic. As of November 30, 2000 there were approximately $342 billion invested in U.S. equity based index funds (mutual funds which mimic a particular index). Of that total, 76% of the assets (or $259 billion) were invested in index funds which track the S&P 500 Index. Indeed, the S&P 500 is the baseline benchmark for U.S. equity funds. In recognition of that, Morningstar always reports the alpha, beta, and R-squared based upon a fund=s correlation with the S&P 500 in addition to that funds best-fit index (which may or may not be the S&P 500).
As shown in Figure 2, many equity market indexes make a distinction between growth and value. This is precisely why the first question is ADo the four benchmark indexes used by Morningstar (shown in larger, bold font in Figure 2) correlate more with growth or value?@ This is a highly relevant question inasmuch as performance between growth and value can be very different at times. In fact, the three year period ending November 30, 2000 was one of those times as can be seen by perusing the 3 year data in Figure 2. For instance, compare the 3 year performance of the Barra Midcap Growth Index (22.4%) against the Barra Midcap Value Index (9.6%). Of course, short time frames, such as 3 years, are more likely to expose anomalies in performance. Over longer periods, say 10 years, the difference in return between growth and value is typically smaller, as is generally seen in Figure 2.
Figure 2. Selected Equity Market Indexes Data as of November 30, 2000
Only Indexes which specify AGrowth@ or AValue@ are listed Four Abest-fit@ equity indexes used by Morningstar in larger, bold font.
|
Equity
Index
|
3 Year
Return
|
10 Year
Return
|
|
Barra Large Cap Growth
|
14.6
|
18.5
|
|
Barra Large Cap Value
|
10.1
|
16.6
|
|
Barra MidCap Growth
|
22.4
|
-
|
|
Barra MidCap Value
|
9.6
|
-
|
|
Barra SmallCap Growth
|
4.5
|
-
|
|
Barra SmallCap Value
|
2.2
|
-
|
|
Dow Jones Large Growth
|
12.6
|
18.0
|
|
Dow Jones Large Value
|
8.8
|
16.1
|
|
Dow Jones Midcap Growth
|
16.7
|
20.1
|
|
Dow Jones Midcap Value
|
8.2
|
16.9
|
|
Dow Jones Small Growth
|
12.7
|
18.1
|
|
Dow Jones Small Value
|
0.8
|
15.7
|
|
Russell 1000 Growth
|
14.4
|
18.1
|
|
Russell 1000 Value
|
9.2
|
17.1
|
|
Russell
2000
|
2.4
|
15.0
|
|
Russell 2000 Growth
|
1.9
|
12.7
|
|
Russell 2000 Value
|
1.9
|
16.8
|
|
Russell Midcap Growth
|
14.8
|
18.0
|
|
Russell Midcap Value
|
6.1
|
17.4
|
|
Russell Top 200 Growth
|
14.7
|
18.5
|
|
Russell Top 200 Value
|
10.9
|
16.8
|
|
Standard
& Poor's 500
|
12.7
|
17.7
|
|
Standard
& Poor's Midcap 400
|
15.7
|
19.6
|
|
Wil-Target Large Co Growth
|
17.7
|
20.2
|
|
Wil-Target Large Co Value
|
3.7
|
14.5
|
|
Wil-Target Mid Cap Growth
|
3.2
|
16.3
|
|
Wil-Target Mid Cap Value
|
1.2
|
15.8
|
|
Wil-Target Small Co Growth
|
1.0
|
15.4
|
|
Wil-Target Small Co Value
|
-4.3
|
14.1
|
|
Wilshire
4500
|
6.1
|
15.6
|
|
Wilshire Large Growth
|
14.6
|
17.8
|
|
Wilshire Large Value
|
8.0
|
16.7
|
|
Wilshire Midcap Growth
|
12.7
|
16.8
|
|
Wilshire Midcap Value
|
6.0
|
17.5
|
|
Wilshire Small Growth
|
7.0
|
14.9
|
|
Wilshire Small Value
|
1.7
|
17.3
|
Figure 3. Growth vs. Value:
Correlations to Indexes over 3 and 10 Years Data as of November 30, 2000
|
Benchmark
Equity Index
|
Growth
Correlation
(3
Yrs)
|
Value
Correlation
(3
Yrs)
|
Growth
Correlation
(10
Yrs)
|
Value
Correlation
(10
Yrs)
|
|
S&P 500
|
.91
|
.82
|
.94
|
.86
|
|
S&P Midcap 400
|
.80
|
.72
|
.85
|
.82
|
|
Russell 2000
|
.96
|
.64
|
.95
|
.76
|
|
Wilshire 4500
|
.96
|
.55
|
.96
|
.67
|
As shown in Figures 3 and 4, the four major equity benchmark indexes utilized by Morningstar showed a distinct bias toward growth stocks, particularly among the small cap indexes (Russell 2000 and Wilshire 4500). Interestingly, despite greater similarity of performance between growth and value over the 10 year period, the benchmark index bias toward growth stock persisted in nearly identical fashion (see Figures 3 and 5).
Over the 10 year period, the correlation of the S&P 500 Index to large cap growth stocks was 94% and 86% to large cap value stocks. The S&P Midcap 400 had an 85% correlation to mid cap growth stocks and an 82% correlation to mid cap value stocks. The Russell 2000 had a 95% correlation to small cap growth stocks and a 76% correlation to small cap value stocks. The Wilshire 4500 had a 96% correlation to small cap growth stocks and a 67% correlation to small cap value stocks. The performance of AGrowth@ and AValue@ stock was determined by averaging the performance of the Wilshire Growth Indexes and the Dow Jones Growth Indexes over the 3 and 10 year periods. Those averages were then compared to the performance of the four benchmark indexes in question.
Now, we move to the second question. AHow does aggregate equity fund performance (as categorized within the Morningstar Style Box) compare to the risk/return history of the four primary benchmarks utilized by Morningstar?@ The data in Figure 6 reveal three risk/return performance planes, with small cap funds (Small Value=SV, Small Blend=SB, and Small Growth=SG) generating the lowest returns compared to mid cap and large cap funds. In each market cap group (i.e. performance plane), value funds were outperformed by blend funds, which were outperformed by growth funds. A total of 1,405 equity funds over a three year period ending 11/30/00 are represented in Figure 6.
The S&P 500's 3 year annualized return of 12.7% was closest to the 11.9% return of large cap blend funds. There were 317 equity funds in the large cap blend category. Large cap value funds (n=270) had a considerably lower average return of 8.0% over the three year period. The 210 large cap growth funds had an average annualized return of 18.8%, or 32% higher than the S&P 500's 3 year return. Of course, higher return among growth funds came at the cost of higher risk. The standard deviation of return for the S&P 500 over the 3 year period was 19.8% compared to 35.6% for large cap growth funds B a risk level nearly 80% higher. The level of risk for large cap blend and large cap value funds was essentially the same as the S&P 500 over the 3 year period.
Let=s now examine the midcap funds in Figure 6. The midcap index used by Morningstar is the S&P Midcap 400. Over the three year period the average return for 172 midcap growth funds was comparable to the S&P 400, but with considerably higher risk. Midcap blend funds (n=54) and midcap value (n=84) had a comparable level of risk to the S&P 400 but much lower average return.
In the small cap category Morningstar uses two indexes, the Russell 2000 and the Wilshire 4500. The aggregate 3 year returns of small cap value funds (n=71) and small cap blend funds were comparable to the Russell 2000 and at similar levels of risk. Small cap growth funds (n=150) had much higher returns (10.8% average) than either the Russell 2000 or Wilshire 4500. But, once again, the higher return comes with higher risk.
In summary, the risk and return of large cap blend funds was mirrored closely by the S&P 500 over the 3 year period ending 11/30/00. The S&P 500 was not an accurate reflection of the average performance of large cap growth funds. This is interesting inasmuch as the S&P 500 (as shown in Figure 3) correlated more closely with growth than with value over the 3 year period. Among midcap funds, the S&P Midcap 400 tracked the return of midcap growth funds quite well. However, the midcap benchmark had a much lower level of volatility, comparable to midcap value and midcap blend funds. Small cap value and blend funds were mirrored well by the Russell 2000 both in terms of risk and return over the three year period. The return of small cap growth funds was well beyond both small cap benchmarks, but closer to the Wilshire 4500. Neither of the small cap benchmarks captured the volatility of small cap growth funds.
Three years is hardly long enough to establish reliable correlations, therefore Figure 7 presents results over a 10 year period. Interestingly, the fundamental relationships demonstrated during a 3 year period are still present. Several slight variations emerge with are worthy of note. First, the S&P Midcap 400 Index becomes an even less accurate measure of the average midcap fund over a 10 year period. Blend funds provided inferior returns relative to straight value or straight growth among mid and small cap funds. Over the longer time frame the returns of the S&P 500 better reflect the returns of large cap growth, which is in harmony with the 10 year data in Figure 3. Finally (and this is very interesting) the returns of value stock (large, mid, and small) are clustered very tightly over the 10 year period. The risk level increases only slightly when moving from large to small. Among growth stocks, returns decrease and risk increases when moving from large to small. This is contrary to both classic theory and long-term empirical evidence. Hence we must conclude the obvious: the past decade has clearly favored (1) large cap stocks and (2) growth stocks.
What then, is the impact of a growth oriented decade on mutual fund analysis? More specifically, does such a decade have an affect on the analytical tools we use, such as alpha, beta, and R-squared? The answer is yes.
If the indexes (at least the four being analyzed here) are biased toward growth portfolios such a bias would be expected to manifest itself in the measurement of R-squared -- and indeed it does. Recall that R-squared is a measure of a fund=s correlation to its benchmark index. In Figure 8 the average best-fit R-squared is considerably higher in the growth style categories for both midcap and small cap funds (as noted by the upper graph line with the hour glass markers). The lower line represents the average R-squared for the funds in each Morningstar equity style category relative to the S&P 500. Notice that R-squared declines when moving from large cap to small cap. This is precisely why Morningstar utilizes benchmark indexes other than solely the S&P 500. Without the use of best-fit indexes the R-squared of small cap funds is below 50%. When more appropriate benchmark indexes are utilized (e.g. Russell 2000 or Wilshire 4500) the R-squared of small cap funds ranges from 65% to 85%. It is worth noting that the best-fit R-squared for midcap value funds is only slightly higher than the R-squared using the S&P 500 as the benchmark. This suggests that midcap value is a particular equity style that is not highly correlated to it=s best fit benchmark, regardless of which benchmark that happens to be. One would expect that the best-fit benchmark for a midcap value fund to be the S&P Midcap 400, but that is often not the case. For example, of the 84 funds classified as midcap value funds in the three year data set, 41 had the S&P 500 as their best fit index, 39 were assigned to the S&P Midcap 400, one to the Russell 2000, and three to the Wilshire 4500. Midcap funds are showing themselves to be a difficult category to accurately track via a single benchmark index. Bottom line: best-fit benchmarks have tracked growth portfolios much more reliably in recent years than value portfolios.
Finally, Figures 9 and 10 demonstrate the impact of a growth bias on alpha and beta. First, it is important to realize that alpha is primarily a correlate of return and that beta is a correlate of risk (i.e. standard deviation of return). Over the 3 year period the correlation between the return of 1,405 equity funds and their best-fit alpha was .86. That=s a high correlation (recall that 1.0 indicates a perfect positive correlation). The correlation between the best-fit alpha and the standard deviation of return was .65. Hence, we conclude that alpha is a better measure of return than risk. Beta, on the other hand, had a .23 correlation to return and .64 correlation to standard deviation of return. Beta is the better risk measure of the two.
In Figure 9, the average 3 year best-fit alpha is shown to track the average annualized 3 year return of each Morningstar equity style very closely, and in every case the alpha is considerably higher in the growth fund categories. We observe that alpha is highly correlated to return and, as return has been higher in growth funds in recent years, alpha is therefore highly positively correlated to the growth style of investing. In short, alpha has been biased toward growth in recent years. I remind the reader that the word Abias@ in this context does not have a negative connotation. Bias simply reflects the natural forces of the market exerting themselves on the tools of measurement (i.e. alpha, beta, R-squared).
Finally, in Figure 10 we can see that the average best-fit beta is higher among growth funds in each of the three market cap categories (large, mid, small). The data period was the three year period from November 1997 to November 2000. As already demonstrated, higher return is generally achieved at higher levels of risk. In recent years, higher levels of return and risk have been in the growth fund categories. Therefore, as beta is a close correlate of standard deviation of return (which is often used as a proxy for risk) beta has been higher in the growth categories.
While all of this may seem to complicate the obvious it should be remembered that growth investing is not always the hot hand. When value stocks are back in favor the bias would likely shift in that direction. Bias in the analytical tools is not a problem. Not recognizing its presence is.
Figure 4. Three Year Correlations
Figure 5. 10 Year Correlations
Figure 6. Three Year Data for 1,405 Equity Funds
Figure 7. Ten Year Data for 528 Equity Funds
Figure 8. Growth Bias in R-Squared (3 Year Data)
Figure 9. Growth Bias in Alpha (3 Year Data)
Figure 10. Growth Bias in Beta (3 Year Data)
|
Sponsored by James J. Eccleston. This Web site contains material of general interest. It is neither intended to, nor constitutes, either legal advice or investment advice.
Always consult an attorney and/or investment adviser when building and protecting your wealth.
All content Copyright © 2010 Advocate Compliance Partners, Inc. except where noted. All rights reserved.
One North Franklin Street, Suite 2620, Chicago, IL 60606
Telephone 312-332-0000 | Fax 312-332-0003
|
|
|
|
|
