It's in the Stars

eginning in July, 2002 Morningstar introduced an improved star rating methodology. Under the old system funds were grouped into four broad asset classes (domestic equity, international equity, taxable bonds, and municipal bonds), evaluated on the basis of historical return and risk, and then given a star rating from 1 to 5 (one star being the lowest score).

Now, under the revised star rating system, each fund is judged within one of 48 specific equity and debt peer groups (large cap growth, mid cap blend, small cap value, specialty health, Europe, international hybrid, muni bond short term, etc.). In addition to broadening the comparison categories, Morningstar has tweaked the way in which they evaluate risk. Downside risk is still weighted more heavily, but now funds that have highly volatile return patterns (even on the upside) are penalized.

The biggest weakness in the old star rating system - too few comparison categories - has been fixed. Now, for example, domestic large cap growth funds will only be compared against domestic large cap growth funds. As a result, the new star ratings should more accurately identify funds that are providing superior risk-adjusted performance within a specific peer group. The old star rating system was too heavily influenced by cyclical market trends. For instance, during periods in which large cap stocks dominated, large cap funds received a disproportionate percentage of high star rankings in the broad "domestic equity" asset class. Likewise, when small cap stocks were top dogs it was small cap funds which earned a higher percentage of 5 star rankings.

So, how did the stars shift in the July 2002 release of Principia Pro? This study focused on that exact question. The funds in this analysis included all distinct domestic equity funds which had at least 10 years of performance data as of April 30, 2002. There were 978 funds which met these criteria. (The term "distinct" meaning that only the largest portfolio was included for funds with multiple share classes). The April 30 data set used the old Morningstar rating system. The same 978 funds were then culled out of the July release of Principia Pro (data as of June 30, 2002) which employed the new star rating system. This study focused on Morningstar's overall star rating, which is a composite of the 3 year star rating, 5 year star rating, and 10 year star rating.

Of the 978 funds, nearly half (47%) experienced no change in their star rating (see Figure 1) as a result of the revised methodology. A total of 42 funds lost two stars, 236 funds lost one star, 225 funds gained a star, and 16 funds picked up two stars. The domestic equity funds that lost two stars tended to be smaller funds, as noted by a higher average equity style box score (Figure 1). The Morningstar equity style box is a nine square grid which places large cap value in the northwest corner, and for convenience in this study is assigned a value of 1. Large cap blend is assigned a value of 2, large cap growth 3, midcap value 4, and so on down to small cap growth which is assigned a value of 9.

The funds which gained two stars tended to be larger funds, and particularly large cap growth funds (see Figure 2). Conversely, the domestic equity category hardest hit by the new rating system were small cap value funds - ironically the fund category with the best performance in recent years (see Figure 3).

From the data in Figure 2 we observe that the general effect of the new star rating was a star downgrade among value funds, particularly small cap. Blend funds, which very likely have been tilted more toward value recently, also lost ground in the star ratings. The exception being among large cap blend funds. Growth funds tended to improve in the star ratings, particularly large cap funds. Small cap growth funds, on average, held their own.

Interestingly, the worst performing domestic equity category in recent years has been large cap growth, as noted in Figure 3 using the Wilshire Indexes as benchmark performance indicators. So, for the new star rating system to reward large cap growth funds with, on average, a one half star increase may, at first glance, seem counter-intuitive. But, in fact, these results are entirely consistent with the intent of the new star rating system.

Small cap value and midcap value domestic equities have been the shining stars in the U.S. market in recent years, and as a result, have been garnering the high star ratings from Morningstar using the old methodology. Growth funds (large, mid, and small) got their teeth kicked in during the past three years and therefore did not fare well in the star rating methodology because they were competing head-to-head with value funds, most notably small cap value funds.

Enter the new methodology. Now, large cap growth funds compete only against large cap growth funds, mid cap blend funds against mid cap blend funds, etc. With these new star rating "rules" the very funds which have lost the most stars in recent years (due to the vagaries of equity market cycles) are the very same funds which have the most to gain by being judged on a peer-to-peer basis.

Using a basketball analogy, the old star system grouped all basketball players together and then rated them. Thus, point guards, shooting guards, small forwards, power forwards, and centers were all competing for the top rating. If the rating criteria happened to be assists (analogous to a market cycle favoring value stocks) the point guards came out on top. If, on the other hand, the rating criteria happened to be slam dunks (analogous to a secular period favoring growth stocks) it would not surprise anyone that power forwards and centers received the highest ratings. If the criteria happened to be scoring (analogous to a period favoring small stocks over large ones) shooting guards were ranked highest. Clearly, the rankings were very susceptible to the criteria and certain criteria favored one type of player over another.

The new star methodology judges point guards against point guards, small forwards against small forwards, centers against centers, etc. Perhaps most importantly, point guards can still shine in the star ratings even if the criteria is slam dunks or shot blocking because they will be evaluated only against their peers. Thus, the best dunking and best shot-blocking point guard will receive 5 stars, despite the fact that these two criteria are seldom their strength. In the old system, it couldn't have happened because the centers and power forwards would have dominated the high star ratings using those two particular criteria.

The new star rating will lessen the effect of secular market trends rewarding some funds and punishing others. Now, the best large cap growth fund will receive a high star rating even if large cap growth stocks are not performing well. Just as basketball players need to be evaluated against other players at the same position, so do equity funds. That day has arrived.

Figures 1 and 4 summarize the "irony" of the new star rating methodology. The funds which have performed the poorest in recent years are the funds which experienced the most improvement in the new star rating methodology. Notice that the funds which lost two stars had an average annualized three year return (as of 6/30/02) of 3.1% whereas funds which gained two stars had an average three year return of -10.5%. The same negatively sloped trend line was evident across the 5 and 10 year periods.

Does this mean that funds with high star ratings may have poor performance histories? No. In fact, just the opposite is the goal. The new star rating methodology assures that funds with a high star rating are the best performers (relative to the criteria being used) within their category. If an entire category has been performing poorly, the funds with the highest star rating are the best of the bunch, even if the bunch hasn't been doing all that well.

One more analogy in conclusion. Using the old star rating system, some funds would simply "rise with the tide". They received relatively high star ratings simply because they resided within a sector currently in favor and not necessarily because they were being managed well (i.e. the fund did not add value above and beyond what their equity market sector was providing). The new star rating system seeks to isolate the best skippers, regardless of the tide. And just as skippers use the stars to navigate, the new Morningstars should provide better navigational bearings for planners and clients.

Figure 1. A Change in the Stars (n=978)

Figure 2. Average Change in Star Rating by Morningstar Equity Style Box (n=978 funds)

Figure 3. Annualized Returns as of June 30, 2002

Figure 4. Stellar Irony