Starry Eyed

orningstar hit a home run when they devised the Star Rating. It's a grading system that's easy to grasp. Just as in school where an 'A' is clearly understood to be better than a 'D', a five star fund is clearly better than a 1 star fund. Case closed, right?

Well, we probably have memories of the "book-smart" kid in school who aced every subject, but wasn't very "street-smart". We learned early on that there are different ways to be smart. Some kids do well on standardized academic tests, but do poorly when required to apply the knowledge in a practical way. The same might be true for mutual funds. It entirely depends on the evaluation criteria. In other words, there might be several different ways to be a "good" fund (or a "smart" child), but if the evaluation criteria only focus on one type of performance the fund (or child) may score poorly. As is always the case, output is a function of input.

This article investigates the characteristics of funds which garnered a Morningstar 5 star rating as of the three year period ending 2/28/2002 in an attempt to address the issue: In what specific ways is a 5 star fund superior to funds of lesser stars? Similarly, in what ways is a 1 star fund inferior to higher rated funds?

First, let's review the evaluation criteria utilized in the star rating. Incidentally, what most of us simply call the "Star Rating," Morningstar refers to as the "Risk Adjusted Rating". This "risk-adjusted" issue turns out to be a very important aspect in understanding the nature of the star rating.

"Often simply called the Star Rating, the Risk-Adjusted Rating brings performance (returns) and risk together into one evaluation. To determine a fund's star rating for a given time period (three, five, or 10 years), the fund's Morningstar Risk Score is subtracted from its Morningstar Return Score. The result is plotted on a bell curve to determine the fund's rating for each time period: If the fund scores in the top 10% of its broad asset class (domestic stock, international stock, taxable bond, or municipal bond), it receives 5 stars (Highest); if it falls in the next 22.5% it receives 4 stars (Above Average); a place in the middle 35% earns 3 stars (Neutral); those lower still, in the next 22.5%, receive 2 stars (Below Average); and the bottom 10% get only 1 star (Lowest)...."

"Morningstar recognizes that no rating system can take into account all factors that one must consider before deciding to invest in a mutual fund. The Morningstar Risk-Adjusted Rating is intended as a way of identifying funds that have produced the highest level of returns relative to the risks they've taken...."

"The Morningstar Return figure puts a fund's returns in context of a risk-free rate. The Morningstar Return figures also reflect the effect of loads (unlike regular total return figures)...."

"To calculate the Morningstar Risk score, we plot a fund's monthly returns in relation to T-Bill returns. We add up the amounts by which the fund fell short of the Treasury bill's return and divide the result by the total number of months in the rating period. This number is then compared with those of other funds in the same broad asset class. The resulting risk score expresses how risky the fund is, relative to the average fund in its asset class. The average risk score for the fund's asset class is set equal to 1.00; thus a Morningstar risk score of 1.35 for a taxable-bond fund reveals that the fund has been 35% riskier than the average taxable-bond fund for the period considered...."

"In a sense, Morningstar's rating system assumes that each fund is the only position in an investor's account. On that basis, it clearly makes more sense to hold a 5-star all-weather fund than a volatile 1-star precious-metals fund. That doesn't suggest, however, that the lower-rated fund is without merit-in this case, the fund simply isn't designed to be an all-weather holding. Accordingly, there can be reasons to hold a low-rated fund. It all depends on an investor's goals and the other holdings in his or her portfolio. Not all 5-star funds are automatically all-weather funds, either...."

"Recognizing the limitations of Morningstar's basic rating system (or those of any quantitative system, for that matter) is highly useful. Morningstar's ratings have validity-but no quantitative system is infallible. Still, as a first-level screen supplemented by a wealth of fundamental information, the ratings work exceedingly well if investors treat them in the spirit intended."

To summarize, the Star Rating system:

Rates funds within broad asset classes (domestic equity, non-U.S. equity, etc.) and does not distinguish by market cap or style orientation.

Uses load-adjusted, excess returns. Excess return referring to the return above the risk-free rate (i.e. T-bill return).

Uses a proprietary risk measure which weights downside risk more heavily, rather than using standard deviation of return (a common surrogate measure of risk) which equally weights upside and downside risk.

Let's examine the first issue, namely the rating of funds within broad asset classes. The data set for this analysis consisted of all domestic (U.S.) equity funds with at least 5 years of performance history and at least 65% of their portfolios in stock as of 2/28/02. Only non-redundant share classes were included (typically A class). A total of 1,499 funds met the criteria. Figure 1 reports the following averages: 3 year average annualized return, 3 year standard deviation of return, median market cap, 3 year Sharpe ratio, manager tenure, 12 month yield, number of portfolio holdings, turnover ratio, and expense ratio. These averages are calculated by star rating group. Morningstar uses "Morningstar Return" and "Morningstar Risk" in calculating a fund's star score. Nevertheless, the results reported here are "raw" 3 year annualized return and 3 year standard deviation of return inasmuch as these are the commonly reported, and studied, figures. For instance, a Sharpe ratio is calculated using generic standard deviation of return. Recall that our purpose is to determine in what specific ways (e.g. across what measurement variables) 5 star funds and 1 star funds differ — other than their star rating.

We can make several important observations from the data in Figure 1. First, the 3 year period being considered was considerably kinder to small and mid caps than to large cap U.S. stocks. Recall that all the funds in this analysis have at least 5 years of performance history. This was done in an attempt to minimize the "new fund" effect in the results.

Inasmuch as Morningstar evaluates all U.S. equity funds together as a single group, regardless of market cap size, the assignment of stars will clearly be affected by the market sector that has currently been in favor. This is not necessarily a weakness, simply an observation of how the ranking is done. In this particular 3 year time horizon, a much higher percentage of small and mid cap funds received high star rankings as noted by the median market cap data associated with the four star and five star rankings.

Second observation: funds with high star ratings (4 or 5 stars) tend to have much higher raw return, but have standard deviation of return similar to funds with lower star ratings. ("Raw" referring to return which has not been adjusted for sales load or the risk-free return). The one exception being the 1 star funds, which, on average, had considerably larger standard deviation of return. Hence, standard deviation does not present itself as a consistent correlate of the star rating, whereas raw return does appear to be highly correlated. It should be remembered that Morningstar does not use the traditional standard deviation measure in its star rating system, but rather an in-house risk measure which does not equate up-side risk with down-side risk (as generic standard deviation does).

Despite standard deviation having an unstable correlation with star ratings, the Sharpe ratio (which is computed using standard deviation) has a very clear correlation. Five star funds have the highest Sharpe ratio (where higher is better) and one star funds the lowest, with a linear pattern in between. These findings suggest that the Sharpe ratio could be used as a screening filter to evaluate funds in a manner similar to the Morningstar star rating system.

Figure 1. All U.S. Equity Funds (n=1,499)

Also note in Figure 1 that one star funds differ considerably from the other funds in terms of manager tenure (less), 12 month yield (much lower), average number of holdings (much lower), average turnover ratio (much higher), and average expense ratio (higher). The differences in these particular variables among the 2, 3, 4 and 5 star funds were slight, suggesting that funds "worthy" of the single star rating really are different. Single star domestic equity funds might be summarized as follows: expensive, dynamic portfolios with fewer, usually non-dividend paying, holdings which exhibit volatile, down-side inclined return patterns produced by managers with less tenure. Or, more simply, they're lousy funds.

Now, let's look at the funds broken down by market cap categories. Figure 2 displays the results for large cap funds. Only 2 percent of large cap funds earned a 5 star rating during this particular 3 year period. As would be expected, the five star funds had the highest average 3 year return. Interestingly, they also had the second highest standard deviation of return. Hence, these 5 star large cap funds had "good" return-volatility of return as opposed to the 1 star funds which had "bad" return-volatility (as manifested by an average 3 year return of -11.5%). In other words, 5 star funds had volatile returns, but there was more upside variation (returns above the fund's mean return) than downside variation.

It is important to note that higher standard deviation may, or may not, be a bad thing. Assuming that funds with higher standard deviation of return are always "riskier" may be presumptuous. These results suggest that such an assumption is too simplistic - which is precisely why Morningstar uses a measure other than standard deviation of return in the calculation of the star rating.

Among large cap funds, Sharpe ratio and manager tenure demonstrate clear, linear correlation with star rating. Five star funds had nearly double the length of manager tenure as one star funds. Likewise, five star funds had an average Sharpe ratio which was over 200% larger than one star funds. Portfolio yield and turnover ratio, among large cap funds, are not clearly correlated with star rating. However, it is evident that single star funds tend to have lower yield and higher turnover than higher rated funds.

The relationship between star rating and number of holdings is intriguing. Five star funds and one star funds averaged about 70 holdings per fund, yet had vastly different outcomes in terms of performance. One intuitive conclusion is the impact of the manager(s) in security selection. Obviously the number of securities in a portfolio is of less importance that the quality of the securities owned. "Security in numbers" only goes so far. On a technical note, the reason for the high average number of holdings among 2 and 3 star large cap funds is due to S&P 500 clone funds which have 500 +/- holdings.

Finally, expense ratios were highest in the 5 star and 1 star funds. In one case (5 star funds) you get what you paid for. In the case of single star funds, caveat emptor. Funds that experience a period of poor performance often lose assets as investors bail out. As a result, some fund companies elect to raise the expense ratio in order to maintain a certain level of revenue from the fund, thus adding to the coefficient of drag on the fund's future returns.

Among mid caps (Figure 3), four and five star funds had significantly higher return, higher Sharpe ratios, more manager tenure, higher yield, lower turnover, and lower expense ratios. Unlike large caps, the relationship between standard deviation of return and star rating is quite consistent - funds with a higher star rating have lower standard deviation of return. A negative linear relationship was also quite evident between star rating and turnover ratios among mid caps. As star rating declines turnover ratios increase. The same relationship was true for expense ratio.

Interestingly, as also observed among large caps, 5 star and 1 star mid cap funds had about the same number of holdings (in both cases the smallest average number of holdings among all mid cap funds). This reminds us of a fundamental reality in portfolio theory - the smaller the number of holdings in a portfolio the greater the impact each holding has on the portfolio return. A small portfolio of successful holdings is the best of all worlds. Conversely, a small portfolio of poorly performing securities is the worst of all (mutual fund) worlds. This phenomena may be the underlying reality when comparing the performance results for 5 star and 1 star funds.

One major difference between mid caps and large caps is that over half (54%) of the mid cap funds were assigned a star rating of 4 or higher compared to only 14% of large cap funds with a star rating of 4 or 5. Again, this is strictly a function of the particular three year period being analyzed in which mid and small caps generally outperformed large cap funds.

Finally, the small cap fund report card (Figure 4). Sixty-eight percent were assigned 4 or 5 stars. Star rating and 3 year annualized return have a unmistakable positive linear relationship. As star rating goes down so does average return. Star rating and standard deviation of return is most clearly correlated among small cap funds. The relationship is linear and negative. As star rating declines, standard deviation of return increases. In fact, the average standard deviation of return for the 1 star funds was nearly 90% higher than among 5 star funds. As with large and mid caps, small cap 1 star funds have lower Sharpe ratios, lower yield, higher turnover, and higher expense ratios.

Small cap funds which were given five stars had managers with more tenure. Unlike large and mid caps, 4 and 5 star small cap funds tended to have the largest number of holdings. Single star small cap funds had a considerably smaller average number of holdings.

So, when an investor hitches up to a 5 star fund, what are they really getting? The answer depends to some degree on whether it's a large, mid, or small cap fund. But, in general, a five star fund will have:

higher return (regardless of cap)

lower standard deviation of return (i.e. less return volatility) particularly among mid and small cap funds

higher Sharpe ratio

a manager with more tenure

higher yield

lower turnover (primarily among mid and small cap funds)

lower expense ratio (primarily among mid and small cap funds)

Users of the star rating system would do well to remember that funds are "star rated" in large groups, such as all domestic equity funds in one group, all non-U.S. equity funds in one group, etc. As such, certain types of funds (large cap, mid cap, small cap) will likely dominate the high star rankings more than another group based upon which sector of the equity market has been performing well. Therefore, planners or advisors who screen funds using the star rating may want to first segregate equity funds by market cap before proceeding. This, however, will be a short-lived solution. Beginning next month, Morningstar rolls out a modified star-rating system which will rate funds across approximately 50 categories, rather than just four broad asset classes. This "shrinking-of-the-radar-screen" promises to more equitably pit peer against peer. In addition, Morningstar will unveil a modified risk-adjusted measure as well as methodology to penalize funds which change investment styles often.

The star rating system will likely undergo changes and modifications on a continual basis with a primary goal being to highlight funds which excel within their peer group. However, regardless of upgrades and improvements, Morningstar suggests that the star rating is designed as a "first-level screen". Based upon the findings of this study, additional screens to consider (after having initially selected out only 5 star funds, for example), could include the variables listed above (standard deviation of return, Sharpe ratio, manager tenure, yield, turnover, expense ratio). Funds which make it though the star screen, and the additional screens, just might be the funds which are "book smart" AND "street smart". Stars among stars, as it were.

Let the star search begin.

Figure 2. Large Cap Funds

Figure 3. Mid Cap Funds

Figure 4. Small Cap Funds