In the Eye of the Evaluator: Morningstar vs Value Line

In the Eye of the Evaluator: Morningstar vs Value Line

Download Acrobat Reader

Quiz Question:

True / False: Mutual fund rating systems are pretty much the same.

he answer, at least if comparing Morningstar Principia against Value Line Mutual Fund Survey, is false. While they both collect a boatload of useful data, they also have very different systems for highlighting top rated funds.

Both firms have a rating measure which is designed to "cut-to-the-chase" in isolating top rated funds in terms of historical risk-adjusted return. Morningstar's approach is the widely-known and often-debated Star Rating system in which 5 stars is the best rating and 1 star the worst. Value Line uses a system termed "Overall Rank", where 1 is the best and 5 the worst. While not a significant issue, the systems do start off using ranking orders which are opposite each other. Perhaps a quick overview of each rating system will be helpful.

Morningstar's Star Rating methodology was modified significantly in June, 2002. Funds are now rated against only those funds in their specific Morningstar category (of which there are 20 U.S. equity categories, 11 non-U.S.-equity categories, 13 taxable bond categories, and 15 municipal bond categories). Morningstar domestic equity categories include: Large Growth, Large Blend, Large Value, Medium Growth, Medium Blend, Medium Value, Small Growth, Small Blend, and Small Value. Morningstar also includes several other domestic equity categories, including: Specialty Heath Care, Specialty Technology, Specialty Utilities, Specialty Communications, Specialty Financials, Specialty Real Estate, Specialty Natural Resources, Convertible Bond, Bear Market, Conservative Allocation, and Moderate Allocation.

Prior to June, 2002 funds were placed in four primary asset-class buckets (U.S. equity funds, non-U.S. equity funds, taxable bond funds, and municipal bond funds) and then assigned a star rating. According to Morningstar, "The biggest impact of this change is that funds are less likely to receive a high (or low) rating due to a market tailwind. For example, under the previous methodology, persistent outperformance by the value investment style resulted in high ratings for most value funds, while growth-oriented funds were likely to receive lower ratings. However, the change also means that some extremely risky funds will now receive 5 stars (those with the best risk-adjusted return within their category), which was difficult under the old methodology." The new methodology also evaluates risk differently in that downside risk is more clearly targeted.

Only the top 10% of funds in each category are assigned a score of 5 stars based upon their risk-adjusted return. The next 22.5% of funds receive 4 stars, while the middle 35% receive 3 stars. The next 22.5% of funds receive 2 stars and the bottom 10% of funds receive 1 star. All loads are taken into account.

Morningstar actually calculates a maximum of four different star ratings for each fund. Funds with at least 3 but less than 5 years of performance history are assigned a 3 year star rating. Funds with at least 5 but less than 10 years of performance are assigned a 3 year and a 5 year star rating. Funds with 10 or more years of performance are assigned a 3, 5 and 10-year star rating. The fourth star rating is the Star Rating of which we are speaking. Perhaps it might be thought of as the aggregate or overall Star Rating. The aggregate star rating (i.e. the one cited in marketing materials) is the same as the 3 year rating for funds with more than 3 years but less than 5 years of history. For funds in the next age bracket (more than 5 but less than 10 years of performance) the overall star rating represents 60% of the 5 year rating and 40% of the 3 year rating. For funds with more than 10 years of performance history the overall star rating is weighted 50% on the 10 year rating, 30% on the 5 year rating and 20% on the 3 year rating. These guidelines apply to funds which did not change category during the time periods involved. Long-term performance is weighted less heavily for funds that change category.

The approach taken by Value Line is different. The Overall Rank (where 1 is the best score and 5 the worst) is determined by 3 variables: 1 Year Growth Persistence score, 5 Year Growth Persistence score, and 3 year Sharpe Ratio. Growth persistence is defined by Value Line as "a measure of the consistency of a fund's positive relative performance. Funds that score best are those that most steadily outperform their equity or fixed income peers. No regard is given to the magnitude of the outperformance." The overall rank score is calculated for each fund within three broad categories: equity and partial equity funds, taxable bond funds, and municipal bond funds. The top ten percent of funds in each category receive an Overall Rank score of 1 (best). The next 20% of funds receive a score of 2, the next 40% receive a 3, the next 20% receive a 4, and the bottom 10% of funds receive a score of 5.

Enough review, on to the findings. In figures 1-4, Morningstar's 5 Star selections are listed side-by-side with Value Line's Overall Rank 1 funds as of December 31, 2003. Morningstar data were obtained from the January 2004 release of Principia. Value Line data were gleaned from the January, 2004 release of the Value Line Mutual Fund Survey. The purpose of this article is to compare and contrast each firm's winners in four U.S. equity categories: Large Cap Value, Large Cap Growth, Small Cap Value and Small Cap Growth.

After winnowing out redundant portfolios (i.e. removing all but one share class among multiple share-class load funds) and selecting only U.S. equity funds with at least 5 years of performance data both data sets were further narrowed by selecting only 5 star funds (in the case of Morningstar) and Overall Rank =1 (in the case of Value Line). There were 194 Morningstar funds remaining and 283 funds from Value Line. The final filter separated out only those funds with 40% or more of their portfolio in each of the four equity styles being analyzed (LV, LG, SV, SG). In the large cap value category, there were 18 Morningstar funds and 20 from Value Line. Twenty large cap growth funds emerged from the Morningstar data, none from Value Line. The difference in the small cap value category was stunning: 5 funds from Morningstar and 86 from Value Line. Finally, there were 12 small cap growth funds from Morningstar and 28 from the Value Line database.

Large cap value funds which earned each firm's highest rating are listed in Figure 1. You will notice that some of the Value Line funds appear (by virtue of their name) to be mid cap value funds. Value line reports portfolio composition (as a percentage) for large cap value, large cap growth, small cap value and small cap growth categories. Absent are the mid cap value and mid cap growth categories which are present in the Morningstar database (in addition to the four already mentioned). Thus, with two less equity style "buckets", mid cap funds in the Value Line database often end up being aggregated with large cap funds. It should be noted that Value Line does provide mid cap equity style coding in the "Objectives" column (MV, MB, MG).

There was no overlap in Figure 1 between the Morningstar 5 Star funds and the Value Line Overall Rank 1 funds. While the specific funds given a top rating are different between the two firms, the number of funds (18 from Morningstar and 20 from Value Line) are very close. The average returns for the 18 Morningstar funds over 1, 3 and 5 years (as of 12/31/03) were 27.7%, 6.2% and 8.8%. The Value Line funds averaged 35.7% over one year (2003), 7.0% for 3 years (2001-2003), and 10.4% over 5 years (1999-2003).

In the large cap growth category (Figure 2) Morningstar funds stood alone. No large cap growth funds in the Value Line dataset were given an Overall Rank of 1. The past several years have been tough for this particular equity style. The 20 Morningstar large cap growth funds had an average return of 31.6% for the prior year (2003), but had an anemic 3 year average annualized return of -1.2% and an average 5 year return of only 4.9%.

The vast differential in the number of small cap value funds assigned a top rating (Figure 3) further illustrates the disparity between the rating methodologies at Morningstar and at Value Line. Five Morningstar funds received a 5 star rating vs 86 Value Line funds that were given an Overall Rank of 1. In terms of raw performance, small caps have outperformed large caps over the past five years. In fact, the average 5-year return for the five Morningstar funds was 19.9% and 12.8% for the 86 Value Line funds. Figure 4 lists the small cap growth funds given the highest rating by each firm. Value Line's offerings (28) more than doubled the number from Morningstar (12). However, for the first time, there was overlap between the two rating systems as noted by the five funds in bold, italicized font. Small cap growth stocks, as a sector, performed well last year. The Wilshire Small Cap Growth Index was up 43.9% in 2003. The 12 Morningstar funds averaged a 57% return in 2003, and a 22.9% return over 5 years (1999-2003). The 28 Value Line funds were equally impressive with an average one-year return of 56.1% and a 5 year return of 17.5%.

The disparity in number of funds given the highest rating among large cap growth, small cap value, and small cap growth funds (Figures 2-4) reveals several important differences in the two different rating systems.

First, Morningstar dishes out 5 star ratings to the best fund(s) in each category regardless of how that particular category has been performing. Value Line's rating system, on the other hand, aggregates funds into three broad categories and then sifts out the best funds on a risk-adjusted return basis. If a certain equity sub-sector, say large cap growth, has done poorly it is unlikely any large growth funds will receive an Overall Rank of 1 even though there may have been significant performance differences among the large growth funds themselves. The Value Line approach rewards the strongest funds - period. The Morningstar approach rewards the strongest funds in each of many equity sectors even if that sector has lagged, which is precisely what the changes to the Star Rating system June 2002 were designed to accomplish.

Second, Morningstar's Star Rating takes into account a longer time period (10 years) if a fund has been around that long. Value Line does not look back more than 5 years. This particular methodological difference explains much of the disparity among large cap growth funds in Figure 2. Going back 10 years picks up a lot a stellar performance in the late 1990's.

Third, Value Line uses one year Growth Persistence in their calculation, hence the Overall Rank is more influenced by recent, short-term performance than is Morningstar's Star Rating which does not assign a star rating to funds with less than three years of performance. This methodological difference very likely contributes to the vast disparity in small cap value funds in Figure 3. Many small cap value funds had impressive performance in 2003 (the Wilshire Small Value Index in 2003 was up 51.2%). Therefore, as Value Line's Overall Rank is more influenced by recent performance it is not surprising that Value Line assigned a high rating to a greater number of small cap value funds.

Finally, as already referred to, Value Line's database does not specifically include the percentage of a fund's portfolio allocated to mid cap stocks whereas Morningstar does. Furthermore, unlike Morningstar, Value Line does not include a "blend" category for large and small caps (though Morningstar refers to it as "core"). Value Line has four buckets in which to report the holdings (by percentage) of each equity fund: large value, large growth, small value, and small growth. Morningstar, on the other hand, has nine buckets: large value, large core, large growth, mid value, mid core, mid growth, small value, small core, and small growth. Thus, a fund that might be a mid cap fund in the Morningstar database (according to the percentage of the portfolio allocated to mid cap stocks) has to be classified as a large cap or a small cap fund in the Value Line database. Furthermore, a small cap blend (or core) fund in Morningstar Principia will end up as a small value or small growth fund in the Value Line database in spite of the fact that Value Line has a coding system that includes B (for blend) in its Objectives column.

The outcome of this particular difference is that the Value Line database aggregates funds on the basis of percentage of equity style in the portfolio into four categories whereas Morningstar segregates into nine. For instance, a fund that is categorized as small cap core by Morningstar (and thus not be a candidate for inclusion in Figure 3) is lumped into the small cap value bucket by Value Line. This contrast in methodology contributes to the huge difference in number of funds reported in Figure 3.

To borrow an analogy from basketball tryouts, the Value Line Overall Rank system rewards the best players regardless of height or position. Morningstar's Star Rating system rewards the best players at each position, even if the guards can't dunk as well as the forwards. Whereas Value Line's approach employs comparison on an absolute basis, Morningstar's is a comparison on a relative basis.

If isolating the hottest funds is the goal, Value Line's Overall Rank rating is appropriate. If identifying the best performing funds in each equity (and debt) category is the goal, Morningstar's Star Rating will be more useful. But, either way, there is an important caveat: both rating systems should not be used as the one and only selection filter. As Morningstar puts it, their star rating "serves as an initial screen, the rating is only the first step toward a more comprehensive evaluation…The star rating is best used as an initial screen to identify funds worthy of further research, those that have performed well on a risk-adjusted basis relative to their peers."

Figure 1. U.S. Equity Large Cap Value Funds as of 12/31/2003

Figure 2. U.S. Equity Large Cap Growth Funds as of 12/31/2003

Figure 3. U.S. Equity Small Cap Value Funds as of 12/31/2003

Figure 4. U.S. Equity Small Cap Growth Funds as of 12/31/2003

____________________________________________________________________________________
Craig L. Israelsen is an Associate Professor in the Department of Consumer and Family Economics at the University of Missouri-Columbia (http://www.missouri.edu/index.cfm) where he teaches courses in Personal Finance and Family Living. He holds a Ph.D. in Family Resource Management from Brigham Young University. He received a B.S. in Agribusiness and a M.S. in Agricultural Economics from Utah State University. Primary among his research interests is the analysis of mutual funds. He writes monthly for Financial Planning magazine.