Rankings: Imperfect and Important

I'm sure that many of you would argue that rankings of business schools are not worth the paper on which they are printed. Despite what we may think of the various methodologies and measures, the reality is that rankings remain the most widely used barometer of a business school's reputation. A business school's stock in the real world rises and falls with where it stands in rankings by Business Week, U.S. News, the Economist, the Financial Times, and the Wall Street Journal, to name just a few. While these metrics may not always reflect Michigan's true value - either our current state or our promise for the future, we all know that we must take the implications of these rankings seriously. Our rankings are important to prospective students, faculty, recruiters, alumni, corporate sponsors, university administrators, and many other important constituencies.

So, we carefully track our rankings, where we stand in each poll, and what makes each one tick. This year, the Full-Time MBA program is 6th in Business Week, 10th in U.S. News, 11th in the Economist, 16th in the Financial Times, and 1st in the Wall Street Journal. The BBA program is ranked 3rd in the primary poll for undergraduate programs by U.S. News. Executive Education is ranked 3rd in Business Week. A major determinant of how well we fare in a given poll is the degree of fit between a ranking's methodology and our chosen strategy as a school. Given our broad-based, general management approach, we naturally tend to do better in some rankings than others. In particular, we generally do well where recruiter opinions are a major input (Wall Street Journal, Business Week) and not as well when what really matters is placing a large percentage of students in one or two very high paying industries (Financial Times, Forbes, and to a lesser extent U.S. News).

Personally, for a wide variety of reasons, I don't like to see us outside the top ten in any listing of business schools or individual programs.

I am not suggesting that we dissect the rankings with the aim of reverse engineering our strategy. Obviously, it would be dangerous to set our course solely on the basis of maximizing our rankings. In particular, those rankings which rely heavily on salary and placement data are inconsistent with our general management strategy. If we try to maximize our position in these polls, it could lead us to recruit students whose primary interest are fields with the highest starting salaries, persuade students to take jobs in those industries once they are here, and focus our resources on training students for entry level positions in those industries as well as attracting those recruiters to campus. We would probably even concentrate these efforts geographically, focusing on areas of the country and parts of the world where salaries are higher due to the greater cost of living. Such distortions are not good for our students, our faculty, and the vast majority of companies that come to us seeking management talent. In a broader sense, the world's economy and society in general would be negatively impacted as well. That's not to say that we don't want to strengthen our program and placement record in such areas (we do!), only that we wouldn't want to do so at the expense of our overall strategy.

While we may not want to re-engineer our school to win in the polls, I think it is appropriate and necessary for us to consider what useful feedback we can glean from each ranking and how we can bring what we learn to our continuous efforts to improve our community. In other words, we need to view such measures as an opportunity to reflect on whether we are being successful in pursuit of our goals and to identify potential areas of improvement - especially if we are not performing as well as we would like in areas that are central to our overall strategy as a school.

A good illustration of how we work to improve our position can be found in the way that the School has responded to the 2002 Business Week rankings. After the BW results came out in October 2002, Dean Dolan and his senior staff followed up on the results with Jennifer Merrit, Business Week's B-Schools reporter. Although BW does not share the actual data with Schools, they are willing to provide verbal feedback on a school's performance. The BW student survey consists of questions on overall satisfaction, as well as specific aspects of the curriculum, teaching, recruiting, and the overall environment of a school. Each question receives equal weighting in calculating the overall rating of student satisfaction with a school. Merrit walked us through our score on each question, indicating whether we were above, at, or below what she referred to as 'the pack' - her term for the top 30 schools in the final BW ranking. She also told us whether our score on a given question was up, down, or about the same as in the last survey.

Merrit told us that we received our highest scores in areas such as our practical, broad-based approach to business education. Efforts to include topics such as ethics, leadership, teams, and diversity in the program were successful. The number, diversity, and quality of firms that recruit here in conjunction with the placement office's help with matters such as interview training, negotiating strategy, and resumes have helped us tremendously. Furthermore, the availability of faculty outside the classroom adds to our strength. Our biggest improvement came from the question that pertains to the extent to which the learning environment is enhanced by the use of technology (e.g., the wireless network). The major areas in which we fell were the core curriculum and the quality of instruction.

We knew that these criticisms were warranted and consistent with our own internal data and analysis. A few months before the 2002 rankings appeared, the Dean had charged a group of faculty and students to review the core curriculum and teaching, and also to recommend actions we could take to improve. The Business Week results provided added urgency to the Dean's charge and the entire community participated in a tough, introspective look at the core curriculum. The result is the first major overhaul of the core curriculum approved by the faculty in 25 years and important changes in staffing. All of these changes were made largely in response to student feedback, but there is no doubt that the BW results influenced both the nature and extent of the changes. We are monitoring the roll out of the new core with equal care and effort. There are definitely areas where we are working to adjust for next year - smoothing the transition from work back to school, better integration within MLP, getting the right 'pace' in certain courses, creating more flexibility to take Finance 513 in Fall A, greater availability of the right set of electives in Fall B, making appropriate staffing changes for next year, and so on. Looking at the big picture, the experience has been generally positive. We've even seen an improvement in average course ratings from past years (from a long-term average around 4.2 to a new high of 4.4). As we all work together on making these adjustments, I know we can make the first-year experience even better.