Rank Rankings in 3-D

Ranking universities is pretty hard and perhaps somewhat pointless. The Times Higher Education has ranked McGill as 18th on the planet, up two spots from last year. This makes McGill the top-rated public school in North America, one spot ahead of Michigan. Um. Ok.

Of course, what matters here is what matters? How are the rankings produced?

• First, they total the number of papers produced and divide by number of faculty. Ok. Well, that really leaves open a lot of questions--do papers in Bob's Journal of Neuroscience count the same as in the top journal in the field? What about books? Do books count? Apparently not, which means that McGill might actually be under-rated because books are a big deal here at least in Political Science and probably throughout the Social Sciences (17th down a few spots from last year).
• Second, staff to student ratio for excellence in teaching. Again, this says something, but not that much as the ratio of bodies to bodies says little about the bodies involved. I think McGill is a great place to learn because the students are so great and one learns from one's peers as much or more than from one's profs. So, a measure here of selectivity might make sense. McGill does really well here, which surprises me since the ratios I see are horrible (600 to one in my one class and more than 35 majors per faculty spot [FTE]), but Poli Sci is incredibly popular here, so perhaps the rest of the campus has great ratios.
• Third, a key component is how international the university is: "we measure the proportion of overseas staff a university has on its books (making up 5 per cent of the total score) and the proportion of international students it has attracted (making up another 5 per cent)." Hmmm. I can see that there is something about international-ness that might have to do with quality, but still pretty strange. This would explain McGill's high ranking as it has always been less concerned with hiring Canadians than other Canadian schools (and the CRC program is a Canadian effort that also has increased the hiring of foreign folks such as myself). Also, Canadian schools have always been a good deal for Americans and McGill especially so since costs (non-tax costs that is) are quite low in Montreal.
• Fourth, 50% of the score is based on reputation--10% from employers, 40% from academics surveyed. Reputation is very sticky, so you will often see schools that are ranked highly even though the reality of their quality has declined.

Of course, this all begs the question of what "best" means. What do we want our colleges and universities to do? More than anything else, we want universities that are known for being best. Which is why McGill promotes its ranking, because, in part, that will shape next year's.

But if one really wants to know of a stat or indicator that will predict the performance of their students, then consider whether they have a successful ultimate frisbee team. McGill's men and women's teams do very, very well, so perhaps we should not be surprised that as an institution, McGill is highly respected.

Of course, the downside is that such rankings will cause some schools to become smug about their status and suggest that reforming old, dysfunctional and strange ways of administrating is unnecessary--that good rankings might lead to institutional arrogance and complacency. The good news, of course, is that reputations are, indeed, sticky so that the costs of such hubris are pretty small in the short term--to one's reputation.