Maclean’s survey skewed

In NewsLeave a Comment

Reading Time: 3 minutes

By Yohannes Edemariam

Ryerson’s poor showing in this week’s Maclean’s university rakings could be misleading, according to some sceptics.

Alex Usher, director of the Education Policy Institute in Toronto, says the main problem with the rankings is that they focus on inputs — such as how much money is spent on libraries or donated towards research — and ignore outputs, such as the quality of education graduates leave with.

“As a result, what you tend to get is a bias towards the size of the budget of the institution,” says Usher.

Ryerson was rated the fourth worst primarily undergraduate university in this year’s survey.

The rankings, which have caused controversy since they first appeared in the fall of 1991, divide universities in three categories: primarily undergraduate, medical doctoral and comprehensive (institutions with a mix of graduate and undergraduate studies).

Ann Dowsett Johnston, the Maclean’s editor who has been in charge of the ranking project since it was first launched, says some output indicators are considered in the rankings, but agrees more are needed.

One such indicator, new this year, is the student retention survey, which tracks the number of students who chose to return after their first year of study. Ryerson ranks 15th out of 21 schools in this category.

“We consider this a form of output measure as it shows student satisfaction with the learning environment,” says Dowsett Johnston.

Usher says that when it comes to hard numbers, output factors are difficult to quantify.

“The problem is that, apart from income, nobody knows how to measure what a fabulous grad is,” he says.

However, Maclean’s does do a “reputational winners” survey of high school principals, guidance councilors, CEOs and recruiters from corporations to see which institutions they favour in terms of “highest quality”, “leaders of tomorrow”, “most innovative” and “best overall.”

Ryerson ranks among the top five in each of these categories and, for the second year in a row, ranked second among undergraduate universities in producing leaders of tomorrow.

Dowsett Johnston says reputation scores indicate Ryerson may be a much better school than its overall ranking suggests.

Ryerson President Claude Lajeunesse agrees. He says that for a young university the school is doing well.

“I have always felt that what’s important about the Maclean’s rankings is our reputation,” he says. “If you look at the fact that we are being compared to universities that have been in existence for a lot longer than we have, it shows that we are doing quite well. We’ve been a university for 10 years and already we’re in the middle of the pack.”

But Usher argues that rankings still don’t do enough to accurately represent a school’s ability to educate students. He also argues that the weightings given to different categories are a result of subjective editorial assignments by Maclean’s and not based on any mathematical logic.

“There’s so many levels of abstraction in a study like this,” he says. “They’ve weighed things so that the libraries are worth 12 per cent and financial aid is worth 4.3 per cent — I mean that’s totally arbitrary.”

“The one thing we wanted to do is have 20 plus indicators,” says Dowsett Johnston. “The belief is that if you have enough indicators, no one indicator can unduly affect the rankings.”

Lajeunesse, who influenced the original Maclean’s survey, feels that with Ryerson, the rankings are like comparing apples with oranges.

“We’re being compared to universities which mostly have a liberal arts base and are much smaller than we are,” he says.

“It’s a subjective exercise with a veneer of numbers of top,” Usher says. “I don’t think they get to the heart of the issue in terms of quality of education.”

While he is not yet satisfied with Ryerson’s rankings, Lajeunesse said it would be unwise to use them as the sole basis for deciding where to study.

“I think it’s one of the tools that parents should look at to get information,” he says. “But I think anybody who would make a decision based solely on Maclean’s would be making a big mistake.”

Dowsett Johnston acknowledges there is work to be done, but she is reluctant to make drastic changes.

“Even those who have a lot of problems with out rankings would give us credit for spending so much time on this,” she says. “I am very proud of the fact that Maclean’s has taken the time to keep this up on behalf of Canadian students, parents, and the country.”

Leave a Comment