By Mark Giberson
Published December 1991
St. Thomas University’s director of admissions Evelyn Sweezey was standing at a table in the gym of Miramichi Valley High School in Newcastle, New Brunswick, ready to meet with prospective students and their parents. It was the school’s annual university fair and scattered around the gym floor were other tables, staffed by university representatives from across Canada, all of them anxious to make a good impression with the local crop of would-be university students.
One of the students, with parents in tow, moved a step closer to Mrs. Sweezey’s table. “You can forget about going to St. Thomas,” said the kid’s mother. “It’s one of the worst universities in the country. I heard it on the news this afternoon.”
For my part, I heard it on the news that morning. It was October 15, and unbeknownst to me, Maclean’s magazine the day before had released a story to news organizations across the country providing highlights from an upcoming issue which ranked the undergraduate arts and science programs at 46 universities in Canada. According to the report carried by CBC radio in Fredericton, St. Thomas had placed among the bottom five.
By the time I reached my office, St. Thomas University president Daniel O’Brien had already had a call from another local radio station and from the Fredericton Daily Gleaner. It was difficult to respond to the story. We hadn’t seen the survey results or the criteria on which they were based. It wasn’t until a few hours later, after I had received a fax of the Maclean’s news release, that Dr. O’Brien and I could go over the criteria and get down to the business of damage control.
News media across the country were quick to pick up on the Maclean’s story. The Ottawa Citizen carried the headline “Carleton’s big headache”, a not-so-veiled reference to Carleton University’s 44th place ranking in the survey. “U of C’s low rating has staff worried”, said the Calgary Herald. The St. Catherine’s Standard was less circumspect, “Ranked among bottom 10 schools surveyed: Brock bristles at Maclean’s rating”. Surprisingly, perhaps, the Globe and Mail’s national edition ignored the furor. The paper’s Metro edition carried a tiny headline in its Toronto In Brief column, “U of T ranked 4th”.
Writing in the November 4 issue of Maclean’s, editor Kevin Doyle says, “The Maclean’s survey was never intended as the final, definitive ranking on the best universities in Canada.” The survey’s modest ambitions seem to have been lost on a lot of headline writers, however.
The Maclean’s survey will be around for some time to come. The October 21 issue in which it appeared was a sellout and the magazine has announced plans to reprint the section and make it available at news stands. Maclean’s staff have also indicated that the magazine hopes to make the survey an annual feature.
Diane Brady, an assistant editor at the magazine who was largely responsible for overseeing the project, says Maclean’s undertook the survey to help prospective students and their parents make informed choices about where to study.
“Our intent in doing this exercise was to basically get information out in the public domain that really has not been readily available in a user-friendly way,” says Ms Brady. ”We were taking a snapshot of certain aspects of universities … We were basically doing what any potential student does … which is to sit down and ask some basic questions.”
What has angered many members of Canada’s university community are the questions Maclean’s chose to ask in ranking the institutions. Stuart Smith, head of a commission of inquiry which recently examined the state of higher education in Canada for the 89-member Association of Universities and Colleges of Canada, says the Maclean’s criteria were biased in favor of old, established, well-known institutions. None of the criteria, says Dr. Smith, “speak to the matter of what actually happens to students once they get there.”
Says Dr. Smith, “An inner-city university that prides itself on being open at all hours and accepting people with unusual credentials is obviously fishing in a different pond from those that are coasting on reputations and picking and choosing those with the highest marks from high school. It could be argued that the inner-city university is doing more for the nation’s competitiveness and more for the social peace of the nation and probably more in the way of pure value-added education than the famous university. Maclean’s has tarred certain universities as inferior based on criteria which I don’t think are in any way fair to what these universities are about.”
David Johnston, principal of McGill University which ranked first in the Maclean’s survey, is more forgiving. “I think the study has a great deal of validity,” says Dr. Johnston. “It’s important that media of all kinds enter into their own assessment of the quality and performance of our universities using methodology that is appropriate for the measurement. The effort to make comparisons is helpful. Whether this effort is the best that could be done on the first attempt is a legitimate question to ask.”
Peter Desbarats, dean of journalism at the University of Western Ontario, echoes Dr. Johnston’s comments. Prof. Desbarats says he has “a few quibbles” with some of the criteria used in the survey. For example, he questions whether the number of residence beds or student services budgets have much to do with academic quality. But, says Prof. Desbarats, “the exercise that Maclean’s went through was healthy and I think they did it reasonably responsibly.”
Meanwhile, Michael Cobden, director of the school of journalism at the University of King’s College in Halifax, says the magazine’s attempt to rank institutions was flawed from the outset. “Universities are very complicated and human beings are very complicated and when you put the two together and try to work out what sort of university would suit what sort of human being you have an enormously complicated equation. To try to reduce that to a table with a handful of criteria is absolutely ridiculous.”
A lot of high school guidance counsellors appear to agree with Prof. Cobden’s assessment. Says Sandra Lass, a guidance counsellor at Riverview High School near Moncton, New Brunswick, “Students often ask very simply, ‘Which is the better university?’ There are so many criteria. ‘It depends’, is usually the way I answer a question like that.”
Ms Lass says she intends to keep a copy of the survey in her office. “I want to be able to point out to students that there are flaws in the ranking and that there are different ways of looking at it. It’s a good conversation starter, but beyond that I don’t think it has overwhelming validity.”
In Ottawa, students at Lisgar Collegiate Institute appeared oblivious to the controversy. Says Lisgar guidance counsellor Doug Arrand, “Not one of our students has raised the poll to the university representatives who have visited the school, even though I have a copy of the magazine in my office.”
Mr. Arrand’s assessment of the survey’s impact with his students should be welcome news to admissions personnel at nearby Carleton. “Carleton’s reputation with respect to the Lisgar community is based on far deeper and enduring relationships,” he says. “I doubt that the Maclean’s story will have much impact.”
What has angered many members of Canada’s university community are the questions Maclean’s chose to ask in ranking the institutions.
At Queen Elizabeth High School in Halifax, the Maclean’s article has been “a pretty hot item”, according to head of guidance Maureen Coady. But Ms Coady, like many of her counterparts across the country, doesn’t put much credence in the ranking exercise. “We tend to say to students, ‘You can get a pretty good bachelor of arts or bachelor of science degree at any university in Canada.’ We’re fortunate in that way. There’s not that much discrepancy between programs.”
Ken Sewell, director of guidance at Campbell Collegiate High School in Regina, agrees. “I don’t think there’s a decent university in Canada that doesn’t provide an adequate undergraduate degree. To rank universities by number is a little ridiculous.”
Ontario’s Minister of Colleges and Universities Richard Allen says the survey has “some really serious shortcomings.” Says Mr. Allen, “there’s a relatively even performance level at most universities, so ranking the institutions one against the other gets you into a rather invidious, competitive mode which I don’t think you learn a great deal from.” He says that unless there are significant changes to the criteria that the magazine uses to indicate quality, it’s unlikely that universities will be willing to cooperate in a “measurement system in which a lot of them simply can’t win at anything.”
The idea that universities might compete for better positions in future Maclean’s rankings worries Stuart Smith. “After I spent a year and a half coming up with criteria that I think are crucial, I hate the notion that, at the very same moment, a bunch of other criteria are now going to be bandied about on campus for fear that if universities don’t pay attention to the Maclean’s criteria they’re going to look bad the next time Maclean’s issues ratings.”
Dr. Smith offers a prescription – universities should be more forthcoming with information that has a direct bearing on quality. “If universities were to make public the kinds of data that I have called for, then they could avoid all of this nonsense,” he says. “The reason they’re subject to this kind of uninformed ranking is because they haven’t taken the steps to inform the public of the things that the public has the right to know.”
In the meantime, my colleagues in the admissions office continue to cope with the fallout from the university’s 43rd place ranking in the Maclean’s survey. Says Evelyn Sweezey, “Most of the students I’ve spoken with have never seen the Maclean’s article, but they’re certainly familiar with the headlines about the story. We’ve had a good reputation for offering a strong arts program, but there’s no doubt in my mind that this article has damaged that reputation. We’re going to be living with the consequences of this piece for a long time to come.”
Mark Giberson is director of university relations and alumni affairs at St. Thomas University and a member of the executive board of the International Association of Business Communicators.