News //

Spanking the rankings

Max Hall investigates flaws in University ranking systems.

Illustration by Laura Precup.
Illustration by Laura Precup.
Illustration by Laura Precup.

Rankings are easy to love. We turn to the guidance of music charts, Buzzfeed’s “Ten greatest crazes of the nineties”, and all manner of other lists to reduce entire subject areas into an easily digestible format. This appeal is not lost in the world of tertiary education, where yearly rankings have come to dominate the discourse surrounding the quality of our universities.

Since the first global list was published in 2003, the number of rankings devised by media and consultancy groups eager to claim their share of the growing industry has grown to fifteen widely recognised lists by 2013.

A report by the European Universities Association found that “rankings enjoy a high level of acceptance among stakeholders and the wider public” largely due to “their simplicity and consumer-type information”. Academics and consumer groups have only recently begun to raise concerns that the popularity of published rankings may disproportionately influence choices made by high school students, affecting the management decisions of universities without regard for the problems inherent to current ranking systems.

Little information about the methods used to compare universities is provided publicly, calling into question the transparency of results. The data used is often given on a voluntary basis by the tertiary institutions themselves. Indeed, the details that are available indicate a worrying disconnect between the frequently changing methodologies used and the outcomes they claim to measure.

For instance, the Academic Ranking of World Universities assesses the “quality of faculty” by counting the number of Nobel Laureates employed at the university. This system also grants 50 percent weight to research output in the final aggregate.

From the perspective of an enrolling high school student, these criteria reveal nothing about the quality of undergraduate education they are likely to receive. The research conducted by a lecturer has little to do with their ability to teach a class, just as the likelihood of mere undergrads being allowed anywhere near well remunerated Nobel Laureates is effectively zero.

Such inconsistencies abound among other popular lists. The Times Higher Education rankings gauge “teaching quality” using nothing more than the ratio of students to faculty members, once again heavily biasing results towards smaller, research focused universities. Other factors including the number of PhDs awarded in the past year, and an obscure survey of “faculty reputation” conducted amongst “experienced scholars” at other universities bear heavily on the final list.

The fatal flaw of current rankings is simple: so long as ranking criterion and the weight assigned to them are purely arbitrary, the subjective interests of groups compiling lists will continue to masquerade as an objective assessment of merit.

The most disturbing outcome of this, perhaps, is recent evidence that rankings now form a key part of the management of universities, where they are used to justify funding, dictate enrolment strategies, and determine the bonuses of executives. This inevitably leads to a focus on those areas most likely to improve rankings, namely research. At USyd, the number of research graduates has risen steadily over the last five years in tandem with a marketing approach that heavily emphasises expanded research, particularly in medical science.

The extent to which consideration of rankings has influenced this expansion is unclear, but the university undoubtedly recognises the usefulness of global lists as a marketing tool. Press releases celebrating ranking gains have been published an average of three times a year over the last five years, legitimising lists as indicators of the quality.

Both the methodology of rankings and the way they are seized upon by the public relations operations of our institutions remain significant barriers to a better discussion about the needs and priorities of universities. As universities continue to face financial strain in light of $2.3 billion of government cuts to tertiary education it is crucial that the pursuit of a better place in global ranking not take place at the expense of the average university student.

Filed under: