The agreement is designed to collect quantitative information grouped under the following broad themes: a) student experience; b) innovation in teaching and learning excellence; c) access and equity; d) research excellence and impact; and e) innovation, economic development and community engagement. The collection of system-wide data is not a bad idea on its own. For example, looking at metrics like student retention data between years one and two, proportion of expenditures on student services, graduation rates, data on the number and proportion of Indigenous students, first-generation students and students with disabilities, and graduate employment rates, all can be helpful.
Where the plan goes off-track is with the system-wide metrics used to assess research excellence and impact: 1) Tri-council funding (total and share by council); 2) number of papers (total and per full-time faculty); and 3) number of citations (total and per paper). A tabulation of our worth as scholars is simply not possible through narrowly conceived, quantified metrics that merely total up research grants, peer-reviewed publications and citations. Such an approach perversely de-incentivises time-consuming research, community-based research, Indigenous research, innovative lines of inquiry and alternative forms of scholarship. It effectively displaces research that “matters” with research that “counts” and puts a premium on doing simply what counts as fast as possible.
How so? Because as universities are inevitably benchmarked and compared with one another on league tables, misguided conclusions about quality will be drawn, thus creating intense pressure to excel on these narrowly conceived performance indicators. A recent report by the Conference Board of Canada entitled Beyond Citations. Knowledge Mobilization, Research Impact, and the Changing Nature of Academic Work, concluded, “numbers alone, via citation metrics, evaluation framework scores, or even funds raised, paint only half a picture of research impact.”
Danger ahead
Even more alarming – and what is hardly being discussed – is how these damaging and limited terms of reference will be amplified when the agreement enters its third phase, SMA3, from 2020 to 2023. In this third phase, the actual funding allotments to universities will be tied to their performance on the agreement’s extremely deficient metrics. Here we can confidently predict that the assessment scheme will completely lose its way, as we have already witnessed in countries with similar national performance-based funding schemes like the Research Excellence Framework in the U.K., the Excellence in Research in Australia, and the Performance-Based Research Fund in New Zealand.
The one-size-fits-all “factory model” of knowledge creation, dissemination and measurement does a disservice to us all. First, not all researchers require funding to carry out their important and potentially ground-breaking work. Low-cost research methods should not be devalued or indirectly categorized as lesser. Second, to meet the demands of an audit culture set on counting quantity over quality, researchers will almost certainly experience greater pressure to “game” results through p-hacking or data dredging, or “salami-slicing” research into the least publishable unit.
Finally, it is beyond dispute that measuring research impact via mere citation counts is not an accurate measure of quality. As an illustration, consider the recent example of the article “The case for colonialism” published in Third World Quarterly (now withdrawn). Such an article’s citation count would have been through the roof, but only inasmuch as scholars would be citing it to critique it and dispute its claims and dubious quality.
As Yves Gingras, who holds the Canadian Research Chair in the History and Sociology of Science at Université du Québec à Montréal, recently noted in his book, Bibliometrics and Research Evaluation: Uses and Abuses, both the Council of Canadian Academies and the Higher Education Funding Council for England have concluded that “mapping research funding allocation directly to quantitative indicators is far too simplistic, and is not a realistic strategy.”
The Government of Ontario needs to rethink its SMA3 strategy now, in order to avert the damage that will follow this misguided policy. Now is also the time for Ontario university administrators to speak up and denounce this agreement before any further harm can result.
Marc Spooner is a professor in the faculty of education at the University of Regina.
Recent Comments
I am very happy to look your post. Thanks a lot and i am taking a look ahead to touch you.
Thanks for sharing the information. I found the information very useful. That’s a awesome story you posted. I will come back to scan some more.
Its like you read my mind! You seem to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a bit, but other than that, this is wonderful blog. A great read. I’ll certainly be back.
I’d like to be able to write like this, but taking the time and developing articles is hard…. Takes a lot of effort.
My brother suggested I might like this web site. He was entirely right. This post actually made my day. You can not imagine simply how much time I had spent for this info! Thanks!
Good points – – it will make a difference with my parents.
This is something that will need all of our combined efforts to address.
Substantially, the post is really the best on this laudable topic. I concur with your conclusions and will eagerly watch forward to your future updates.Just saying thanx will not just be enough, for the wonderful lucidity in your writing.
My brother suggested I might like this websiteHe was once totally rightThis post truly made my dayYou can not imagine simply how a lot time I had spent for this information! Thanks!
How come you do not have your website viewable in mobile format? cant see anything in my Droid.
Howdy, a helpful article for sure. Thank you.
I truly appreciate this post. I have been looking everywhere for this! Thank goodness I found it on Bing. You have made my day! Thank you again
Easily, the post is really the greatest on this laudable topic. I concur with your conclusions and will thirstily look forward to your future updates. Saying thank will not just be sufficient, for the wonderful c lucidity in your writing. I will instantly grab your rss feed to stay privy of any updates. Solid work and much success in your business enterprise!
The post is absolutely fantastic! Lots of great info and inspiration, both of which we all need! Also like to admire the time and effort you put into your website and detailed info you offer! I will bookmark your website!
Amazing! Your site has quite a few comment posts. How did you get all of these bloggers to look at your site I’m envious! I’m still studying all about posting articles on the net. I’m going to view pages on your website to get a better understanding how to attract more people. Thank you!
Unquestionably believe that which you said. Your favorite reason seemed to be on the net the easiest thing to be aware of. I say to you, I certainly get annoyed while people consider worries that they plainly don’t know about. You managed to hit the nail on the head. Will probably be back to get more. Thanks
This is really interesting, You’re a very skilled blogger. I have joined your feed and look forward to seeking more of your fantastic post. Also, I have shared your website in my social networks!
Mental game is so strong from both sides. WHAT A GAME!
When we look at these issues, we know that they are the key ones for our time.
I’ve read several good stuff here. Definitely worth bookmarking for revisiting. I surprise how much effort you put to make such a magnificent informative site.