Alex Usher is the president of Higher Education Strategy Associates.
On Thursday, several media outlets published a story about a report by academics from the University of Toronto and Brock University, which purported to find evidence of a “brain drain” in Canada. Unfortunately, the focus was on the headline results and the conclusions, and not on the actual, highly problematic methodology.
The report, entitled Reversing the Brain Drain: Where Is Canada’s STEM Talent Going?, purports to examine brain drain among Canadian STEM graduates, and at many reprises suggests that this is a national problem affecting all of Canada. But in fact, the data covers only a tiny slice of Canadian STEM graduates because of three methodological choices.
First, instead of covering all universities, the data covers but three: the University of Waterloo, the University of Toronto and the University of British Columbia, which because of their recognized high quality in tech fields and (in UBC’s case, at least) proximity to Microsoft’s and Amazon’s headquarters and their high proportions of international student enrolments, might be expected to have higher-than-average rates of students leaving the country. Yet there is not a word in the report addressing whether it makes sense to generalize across the country from these three examples.
Second, the study design actually excludes most STEM graduates. Of the 30,000-odd STEM graduates produced in 2015 and 2016, only 22 programs covering 6,600 graduates were included. The report does not explain why these fields were chosen, but even a casual glance shows that mostly what is included are IT-related fields (math, computer science, computer/electrical engineering) and mostly what is excluded are more traditional science/engineering fields (civil/mechanical engineering, physics, biology, environmental sciences etc.). So while the results might be generalizable to a few specific fields, they are definitely not generalizable to STEM as a whole.
Third, for what one assumes are cost reasons, the study examines not all graduates of (mainly) tech programs, but only the half which use LinkedIn and which are therefore easy to track. This is of course a massive potential source of sample bias, the kind of thing they warn about in Intro to Methods classes, because there is no guarantee at all that those who use LinkedIn have similar mobility patterns to those who do not.
So, what the study actually finds is that LinkedIn-listed 2015 and 2016 grads from tech programs at UBC, Toronto and – especially – Waterloo do have a reasonably high propensity to leave Canada after graduation, with 19 per cent, 25 per cent and 41 per cent, respectively, choosing not to stay and for the most part choosing the United States as a destination. A better report title would therefore have been something along the lines of “top young tech talent from top Canadian universities gets recruited by top American tech firms.” It wouldn’t have created much policy buzz – more likely it would have been used as marketing material by the universities – but it would have had the merit of being accurate.
Unfortunately the authors chose a different route, one guaranteed to get more media take-up because it could generate clickbait headlines such as “Canada has a STEM graduate brain drain.” These kinds of headlines might be very convenient to outfits such as the Canadian Council of Innovators, whose ongoing quest is to get government to help them recruit more tech grads in the face of competition from American tech companies, who just happen to pay their employees about $50,000 per year more than Canadian ones do. But Canada isn’t just three universities, STEM isn’t just tech and “graduates” includes more than just the ones who use LinkedIn.
To pretend otherwise, to casually wave aside any caveats that spring from the data, is just bad social science. It’s not that the conclusion is necessarily wrong, it’s that the actual evidence is far too slim to support the claims being made. Policy-makers should handle this study with extreme caution.
https://www.theglobeandmail.com/opinion/article-is-canada-really-facing-a-brain-drain/
Recent Comments