Social Policy in Ontario

A pollster’s painful reckoning…

Posted on June 20, 2011 in Governance Debates

Source: Globe & Mail — Authors: Michael Valpy

TheGlobeansMail.com – news/politics – ‘How could I have screwed up so badly?’ The Tell
Published Friday, Jun. 17, 2011.    Michael Valpy

On the night of Monday, May 2, Frank Graves, president of EKOS Research Associates, opened a bottle of Mer Soleil in his Ottawa home and settled in front of the television to watch the federal election results. He felt smug, very smug.

He had every reason to believe that he had surveyed as accurately throughout the 2011 campaign as he had in 2008, when he forecast that election’s almost exact outcome.

He had been exceptionally careful with his methodology and analysis. His numbers had been stable for three days. To avoid being caught by last-minute voter shifts, he had surveyed as late as Sunday, later than in any previous campaign.

He pleasurably imagined incoming kudos from his clients. Instead …

The moment he started seeing results from the Atlantic, he knew something had gone wrong and that his projections – another minority Conservative government with a vote tally that turned out to be nearly six percentage points below the party’s actual level of support – were way off.

“I thought, ‘Oh my god, this hasn’t happened before.’ I had this frankly sick feeling … somewhere between having people forget your birthday and being buried alive. How could I have screwed up so badly?”

The answer is that he tripped over Canada’s unique demographics: its peculiar mix of politically hot oldsters and tepid, disengaged young. From having been the most accurate in 2008, he had gone to being the farthest off-base in 2011. It was particularly painful for a social researcher accused periodically of anti-Conservative bias, although he was still within margin-of-error range of Canada’s half-dozen other major pollsters, who had all low-called the Conservative vote.

Election polling is like the Olympics for people in Mr. Graves’s line of work. It’s where they publicize their names, bolster their reputations. For the widely respected public-opinion researcher who so often has taken the lead in survey methodology, what happened to him in Election 2011 was akin to the president of one of Canada’s Big Five banks discovering he had advised his most valued customers to invest in a Ponzi scheme.

Mr. Graves apologized the next morning on the EKOS website, publicly conceding that he didn’t know what had gone wrong. He vowed to ferret out his error.

This week, he produced a 4,000-word analysis with the unexciting title Accurate Polling, Flawed Forecast.

It’s a polling wonk’s great read.

In addition to explaining how EKOS got its numbers wrong, it shines a light on Canada’s political and demographic culture. It raises worrying questions about the nature and future management of Canadian democracy. It illuminates generational differences in telephone use and offers a thoughtful commentary on polling responsibility.

And it points out that the EKOS president had the evidence in front of him all along that his polling forecast was flawed, but he chose not to believe it.

The report is pure Frank Graves, fitting the thinking of a man who is fascinated by the behaviour of his fellow citizens. He founded his now-multimillion-dollar company 30 years ago with $5,000 borrowed from his father when he was still in his 20s and decided to quit doctoral studies in sociology at Carleton University.

Since then, Mr. Graves has made a practice of going beyond other pollsters to tell the country’s story, tracking Canadians’ values, attitudes, generational conflicts and thoughts on everything from nationalism and the state to the role of emotion in politics, and class and age dispositions toward knowledge and morality.

The danger of polling cellphones

In a nutshell, what went wrong is that Mr. Graves created state-of-the-art methodology to effectively random survey the entire voting-age population. In doing so, he drew in a big segment of potential voters who use cellphones only – 15 per cent of the total, double the percentage in 2008. This segment of the voting-age population has been either excluded or underrepresented in conventional polling methodology. They tend to be younger and to belong to the more than 50 per cent of the electorate under the age of 45 who don’t vote.

Thus his nose count was accurate. But his forecast was off, because someone who says he will vote Liberal but doesn’t vote can’t be assigned the same weight as someone who says she will vote Conservative and does, indeed, vote Conservative.

So by solving one problem, Mr. Graves opened the door on another: the growing gap between the world of the voter and the world of the non-voter, two worlds with boundaries etched increasingly by telephone usage and, most important, age. Election 2011 revealed a voting fault line delineated by a generation gap.

On one side of the gap: Canadians over 45 enthusiastically favouring the Conservatives, with a likelihood of voting starting at about 60 per cent and rising with age to more than 80 per cent.

On the other side: younger Canadians generally disliking the Conservatives, but with a voting likelihood of at most 40 per cent, decreasing to about 30 per cent for the youngest electoral cohort, those under the age of 25.

The proportion of Canadians who vote has always increased with age, but the differential has never been as great as it is now. In addition, the values gulf between young and old probably has never been greater. And the demographic skewing of the population – proportionately so many older people – is almost certainly unprecedented.

When Mr. Graves retraced his steps, he found that if under-45 Canadians had voted in the same proportion as over-45 Canadians, there would have been no Conservative majority but more likely an NDP-led coalition.

He found there was no 11th-hour bolt of NDP-fearing Liberals to the Conservatives (if anything, there was the reverse: Liberals going to the NDP). When he separated the two worlds of voters and non-voters, he found that nearly 60 per cent of voters had their minds made up before the campaign began, and were favouring the Conservatives because they were tired of minority government and wanted stable economic management.

Mr. Graves had actually designed a voter-commitment index to identify non-voting members of the electorate in his surveys, but he had never used it before, was nervous about the results it produced – the correct results, as it turned out – and so decided to discount it.

The differential voter turnout concerns Frank Graves, as it should all Canadians.

The fact that more than half of the under-45 population doesn’t vote can be labelled their problem, Mr. Graves says. They can be ignored by political parties and policy-makers and be treated as a nuisance, which will further alienate them and suppress their vote.

But all that, he says, ignores the bigger issues: the fact that Canada has a unique and difficult demographic structure, proportionately the largest baby-boom population in the world next to Australia’s, and a federal government more rooted in older Canadians than any previous government in Canada’s history.

In 1967, the year of Canada’s centennial, the median age was 26. It’s now 42. Today’s typical voter is in his or her early 60s.

Rather than have the grace and wisdom of age being balanced by the enthusiasm and imagination of youth, Mr. Graves said, Canadians risk having a democracy where younger voters are in the process of disengaging from politics, with the voices of the youngest adults already missing.

Harper is no Obama

There’s no one reason why the young are disengaged. Young voters across the border were attracted to Barack Obama’s presidential campaign because he talked about the politics of hope and aspiration. But young Canadians find little to interest them in Stephen Harper’s agenda of strengthening the military, prisons and border guards, but otherwise trimming government social policy and engagement in the lives of citizens.

Rob Ford swept into the Toronto mayor’s office on the strength of older voter support. Ontario Conservative Leader Tim Hudak is probably hoping the same generational fault line will work to his advantage in the October election, Mr. Graves said.

And pollsters today, he said, have the responsibility and the technology to keep young Canadians in the national conversation, to let their voices to be heard on policy issues not co-joined with political issues, to explore using technology that was unimaginable five years ago to establish things like citizens’ juries and other forms of interactive panels that may well make greater contributions to democracy than polling.

The president of EKOS has come a distance from the smug night of May 2.

Michael Valpy is a freelance writer based in Toronto

< http://www.theglobeandmail.com/news/politics/a-pollsters-painful-reckoning-how-could-i-have-screwed-up-so-badly/article2065573/singlepage/#articlecontent >

Tags: ideology, participation, rights, youth

This entry was posted on Monday, June 20th, 2011 at 11:20 am and is filed under Governance Debates. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.