Social Policy in Ontario

How to navigate a thicket of polls

Posted on April 1, 2011 in Governance Debates

Source: Toronto Star — Authors: Carol Goar

TheStar.com – opinion/editorialopinion
Published On Fri Apr 01 2011.   By Carol Goar Editorial Board

You’ll be inundated with roughly 50 election polls in the next month. Each will be heralded as the most accurate snapshot of voters’ intentions.

But the results will vary widely; the numbers will bounce around frenetically; and comparisons will be deceptive because the polling agencies use different techniques and measures.

What is a confused — and probably irritated — voter to do?

Start with a small experiment. Ask your friends, relatives, colleagues and acquaintances if they’d be willing to devote 15 or 20 minutes to telling a pollster which party they support, which leader they prefer and which issues will determine how they vote — then provide personal information.

If they all, or almost all, say no, you’ve got a pretty fair indication that pollsters are pushing the limits of probability to get their readings.

Now look back five months. On the eve of Toronto’s mayoral election, every major pollster (with a single exception: Frank Graves of Ekos Research) said Rob Ford and George Smitherman were locked in a dead heat. The outcome wasn’t even close. Ford won by a 93,669 vote margin.

You already have two good reasons to treat polls with skepticism. But you can’t ignore them; they’re too much a part of election campaigns. And you can use them, if you filter out the hype and misinformation.

So here is a rudimentary guide to election polls:

• Don’t assume they’re all equal.

Some polls use a large sample of the voting population (more than 6,000 adults); others rely on much smaller cross-sections (as few as 850). Some are conducted by phone and some take place online. A few polling agencies use both methods.

Some are pulled together in a few hours; others are conducted over two or three days. Some pollsters pay participants; most don’t. Some firms allocate undecided voters according to their leanings; others don’t. Some polls are commissioned by news organizations or universities; others are paid for by corporate clients, who allow a few political questions to be tacked on the end.

Generally, the larger the sample, the more reliable the result. Polls taken over an extended period usually provide a clearer picture than instantaneous snapshots. And dedicated political surveys are almost always superior to commercial ones with add-ons.

The question of how to treat “leaners” is a matter of dispute. If the objective is to produce a simple projection, a poll that assigns people according to their inclinations is better. If the purpose is to show the state of the race, including how much room there is for movement, a poll with a separate category for the undecided is better.

There is also vigorous disagreement over the merits of online vs. telephone polling. Online surveys tend to be skewed in favour of younger, more affluent voters. Telephone surveys tend to over-represent older, rural and lower-income voters. Web polls are typically larger, but voice polls, which use random sampling, offer a better microcosm of the electorate.

• The way polls are reported can matter as much as their content.

The Canada Elections Act requires the media to provide certain basic information: who conducted a poll, who paid for it, the dates it was conducted, the size of the sample, the response rate, the margin of error and how to obtain the data. Very few news organizations consistently meet these requirements.

Even when they do, journalists writing poll stories seldom know a lot about sampling methods or probability theory. This can lead to faulty assumptions, factual errors and misleading headlines.

Check for mistakes, missing information and other trouble signs.

• The way polls are interpreted can set off unwarranted stampedes.

The culprit can be a pundit who labels a minor fluctuation a pivotal shift; an editor who tries to inject a little drama into a dull story, an academic who theorizes aloud, or a spotlight-enamoured pollster who goes far beyond what the numbers show.

Make sure you trust the analyst and look at the evidence.

None of this means, as cynics claim, that polls are of no value. They signal important trends. They allow people to vote strategically. They’re right more often than they’re wrong.

But don’t let any poll supplant your instincts and don’t let anyone tell you tell polling is a sure science.

< http://www.thestar.com/opinion/editorialopinion/article/967092–goar-how-to-navigate-a-thicket-of-polls >

Tags: participation

This entry was posted on Friday, April 1st, 2011 at 6:44 pm and is filed under Governance Debates. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.