Margin of Error

Posted on by CalgaryGrit in by elections, Featured Posts, Polls | 2 Comments

They were up late counting the votes in Brandon-Souris Monday night. That, despite the fact that the final poll of the campaign showed the Liberals with a commanding 29-point lead. In the end, the Liberal vote was 16 points lower and the Tory vote was 14 points higher. It would have been a shocking result, if Forum’s reputation wasn’t such that pundits were already chuckling about that poll long before the results rolled in.

Before continuing on, I think it’s important to recognize just how far these numbers missed the mark. Some have talked about the “19 times in 20″ disclaimer at the end of every margin of error, writing this off as (yet another) 1 in 20 rogue poll. I won’t turn this into a statistics seminar, but bell curves are such that most misses should be close misses. This weather website predicts that the average high in Brandon ranges from -15 to +13 ºC nine days out of ten in November – but that doesn’t mean there was a 1 in 10 chance scrutineers would be pulling out the Bermuda shorts on election day. That tenth day is usually going to be fairly close to the range.


The reality is, based on Forum’s quoted margin of error and sample size, they were off by around 6 standard deviations. And based on sampling theory, the odds of that happening in a poll from a truly random sample are non-existent. Somewhere in the neighbourhood of 150 million to 1. (So more likely than the Liberals winning Macleod, but still very unlikely)

forum last night

OK, no more math. But I hope I’ve made the point that this isn’t the sort of thing that can happen by chance.

Forum President Lorne Bozinoff has his own theories:

He speculated that the difference between the final Brandon poll and the actual by-election outcome may have been that the Conservatives had a better “get out the vote” ground game than the Liberals. As well, he said some constituents who were angry about the perception of a fixed Tory nomination may have found they just couldn’t bring themselves to vote Liberal once they got into the ballot box.

I absolutely agree the ground game matters. However, you won’t find anyone in politics that believes the difference between the best ground game and no ground game at all is more than 5 or 6 points. Both parties were pumping resources and volunteers into Brandon, so every door got knocked – GotV may very well have been the reason the Tories won, but it doesn’t explain a 15-point swing. If it made that kind of difference, there’s no way the NDP would have swept Quebec last election.

I can somewhat buy last second switches playing a big role in the recent Alberta and BC elections, but it shouldn’t have been an issue in Brandon – especially when Forum showed Liberal support trending up. If that was what happened, then Bozinoff is basically saying that opinion polling is worthless. Because if the electorate is actually going to swing 15 points in under 24 hours, that means a poll showing the Tories at 30% on election day means they’ll finish anywhere from Kim Campbell territory (15%), right up to a majority government of historic proportions (45%).

Instead, what we’re dealing with appears to be flawed methodology. Bozinoff has admitted that some respondents may have been called for 3 consecutive polls, and that likely wouldn’t have happened unless the response rate was in the neighbourhood of 1% (typical for robo-polls, when you don’t do callbacks). Heck, Sunday being the Grey Cup, it may have been even lower. Sampling methodology only works if you assume survey respondents are similar to the public at large – otherwise, these polls are no more accurate than the “self selecting” click polls you see on websites, asking what you think of Miley Cyrus’ antics.

The obvious solution is more regulation on the polling industry, in terms of standards and disclosure. In the absence of that, it’s up to the media to show restraint when reporting what are clearly flawed robo-polls. Yes, they’re free. Yes, they make for an interesting “news” story – and bad polls make for an especially interesting “news” story because they run counter to the common wisdom.

Polls provide information, and information is a valuable tool. However, passing off faulty information as accurate, and giving it what is clearly not a “real” margin of error is dangerous. These freebie polls have shown themselves to be no more useful than “word on the street” anecdotes, and they should not be given any more credibility than that.

Please note – part of this issue, as noted by John Wright, is a lack of disclosure. So, in the interests of full disclosure, I should remind readers that I work at a polling company, albeit one where “robocalls” are considered a four letter word.

Charter Polling Misses Mark

Posted on by CalgaryGrit in Polls, Quebec Politics | 9 Comments

For those hoping Quebecers would abandon Marois over her utterly repugnant charter, this is an encouraging headline:

Quebec Liberals jump to 7% lead over PQ as backlash grows over values charter

A recent boost in support for the Quebec Liberals means the party could secure a “hair thin” majority in the province if an election were called today, suggests a new public opinion poll.

The poll, conducted by Forum Research, found support for Liberals in the province has jumped to 42% — up more than 10 points since the 2012 election — in the wake of the proposed Quebec charter of values.

Support for the Parti Quebecois sits at 35%, according to the poll. Coalition Avenir Québec (CAQ) came in third place with 12% support.

“We know from our polling that the proposed Charter is very popular among PQ supporters,” Forum president Lorne Bozinoff said in a Saturday statement, “but it appears that the ire it has raised amongst everyone else has blunted its usefulness as an electoral tool.”

An encouraging headline indeed. Sadly, that headline is about as far from reality as Marois herself.

Let’s put aside the usual critiques of robo-polls, and Forum’s spotty track record in Quebec. The real questionable aspect of this poll is how it’s being reported.

Yes, there has been a noticeable shift since the last election, but there’s no indication whatsoever that this is because of a “backlash” to the Charter. Forum is tracking changes from a year ago, oblivious to the fact that the world, and Quebec, have changed since then. I mean, who knew what “twerking” was a year ago?

More relevantly, Marois has had to govern, she delivered a budget, there was a tragedy in Lac-Mégantic, and, most importantly, the Quebec Liberals replaced Jean Charest with Philippe Couillard. There’s nothing in the data to suggest this revirement is due to the Charter – especially considering every single poll since Couillard won the leadership has had the Liberals on top. CROP showed Couillard up by 13 points in April and 11 in August. Leger had Couillard up by 4 in March, 11 in June, and 4 in August. The fact that Forum now shows the gap at 7 doesn’t support Bozinoff’s narrative of public “ire” over the Charter.

Indeed, a much more thorough poll on Friday by Angus Reid showed Quebecers are generally supportive of the Charter, and most specific elements of it.

That’s not to say this support will translate to votes for Marois, or that it will last to Christmas (Christmas is still going ahead under the Charter, right?). This is going to be a heated debate. And with lightning rod policies, voters don’t always focus on the issue itself, but what proposing the policy says about a leader’s judgment. Think John Tory and separate schools.

But that’s all speculation. At this stage, despite what the headlines say, there’s nothing in the polls to suggest the Charter is dragging Marois down.

UPDATE: And now Forum is contending that Trudeau has received a boost from his opposition to the Charter, with Forum President Lorne Bozinoff claiming the Charter debate has “moved a lot of support from other parties into the Liberal column, both federally and provincially”. Indeed, the Trudeau Liberals now lead at 36% in the province.

Mercifully, unlike the provincial poll, Forum hasn’t tracked these results against the last election. Actually, they haven’t tracked them at all, and it’s no wonder why. A quick google search shows that their most recent poll had the Liberals at 38% in Quebec.

So, to recap, Trudeau has surged from 38% to 36% on this issue.

I recognize the need to create a fresh story and narrative out of every poll. That’s why relatively insignificant changes in August were spun as Trudeau “surging” on the marijuana issue. But is it asking too much for polling companies and the media who print these stories to take a quick glance at their own numbers to make sure they don’t contradict what is being printed?

Byelection Results Roll In

Posted on by CalgaryGrit in by elections, Ontario Politics, Polls | 2 Comments

With most of the results in, it appears the Liberals have held Scarborough-Guildwood and Ottawa South, with the NDP winning Windsor-Tecumseh and London West, and Doug Holyday squeaking it out in Etobicoke-Lakeshore.

On the surface, this is a disastrous result for Kathleen Wynne, losing three seats and seeing her party’s vote fall by an average of over 15 points. The Liberals finished a distant third in Windsor and London, two ridings they won handily in 2011.

But byelections are all about expectations, and this is roughly what people expected from the Liberals. So it’s Tim Hudak who, despite scoring a long sought-after breakthrough into fortress Toronto, finds himself being declared the “loser” of the night by many. Go figure.

While I would never dispute that Tim Hudak is a loser, he didn’t receive much help in the expectations game from Forum research, which had the PCs leading in 3 ridings – and up by 16 points in Ottawa South. Consider this another reminder we really didn’t need to put much stock in superficial robopolls answered by 1 in 100 numbers called, in byelections where under a third of of public votes.

Chickening Out On Change

Posted on by CalgaryGrit in BC Politics, Polls | 18 Comments
"Well this is awkward"

“Well this is awkward”

We’ve been reminded this week that an 8-point lead in the dying days of an election campaign is about as safe as a 2-goal lead in the final 90 seconds of a playoff hockey game. Never take anything for granted.

Despite leading by between 2 to 9 points in every poll fielded over the past week, Adrian Dix managed to snatch defeat from the jaws of victory. It was a stunning result that no one saw coming – even though the exact same thing happened just one year ago in Alberta. In that campaign, Allison Redford trailled by 2 to 10 points in every poll, but still crushed Danielle Smith’s Wildrose Alliance on election night.

This has, of course, set off another round of polling post-mortems. I blogged about six possible polling error after the Alberta Surprise, and the issues are largely the same in British Columbia. So rather than rehash each point I want to look at the big picture.

We can quibble about things like question wording and ordering, but the largest problem cuts to the very core of the science of sampling – simply, polls are not truly drawn from a random sample of voters. I have no doubt if everyone was forced to vote and everyone was forced to answer the phone when pollsters came a calling, we’d see results within the margin of error. But that’s simply not the case, even though we pretend it is.

Indeed, only half of British Columbians bothered to vote yesterday. Admitedly, it’s difficult to figure out who is really going to vote in a world where 80% of people intend to…but then don’t bother showing up because they get distracted…or tied up a work…or because the weather sucks…or because the weather’s too nice to spend voting. There are ways to minimize this source of error, but it doesn’t appear polling companies made any effort to screen out unlikely voters or to gauge how solid support levels were. If they did, it wasn’t reported in the methodology, which is another problem in and of itself.

Moreover, there were warning signs the NDP was destined to lose the turnout game. Both Ipsos and Angus Reid showed the NDP and Liberals neck-and-neck among older voters, with the NDP up by 20-30 points among the under 35 crowd – a group notorious for their loud music, baggy pants, and low voter turnout rates.

The other side of the equation is that, sadly, not everyone is forced to respond to pollsters when the phone rings during Survivor. If you’re willing to spend the money, you can get a respectable response rate via traditional phone surveys, but all polls published during the BC campaign used either robocalls or online pannels.

Both of those methodologies have inherent problems. You often need to make 50 to 100 robo calls to find one sap willing to complete the survey. So we know Adrian Dix is popular with shut-ins, but extrapolating beyond that is risky. Moreover, since robocalls can only ask 5 simple questions before respondents drop off, you rarely have the opportunity to collect enough demographic information to judge how representative the sample is.

You can get those demographics using online panels, but while a national panel will have hundreds of thousands of Canadians on it, you’re fishing from a much smaller pool when you get down to the provincial level. You can always try to correct for demographic biases via weighting, but this can lead to a whole new set of problems. And it’s almost impossible to correct for attitudinal biases. The bottom line is that if you don’t have a large enough sample from Vancouver Island on your panel, you’re not going to get good data from Vancouver Island. It’ll be the same hundred people answering every survey.

Still, when different methodologies in different provinces keep missing the mark in the same direction, it feels like there’s something larger at play here. While the Clark and Redford miracles stand out, Jean Charest exceeded public polling numbers in 2012, as did Stephen Harper in 2011.

In all cases, voters had the opportunity to turf long-time and largely unpopular governments – then chickened out on change at the last minute. If an increasingly disengaged electorate truly is making up its mind more and more in the dying days (or hours) of the campaign, then a horse race poll is never going to predict the outcome spot on.

But maybe that’s not the end of the world. After all, superficial media polls are not designed to provide anything deeper than cheap entertainment. And where’s the fun in cheap entertainment, if the chance for a last-minute comeback doesn’t exist?

Let’s Not Get Ahead Of Ourselves

Posted on by CalgaryGrit in Polls | 5 Comments

Lorne Bozinoff, discussing Forum’s latest poll, explains what “should be clear by now”:

“It’s clear by now that the Trudeau phenomenon is no one-day wonder, and that a Liberal Party led by him would be the prohibitive favourite to beat in the next federal election,” Forum Research President Dr. Lorne Bozinoff said.

This likely makes me a bad Liberal, but if Bozinoff thinks Justin Trudeau is the “prohibitive favourite” to win the next election, I’m willing to put $1,000 on Stephen Harper – so long as he gives me “prohibitive underdog” odds.

Quebec Votes Tonight

Posted on by CalgaryGrit in Polls, Quebec Politics | 9 Comments

Today, Quebecers head to the polls, ready to elect what may be the least scary PQ government in the province’s history. That’s not a commentary on Marois, who is running on a disgustingly xenophobic platform. But even if the PQ gets a majority, it will be a majority built on only a third of the votes, at a time when there is no real appetite for another referendum. Premier Marois will change the country’s political dynamic and will put federal politicians on the hot seat, but this outcome won’t prompt the large-scale national panic that usually follows a PQ victory.

That is, assuming she wins. As I wrote on Friday, the conditions are ripe for a surprise – though that surprise would likely only be in the size of Marois’ win, or the composition of the opposition benches.

I’ll be blogging the results as they roll in tonight, but won’t hazard a guess as to the outcome. Here are the final polls and projections from those who are:

Forum (Sep 3, n = 2781 robo-dial)
PQ 36%
Lib 29%
CAQ 25%
QS 6%

Ekos (Aug 31 to Sep 3, n = 1749 robo-dial)
PQ 36.0%
CAQ 24.5%
Lib 23.2%
QS 10.7%

CROP (Aug 27-29, n = 1002 phone)
PQ 32%
CAQ 28%
Lib 26%
QS 9%

Leger (Aug 29-31, n = 1856 online)
PQ 33%
CAQ 28%
Lib 27%
QS 7%

308.com: PQ 63, Lib 33, CAQ 27, QS 2 (PQ 34.1%, Lib 27.9%, CAQ 26.3%, QS 7.1%)

Too Close to Call: PQ 66, Lib 33, CAQ 24, QS 2

Democratic Space: PQ 55, CAQ 35, Lib 33, QS 2

Forum: PQ 72

Clare Durand from WAPOR bucks the trend, concluding the undecideds will break Charest’s way, giving him a 33.1% to 29.5% edge in the popular vote and a minority government.

Your Guess is as Good as Mine

Posted on by CalgaryGrit in Polls, Quebec Politics | 11 Comments

Three parties entered the Quebec election with a chance to win, and while the PQ appears to be stumbling to victory, it’s nearly impossible to predict what will happen on Tuesday.

Of course, that’s not stopping anyone.

La Presse (thanks to CROP) projects a PQ minority, while the National Post (thanks to Forum) and the Globe (thanks to 308.com) project a majority. TooCloseToCall has the PQ with a 1-seat majority.

Even in the best of times, seat projections are an inexact science, but this is an election where they might not fare much better than the TSN playoff monkey.

Unlike Herman Cain, I’m not someone who distrusts the polls – but as the Alberta election showed us, shaky methodology and a volatile electorate can lead to journalists frantically re-writing their copy on election night. A robo-dialled Forum poll last week showed the Liberals riding a post-debate momentum wave to a 6-point lead over the PQ and an 11-point lead over the CAQ. That poll sticks out like an anglo at a PQ rally, but it’s still being included in the cumulative poll projections, and whose to say we won’t get another outlier in the final week?

Complicating things is the turnout issue. We know the PQ and Quebec Solidaire vote skews younger and young people are too busy smoking pot and listening to loud rock and roll music to bother voting…but are the polls factoring this in? Do the student protests change this? It really comes down to which party’s supporters are most motivated after an uninspiring August election, and I don’t think there’s a clear answer to that question. 308.com gives the Liberals an “advantage” for being in government, but this is based on a study of just 16 elections, and I’m not convinced it will translate through to a party that’s on the way out of power. And if it does, the other seat projections are under-estimating Liberal support.

Even if we had 100% confidence in the polls, transferring that confidence to seat projections is a recipe for disaster. Seat projections work reasonably well when there’s a good baseline to work off and relatively uniform shifts. Last year’s Manitoba election is a good example of this – with support levels only moving slightly from 2007, seat projections were spot on the money.

The problem with this election is that we’re dealing with a jumble of 3-way races and the rise of a new party, so there’s no baseline to work off. We can’t simply assume the CAQ will build off ADQ support uniformly, because this is a very different “third party” than the one Quebecers rejected four years ago, and they’re targeting a different type of voter.

So even when we’re treated to detailed regional numbers, there’s no way of knowing how that vote will transfer to individual ridings. Precise regional splits are a rare luxury, as many polls treat “Montreal”, “Quebec City”, and “Rest of Quebec” like three distinct and uniform regions. Anyone who has looked at riding-by-riding results in Montreal will know that’s certainly not the case.

My gut feeling is we’re heading for a Marois minority (and we all know how much she hates minorities!), but conditions are ripe for a surprise. Tune in Tuesday.

Trudeau’s Poll Position

Posted on by CalgaryGrit in --- 2013 LPC Leadership Race, Federal Politics, Polls | 15 Comments

Many Canadians polled have heard of this man

It’s no secret that Liberals love power above all else, so it’s hard for any Liberal to stay immune to Trudeaumania when you see headlines like “Justin Trudeau could lead Liberals to first place, new poll shows” or “Trudeau could be Grits’ only hope, says pollster Graves“.

I’ll admit that seeing a Trudeau-led Liberal Party at 40% in the Angus Reid “fantasyland” poll caused my heart to flutter a little bit. Hell, it’s been a while since the Grits were anywhere near the 32% they hit in Forum’s less rosy Trudeau scenario. With numbers like this, it seems foolhardy for any other candidate to even consider running against the reluctant saviour. Let’s just arrange catering for the victory party, and wrap this leadership race up now!

While there are many good reasons why Justin Trudeau should run, and there are many good reasons why Liberals should consider supporting him, polls like this should be low down the list.

What’s being measured in these surveys is not how a politician will perform as leader, but how familiar voters are with them now – and what their superficial opinion of that politician is now. While voters may know and like Justin Trudeau, I suspect a Liberal Party led by Justin Bieber, Justin Timberlake, or Justin Long would poll just as high.

Obviously, it’s good that Canadians have a favourable image of Trudeau, since that will make it harder for the Tory smear machine to define him. But despite this level of name recognition, I’d wager that Trudeau is still largely undefined. Voters may remember his eulogy and his boxing match – they might like his name and his hair. That doesn’t mean they have any idea of the type of leader he’d be or how he’d perform as Prime Minister. They may not be considering those factors when a pollster calls them on a Saturday afternoon three years out from a general election, but they certainly will be before they vote. For proof on how quickly things can change, a similar hypothetical poll last fall had Thomas Mulcair 13 points behind the Conservatives – today, he leads.

Polls like these will no doubt tempt many Liberals. However, we’ve all spent the past year talking about how much the Liberal Party needs to do to rebuild and make itself relevant to Canadians again. It would be foolish to assume we’re one saviour away from a trip to the promised land.

Even if you believe the Liberals are one leadership change away from power (and, hey, we can all dream), the best leader is not neccesarily the one with the highest level of name recognition today. If it were, I’d be starting a “Draft Bieber” blog.

The Third Way

Posted on by CalgaryGrit in Boring internal Liberal Party matters, Federal Politics, Policy, Polls | Leave a comment

The latest Ipsos poll paints a rather dreary picture of Liberal fortunes, with what was once the natural governing party languishing more than 15 points behind both the NDP and the Conservatives.

Of course, the NDP are in their post-leadership honeymoon, the Liberals don’t have a permanent leader, and a horse race poll when politics is the farthest thing from the electorate’s mind won’t tell you a lot. But I think we can safely assume the Liberals are a distant third, trailing two parties who are both intent on hugging the centre of the road, making it almost impossible to pass them. So what’s a centrist party to do?

I agree with Rae’s message of staying to the middle of the spectrum, but the days of finding sunny compromises between the NDP and Conservative extremes on every single issue are numbered. When you’re the third place party you need to give people a reason to vote for you, and a milquetoast platform topped with some language about the “extremist” positions of two very non-extremist parties isn’t going to be convincing.

Faced with this new reality, the challenge is standing out and being noticed. That likely means on occasion passing the two parties ahead of you on the right, and on occasion passing them on the left. So maybe the Liberals adopt a few “right wing” economic policies even the Conservatives dare not touch, like the abolishment of supply management. Maybe it means “out-NDPing” the NDP by proposing a national pharmacare program.

Of course, the entire concept of a left-right political spectrum is somewhat arbitrary when you think about it. Is democratic reform a right wing or a left wing issue? Either way, parties talk a lot less about it the closer they get to power, so there may be an opening there for the Liberals who are decidedly nowhere near power. There’s certainly an opening on the “Quebec question”, given the PQ may be in power a year from now, and both the Tories and NDP have spent long nights flirting with the separatists in recent years.

The other thing to consider is the dirty little secret that most voters aren’t reading through party platforms and casting their vote based on policy. Did Jack Layton leap from third to second because voters found his policies that much more compelling than Ignatieff’s? Most voters would be hard pressed to identify a single area of cleavage between the two party platforms.

Now, I’m not saying the Liberals are one leadership change away from power. As I’ve written before, there’s a lot of structural work to be done, and even if voters didn’t know the intricacies of the Liberal and NDP platforms last election, they had a clear impression of party brands, and an overall sense of party values. But a party’s leader does matter, and it’s just as important to have a leader who can differentiate himself or herself from Mulcair and Harper, as it is to have policies that can be differentiated from the NDP and CPC platforms. That doesn’t mean the Liberals should search for the anti-Mulcair or shy away from an experienced and polished politician like Harper – only that there needs to be some kind of “value add” that makes their leader stand out. The brilliance of Jack was that he always smiled and could connect with voters – that’s an ability Michael Ignatieff lacked completely, and one both Harper and Mulcair struggle with.

In the past, all the Liberals needed to do to get elected was wedge themselves squarely between the extremes. There are still many issues for which that strategy makes sense from both an ideological and political perspective. But adopting that knee-jerk approach on every issue and failing to stand out is a sure fire path to irrelevance.

Margin of Error

Posted on by CalgaryGrit in 2012 Alberta Election, Alberta Politics, Featured Posts, Polls | Leave a comment

I don’t think I’ve ever followed an election where the polls were as horribly off the mark as they were in Alberta.

Last May, when the media jumped on the “pollsters blew it” bandwagon for not projecting a Tory majority, most companies were still within the margin of error on the final vote intent numbers. Even during the 2004 federal election, the case study in pollsters “missing” a late swing, there wasn’t a poll the final week of the campaign that had the Liberals behind (even if seat projections did), and most only under-estimated Liberal support by 3-5 points.

But last night? This wasn’t just a case of shanking a field goal “wide right”, but booting it in the complete opposite direction of the goal posts. Here’s how the final polls stacked up with the results.

As most have commented, Forum’s Sunday afternoon poll picked up part of the late swing but, even then, to go from a 2-point Wildrose lead and 10-point PC win is under 24 hours is shocking. It wasn’t just a case of last second “strategic voting”, since most polls in the final week correctly pegged Liberal and NDP support levels.

So what went wrong? I can think of 6 possibilities:

1. The polls made little effort to screen out the 43% of Albertans who didn’t bother to vote on election day. Just asking respondents if they were absolutely certain to vote would have been a good start, even if few followed through on those intentions. But there are other attitudes and demographics that can help predict intent (i.e. older people are more likely to vote), and because of a lack of transparency in how these questions were asked or weighted, we have no way of knowing what steps were taken to screen out unlikely voters.

2. Building on the above point, the Big Blue Machine may have had a superior get out the vote operation than the relatively new Wildrose Party. I suspect this is part of the reason the federal Conservatives have “over performed” the polls on election day in recent years. Still, the best GotV operation will only bump you up a few percentage points, and it’s not like the Wildrose Party was short of former Tory organizers, money, or volunteers.

3. The PCs had better candidates and more incumbents. Even though local candidates rarely have a big impact on the results (see Quebec, 2011), it’s possible Albertans “voted” for the party and leader they wanted when asked that question on the survey, then considered the local candidates when they saw the names on the ballot. Still, once again, I can’t imagine this would translate to more than a point or two at the province-wide level.

4. With voters growing increasingly disengaged and disinterested in the political process, it’s possible many simply made up their mind in the voting booth. Since most polls only asked vote intent, there was little analysis in terms of strength of support, or where undecideds and soft voters might break before election day.

5. The most popular theory is that there was a “late swing” back to the PCs. This is born out by the Sunday Forum poll but, even then, a 20-point swing in the margin over the course of 5 days, or a single day 12-point swing is almost unheard of in politics. I don’t doubt there was a late shift, but from what I hear, the PC Party’s internal numbers showed them in much better shape than any of the media polls, suggesting that Smith’s lead was never as big as it was reported.

6. So how could all those polls have been wrong? Well, if you look at that table above, you’ll notice that Leger was one of the closest to the final mark, despite leaving field a week prior to the vote, before any “swing back” to the PCs was fully felt. The pollsters who overshot Wildrose support the most all used robo-diallers and online panels.

Both of those methodologies have inherent problems. You often need to make 50 to 100 robo calls to find one sap willing to complete the survey. So we know the Wildrose Party was popular with shut-ins, but that’s about it. Moreover, since robo calls can only ask 5 simple questions before respondents drop off, you rarely have the opportunity to collect enough demographic information to judge how representative the sample is.

You can get those demographics using online panels, but while a national panel will have hundreds of thousands of Canadians on it, you’re fishing from a much smaller pool when you get down to the Alberta level. Companies who don’t frequently conduct political polling in Alberta might not have a good understanding of the biases inherent to the panel they’re using, opening up the risk of skewed results.

If you’re looking for more background on some of the problems associated with robocalls and online polls, I’d suggest this excellent letter by Darrel Bricker and John Wright, or this article featuring blunt comments from Allan Greg and Andre Turcotte.

The blame doesn’t rest solely on the polling companies. The fact is robocalls and online polls are cheap to produce, and that’s all the media is willing to pay for. The internal Tory polls used live callers, and asked more demographic and attitudinal questions than just vote intent – this no doubt let them verify the validity of their sample, and provided direction on what levers could cause the public to swing back to the Tory fold. There’s something to be said about the old “you get what you pay for” adage, and most newspapers simply don’t have the budget to invest in getting the job done right.

We’ll probably never know which of the above factors were actually in play. And hell, this being Alberta, it could just be part of the deal with the devil the Alberta PCs signed long ago that ensures PC victory after PC victory.

1 2 3 4 5 6 7 8   Next »
Plugin from the creators of Brindes Personalizados :: More at Plulz Wordpress Plugins