Fun with Numb3rs

Fun with cluster analysis

Posted on by CalgaryGrit in Fun with Numb3rs, Toronto Municipal Politics | Leave a comment

This is really a propos nothing in particular, but I stumbled across a nifty bit of analysis looking at the voting records of Toronto City Councillors. Using a cluster classification, the numerically-gifted Buzzdata user “Haz” was able to group City Council members into clusters as follows:

For those who weren’t forced to spend your early 20s labouring through fourth year statistics courses and cursing SPSS on a nightly basis, the closer two names are “linked” together, the more often they voted together. For example, the Ford brothers are linked directly to each other since they tend to vote together. Their next closest links are to Giorgio Mammoliti and Mike Del Grande.

If you keep moving down the line, council can be cleaved into two groups – the “green” and “red” lines above. Those familiar with municipal politics in Toronto will quickly recognize which side of the political spectrum each group falls.

To come up with a visual, the five largest clusters were coloured into the ward map – blue and green clusters represent councillors voting with the “lefties”, with the yellow, red, and purple clusters representing those voting with the more conservative elements of council.

As you might expect, the blue in green clusters are concentrated downtown, with the red, yellow, and purple clusters generally further out in the suburbs.

There are, of course, exceptions. York councillors Anthony Perruzza and Maria Augimeri find themselves close to the Adam Vaughans and Paula Fletchers of City Council. Meanwhile, Davenport’s Ana Bailao checks in with a more conservative voting record than you might expect given her riding went NDP this May.

Fun with Numbers: The Conservatives

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

Last month, I looked at ridings where the Liberals exceeded expectations. Today, a look at ridings where the Tories did better than we would have thought given what was going on around them. If you’re confused about what the numbers in brackets mean, check out my first post on this topic. I’m not saying it will end your confusion, but it may.

1. Cumberland Colchester (+33 percentage points above expected): Bill Casey wiped the floor clean of everyone in 2008 as an independent. And since this exercise uses 2008 as the baseline, it’s only natural this riding would rise to the top. Let’s consider it an aberation and move on.

2. Labrador (+20): Canada’s new minister of intergovernmental affairs likely pulled off the most remarkable victory in the country on May 2nd. In 2008, Todd Russell won the riding for the Liberals with 70% – the CPC were a distant third at 8%. And yet, on election night, Peter Penashue emerged triumphant. Even when we factor in the Tory jump in Newfoundland, this riding stands out as one of the most astounding election night results.

3. Nunavut (+14): Seat projections and strategic voting sites had this riding as a 3-way race…in the end, Leona Aglukkaq increased her 2008 level of support by 15 points.

4. Vaughan (+13): This one isn’t a surprise given the by election. I’d argue Lamoureux’s hold was more impressive, but Fantino still built upon his by election success, earning 56% of the vote.

5. Montmagny-L’islet-yada yada (+11): This was a rare seat in Quebec where the Conservative vote actually increased from 2008 but, once again, the results are somewhat deceiving since the Tories picked it up in a 2009 by election. Clearly by election results will need to be factored in if this methodology is ever refined.

6. Megantic-L’Erable (+8): Christian Paradis not only survived, but increased his vote from 2008.

7. Mount Royal (+8): It’s no secret the Tories were targeting the Jewish population in Mount Royal, and they appear to have had some success at it. Of note, Mount Royal was number 12 on this same list in 2008, so this wasn’t a one-time fluke – they’re making sustained inroads.

8. Roberval-Lac Saint Jean (+8): Like Paradis, Denis Lebel managed to keep his head above the orange wave.

9. Timmins-James Bay (+8): Although the riding stayed orange, Conservative support jumped 14 points there.

10. Esquimalt Juan De Fuca (+7): Not surprising given the retirement of Keith Martin.

Other Notables: Algoma-Manitoulin-Kapuskasing, Sydney-Victoria, Miramichi, Mississauga-Brampton South, Egmont, York Centre, Hamilton East-Stoney Creek, Saint John, Richmond

Fun with Numbers: The Liberals

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

As we learned last election, when the political wave rises, it’s hard to avoid it. If ever you wanted evidence of just how powerless local campaigns are, look no further than some of the quality men and women who were defeated by phantom candidates who hadn’t even set foot in the riding, never mind campaigned there.

But that’s not to say local candidates can’t make a difference. After all, it’s a safe bet that lone spec of red between Winnipeg and Vancouver also known as Wascana would be blue if anyone other than Ralph Goodale had been the Liberal candidate.

It’s very difficult to measure the success of local campaigns, but it is possible to get a sense of which candidates did better or worse than expected. To measure this, I’ve used the same methodology as after the 2008 election. Namely, I’ve carved Canada up into 30 subregions (i.e. Calgary, Southwest Ontario, Nova Scotia, Northern Quebec, etc) and compared the vote change from 2008 in each riding to the regional swing. So if a given candidate held his support while her party lost 10 points in the region, that’s a sign the local campaign had some positive mojo.

With that, I present a list of the 10 Liberal campaigns that most exceeded expectations. The important thing to keep in mind is that this is relative to 2008. People like Gerard Kennedy and Ken Dryden may very well be star candidates who ran top notch campaigns but their residual value for this exercise still comes in around “0” – which is what we’d expect, assuming they ran comparably strong campaigns in the last go around.

So treat this more as a list of ridings that stood out, for a variety of reasons.

1. Winnipeg North (+27 percentage points above expected): No surprise here since I used 2008 and not the by election as the baseline. But even if we give by elections their historical weighting, Lamoureux nets a +10. To become relevant in Western Canada, the Liberal Party needs to find whatever factory made Kevin Lamoureux and order a dozen more.

2. Guelph (+17): Frank Valeriote’s vote actually jumped 11 points in Guelph this election, due in part to the collapse of the Greens in this riding.

3. Central Nova (+15): Central Nova makes this list for obvious reasons…it’s not hard to improve upon 0 votes. Central Nova scored a -17 in 2008, so this marks a return to the traditional level of Liberal support..

4. Sault Ste. Marie (+12): While 19% is not a great showing, it still marks an improvement from 2008, at a time when the Liberals were falling everywhere else in Ontario. Still, this riding put up a -7 in 2008, so some of this is nothing more than a “bounce back”.

5. Bruce-Grey-Owen Sound (+11): Basically the same story as above.

6. Cumberland-Colchester (+10): The Liberals benefit from the retirement of Bill Casey, who pretty much everyone in the riding voted for in 2008.

7. Don Valley West (+10): This is small consolation for Rob Oliphant, who still finds himself unemployed.

8. Calgary Northeast (+8): This was actually the Liberals’ best riding in Alberta, with their support jumping 8 points, to 28%. The LPCA should treat this riding the same way the NDP treated Edmonton Strathcona after Linda Duncan’s strong second in 2006. Nominate Stewart early, door knock the riding heavily in advance of the next election, and pour resources into it. At the very least, they can make the CPC play defense in Calgary next time.

9. Avalon (+8): Scott Andrews held his ground, despite the Liberals dropping elsewhere in Newfoundland.

10. Surrey North (+8): Once again, 18% isn’t a phenomenal result, but it still marks an improvement from 2008.

Honourable Mention: Kitchener-Waterloo, Papineau, Ancaster-Dundas-Flamborough, Ajax-Pickering, Nepean-Carleton, Carleton-Mississippi Mills, Eglinton-Lawrence, Malpeque, Random-Burin-St.George’s, Medicine Hat, Westmount-Ville Marie

Fun with Numbers: Most Volatile Ridings

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

Unless Stephen Harper breaks his own fixed election date law (ha ha ha!), we’ll have a new set of ridings for the 2015 election.

So for kicks this summer, I’m looking back at life in the old ridings over the past four elections. I already posted a list of the most exciting ridings in the country – today, a list of the most volatile.

To keep it simple, I’ve simply tallied up the percentage point swing between each election, to find out which ridings had the largest changes. So if from 2004 to 2006 the Tories fell 8 points in a given riding, the Liberals fell 8 points, the NDP rose 8 points, and the Greens rose 8 points, that riding would be recorded as a 32-point swing.

And with that, the most volatile ridings in the county over the past decade:

1. Cumberland Colchester (285-point swing): No surprise here, as the Tory vote yo-yo’d thanks to the Bill Casey fiasco.

2. Portneuf-Jacques Cartier (194): Like Cumberland Colchester, Portneuf rises to the top due to the rise and fall of an independent conservative, Andre Arthur.

3. Jonquiere-Alma (184): This riding was on the frontlines of both the rise of the Tories and the rise of the Dippers in Quebec.

4. Labrador (179): Labrador saw a 30-point shift from the Tories to the Liberals this May.

5. Beauce (169): I feel like the riding should get “volatility bonus points”, due to the personality of its MP.

6. Trois-Rivières (162): The orange wave was particularly strong here last election, with NDP support sextupling from 9% to 54%.

7. Charlesbourg-Haute-Saint-Charles (159): A similar story to many Quebec ridings on this list – the Tories jumped 26 points in 2006, and the NDP jumped 32 points in 2011.

8. St. John’s East (153): Thanks in large part to a mind blowing 57-point jump in support for Jack Harris during the 2008 election.

9. Abitibi-Baie-James-Nunavik-Eeyou (150): Ninth in volatility, but first in cumbersomeness of the riding’s name.

10. Rivière-du-Nord (150): Over half that swing came last election.

And just missing the Top 10…

11. Central Nova (148): Thanks to Lizzie May.

12. Saanich – Gulf Islands (141): See above.

On the list of most stable ridings, there are few surprises with 7 tory-blue Alberta ridings in the top 10, led by Crowfoot (only a 20-point swing). Dartmouth-Cole Harbour (28%) comes in at number 3 to add a bit of orange to the list, with Malpeque the only solidly Liberal riding in the top 10.

And finally, taking the riding-by-riding averages, the most volatile provinces politically over the past decade:

1. Newfoundland: Thanks to Danny Williams
2. Quebec: I corrected for the presence of the Bloc here, dropping them to second
3. Nova Scotia
4. Manitoba
5. New Brunswick
6. BC
7. Ontario
8. Saskatchewan
9. PEI
10. Alberta

The top 2 and bottom 2 stand out, with the rest all bunched closely together.

Could Have Been Elections: The Liberal-Democrats

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

In response to my previous “what if” post on running elxn41 under a preferential ballot, a few blog readers wondered how the election would have turned out had there been a Liberal-NDP merger.

The challenge with that kind of analysis is that we have no real way of knowing what 1 + 1 equals. Even merger proponents are not so naive as to assume every current Liberal and every current New Democrat would vote for a new Lib-Dem Party. The tricky part is figuring out how many would stay home or jump to other parties.

I’ve run these kinds of exercises before, but it doesn’t hurt to update it given the new realities of the day. After all, even though a merger seems unlikely, it will be talked about at various times over the next four years – pretty much whenever columnists or bloggers are looking for something to write about during the otherwise dull life of a majority government.

I decided to go into this exercise using a “best case” scenario for the Lib-Dems. That is, I assume that every Liberal with the Conservatives as their second choice (17%) would vote Conservative and every NDP member with the Greens as their second choice (21%) would vote Green. But everyone else would vote for the new party. This would give the Lib-Dems a retention rate of over 80% – higher than the hold rate of the merged Conservative Party in 2004, despite the CPC being given the gift of Adscam (they held just 78.5% of the combined 2000 PC + CA vote).

Under this scenario, the end result is largely the same as now – a 160-seat Conservative majority and a 144-seat Liberal Democrat opposition.

Of course, this was the best case scenario. If the Liberals lose a quarter of their voters to the right and the NDP lose a quarter of their voters to the left, then Harper leads the Lib-Dems 178 to 125 seats in this restrospective hypothetical.

Perhaps a merger would make sense in the long term. But anyone who assumes it would be a quick-fix for booting the Conservatives just simply isn’t looking at the numbers.

What Could Have Been: Elxn41 Under a Preferential Ballot

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

One of the democratic reform initiatives that never seems to get much publicity is the preferential ballot. Yet it’s simple, assures the majority of the riding backs the winning candidate, and helps avoid some of the dangerous strategic voting mis-steps we see all too often. It’s how parties elect their leaders, yet we assume Canadians can’t handle ranking candidates 1-2-3.

So what kind of impact would a preferential ballot have had on the previous election?

To determine this, I looked back at the final “second choice” poll numbers from the last campaign (seen here, here, and here), applied a few minor regional corrections, and ran run-offs in each riding where the winning candidate received fewer than half the vote. So if the Greens were fourth, their votes were scatered based on the second choice of Green voters. Then if the Liberals were third, their votes were scatered based on the second choice of Liberal voters.

It’s not an exact science, but it’s close enough for a fun “what if” exercise.

The result?

CPC: 147
NDP: 115
Lib: 44
BQ: 1
Green: 1

In the end, only 25 seats change, but that’s enough to knock the Conservatives down to a minority. The majority of their loses come from Ontario, with ridings like London North Centre, Willowdale, and the Don Valleys staying red. While the Liberals snag 13 Tory seats under this system, they’d lose some star power with Misters Trudeau, Garneau, and Lamoureux all drowning under the orange wave. After all, as bizarre as it may sound, even Conservative voters were more likely to lean NDP than Liberal by the end of the last campaign.

The NDP would therefore snag seats from the Tories, Liberals, and Bloc, and would come out of Quebec with a remarkable 66 seats.

So while it makes a certain amount of sense, it’s not something I’d expect to see anytime soon unless Stephen Harper is feeling in a particularly self-destructive mood.

UPDATE NBPolitico looks at what might have been under other voting systems

Most Exciting Ridings in Canada

Posted on by CalgaryGrit in Fun with Numb3rs | Leave a comment

We’ve now had 4 elections under the current 308 riding electoral map, a period over which the Canadian political landscape has completely transformed itself.

2004:

2011:

Still, that doesn’t mean every local election has been exciting – after all, if you live in Calgary East or York West, your vote hasn’t been overly meaningful at any point over the past decade.

For interests sake, I’ve calculated an “excitement score” for every riding based on how close the last four elections have been and how many times they’ve shifted hands. Yes, this is a fairly arbitrary exercise, but consider it like those “best places to live” lists…only for political junkies.

Most Exciting Ridings in Canada

1. Brossard-La Prairie: The riding where I grew up has changed hands every election – from Liberal to Bloc to Liberal to NDP. In 2008, it took a recount to sort it out.

2. Louis-Hebert: You’ll notice a few Quebec seats on this list, for obvious reasons. Like Brossard, it has switched hands each election.

3. Vancouver Island North: A perpetual Conservative-NDP battle, the margin of victory has been under 5% in each of the last four elections.

4. Parkdale-High Park: This riding has made headlines during the last two campaigns due to the high profile and often nasty Gerard Kennedy-Peggy Nash battles. But even before then, the Liberals and NDP each pulled off narrow victories in 2004 and 2006.

5. Newton North Delta: Gurmant Grewal’s old riding has not been a warm comfy mat with lots of fur for incumbents, with the Tories losing it in 2006 to the Liberals, who in turn were turfed in 2011.

6. Papineau: Believe it or not, Justin Trudeau’s margin of victory in 2011 was by far the most lopsided result in Papineau over the past decade, with no other election won by more than 3 percentage points.

7. Surrey North: From Chuck Cadman (Ind) to Penny Priddy (NDP) to Dona Cadman (CPC) to Jasbir Sandhu (NDP).

8. Jeanne-Le Ber: Turned into an NDP rout this election after three close Liberal-BQ battles.

9. Ahuntsic: No party has won this riding by more than 3 points in any of the past four elections.

10. Esquimalt-Juan De Fuca: Keith Martin always clung to victory and, with his departure, the riding quickly morphed into a tight CPC-NDP duel.

Those are the most exciting ridings. The least? Well, it should be no surprise that all 10 are in Alberta, with Crowfoot topping the list. The closest Crowfoot has been over the past decade was in 2004, when Kevin Sorenson won it by a measly 73 percentage points, earning over 10 times as many votes as the second place candidate.

Predicting the Unexpected

Posted on by CalgaryGrit in 2011 Federal Election, Federal Politics, Polls, Seat Projections | Leave a comment

I’ll announce the winners from my election pool later this week. One of the questions there asked which polling company’s final poll numbers would hit closest to the mark.

You can browse the numbers here. To pick a winner, I simply added up the difference between their numbers and the results, producing the following:

Angus Reid: 5.2%
Nanos: 5.5%
Ipsos: 6.0%
Decima: 6.4%
Leger: 7.2%
Abacus: 9.0%
Forum: approx 9% (BQ and Green numbers extrapolated)
Ekos: 10.3%
Compass: 14.0%

So congrats to those of you who picked Angus. The top 6 companies on that list were within the margin of error on their numbers, so they too deserve a round of applause.

As for the seat projections, here’s the total seat miss:

Riding by Riding: 52
LISPOP: 56
Calgary Grit: 56
Ekos: 58
Democratic Space: 58
Trendlines: 59
308.com: 98
Election Prediction Project: 118

So a similar performance by all the mathematical models, except for 308 who has already offered a brief post mortem. I will add that my prediction was further off the mark from my projection – I made the same faulty assumptions as the EPP did, assuming strong incumbents could hold their seats.

The largest problem with my projection was the polls it fed off – specifically the low Conservative numbers (which I did foresee as a potential problem). If I plug the actual numbers in, my model projects: CPC 168.8, NDP 94.6, Lib 34.0, Bloc 10.1. The regional splits break down nicely too, except for Quebec where I’m a bit high on the Bloc and low on the NDP.

But this model was supposed to handle pollsters missing the mark. A few of the results fell outside the 95% confidence interval so this is, as Jack Layton would say, a hash tag fail.

I’ll put this one to bed for a bit and start tinkering again over the summer, but I think this speaks to the limitations of any seat projection model. They’re useful tools, but it’s incredibly naive to assume they can predict the total seat count, much less individual riding results.

But that’s ok. If they worked, it would make election nights a bore.

Final Seat Projections

Posted on by CalgaryGrit in 2011 Federal Election, Federal Politics, Seat Projections | Leave a comment


I’ll follow up tomorrow with a round-up of all the final polls and projections but for now, here’s the running average. For those just tuning in, I assign weights to all polls based on sample size, the polling company’s track record on Canadian elections over the past 6 years, and the recency of the data:

CPC 36.5%
NDP 29.9%
Lib 21.3%
BQ 6.6%
Green 4.9%

Keep in mind, the Conservatives exceeded their final poll numbers by 2-3 percentage points in 2008. Whether or not that happens again remains to be seen, but they do have the best ground game and the most incumbent candidates. Their vote also tends to be older and, I suspect, is more likely to turn out at the polls. But that’s all conjecture on my part – the exercise here is to make projections using the data on hand.

Using the vote numbers above, I’ve simulated the election 10,000 times. These simulations factor in results from the past 3 elections, the demographics of each riding, by election results, and incumbency advantages. Each simulation looks at how regional shifts traditionally transfer to the riding level – after all, if the NDP vote jumps from 12% to 36% in Quebec, it doesn’t mean their vote will increase by 24 points in every riding…and it doesn’t mean their vote will triple in every riding. This is why the simulation model is, in my humble opinion, superior to all other models out there. It doesn’t say the Tories will win a given seat – it only says they have a 70% chance of winning a given seat.

Finally, when running the simulations, I also consider the fact that the polls could simply be off the mark, as they have been in some recent elections.

For the long and boring explanation of this methodology, click here.

So, with that said…drumroll please…

CPC: 139 to 163 seats (mean: 151.0)
NDP: 76 to 106 seats (mean: 90.9)
Lib: 35 to 58 seats (mean: 46.9)
BQ: 9 to 30 seats (mean: 18.5)
Ind: 0.6

I would love to tighten some of those confidence intervals up, but the reality is this is as close to certain as we can be given the public polling data on hand. Quebec is a complete wild card this election, since there isn’t a projection model out there capable of saying what a 30-point jump in NDP support actually means in terms of seats.

All 10,000 simulations had the same order of the parties, with the Conservatives winning a majority 28% of the time. So we may have to stay up late to see whether or not Harper gets that long sought after majority…or at least is close enough to “top it up” with a few floor crossers.

Now, what I like about this simulation model is that it’s 100% data driven. I haven’t arbitrarily decided who is or isn’t a star candidate, or tweaked the model to get the results I want. So like all seat projection systems, it should be used as a tool to make predictions, not as a prediction in and of itself.

This is why I’ve spent the weekend going riding by riding (Atlantic, Quebec, Ontario, Prairies, Alberta, BC). Based on what the model is projecting and what the individual ridings look like, I’m going to stand by the prediction I made on James Bow’s blog yesterday:

CPC 146
NDP 83
Lib 55
BQ 22
Green 1
Ind 1

I may be wrong, and that’s cool if I am. It wouldn’t be the first time. By all means, make your own predictions below, and I’ll be sure to give a special shout out to whoever is closest to the mark.

Your Daily Seat Projections

Posted on by CalgaryGrit in 2011 Federal Election, Federal Politics, Seat Projections | Leave a comment

My final projections will be posted tomorrow, but we’ve already got the final numbers from Leger, Ipsos, and Angus. Here’s the running average:

CPC: 36.5% (+0.4)
NDP: 30.4% (+2.7)
Lib: 20.8% (-1.9)
Bloc: 6.6% (-)
Green: 4.7% (-0.9)

As a reminder, the following projections are based on 10,000 simulations, taking into account the polling margins of error, how shifts in regional support have historically transferred to individual ridings, and the chance the pollsters could just miss the mark, like in some previous elections.

This is the only simulation model out there and, as such, it’s far more effective when projecting tight races or three-way races since it recognizes the riding could go either way depending on how the numbers break.

CPC: 139 to 163 seats (mean: 151.0)
NDP: 78 to 108 seats (mean: 93.6)
Lib: 34 to 57 seats (mean: 45.2)
Bloc: 8 to 28 seats (mean: 17.6)
Ind: 0.6

Odds of Tory victory: 100%
Odds of Tory majority: 29% (up from 20% on Friday)
Odds of NDP official opposition: 100%

Since Friday, the NDP are up another 15 seats on average, with the Liberals down 10 and the Bloc down 5. The NDP range in Quebec now sits at an astonishing 33 to 58 seats. However, even though the polls are good for them, if you happen to find yourself in Vegas this weekend, it might be safer to bet on the low end of that. In a lot of close races, on the ground organization will make a difference, and that’s one area where the NDP are lacking in Quebec. Heck, it’s unclear whether some of their candidates will even be around to vote for themselves. I’m not trying to be snide, I’m just trying to point out the weakness in any seat projection model – the same problem would present itself if the Liberals surged across Alberta. We’ll have to wait until Monday to see how it plays out.

Becoming almost as unpredictable as Quebec is Ontario. Decima has the Liberals winning in the province, while Ipsos has them 13 points back of the NDP, in third. As a result, the 95% confidence interval for number of Liberal seats there ranges from 16 up to 33. There are very few safe seats left in this Liberal fortress.

(short methodology, long methodology)

Plugin from the creators of Brindes Personalizados :: More at Plulz Wordpress Plugins