Nova Scotia Election: Based on first polling results, want to bet $1,000 on the outcome?

According to a well-known Halifax polling company, Nova Scotians are "highly likely" to elect an NDP government on June 9th. The prediction is based on an opinion survey released today by Corporate Research Associates. CRA reports that its poll of 627 Nova Scotia voters shows that 37 per cent of the decided and leaning ones supported the NDP when the survey was conducted from May 7 to 16. The Nova Scotia Liberals came second with 31 per cent support, while the governing Conservatives trailed with 28 per cent. The Green Party scored only three per cent.

Looks like the NDP enjoys a comfortable lead, doesn't it?

But a closer look at the numbers shows that great caution is needed when interpreting them. True, the results of the latest poll are consistent with four other CRA polls conducted over the last year. The trend in voter support seems to consistently favour the NDP as the party came first in every one of those surveys. However, some simple math calculations show that the outcome of the provincial election is a lot less certain than this poll seems to show.

Taking the poll at face value

OK. Pollsters like to give precise numbers to show the support each party enjoys, but poll results actually fall within a range. This is what's called the poll's "margin of error." CRA reports the overall margin of error for this survey was plus or minus 3.9 percentage points, 95 times out of 100. The numbers and terminology seem complicated, but their actual meaning is quite simple. If the same question had been asked of a similarly sized sample of voters during the same time period, any reputable polling company would have found results within that 3.9 percentage point margin of error, 95 times out of 100. Or, to put it another way, if the election had actually been held during the polling period, each party's vote tally would have fallen somewhere within that range: plus or minus 3.9 percentage points.

Thus, the CRA poll shows that support for the NDP could have been as high as 40.9 per cent [37+3.9] or as low as 33.1 per cent [37-3.9]. The Liberals could have been as high as 34.9 per cent [31+3.9] or as low as 27.1 per cent [31-3.9]. Obviously, the ranges (shown below) overlap creating uncertainty about which of the main parties was actually in first place:

NDP: 40.9--37--33.1
Libs: 34.9--31--27.1
PCs: 31.9--28--24.1

A glance at the figures shows that either the NDP or the Liberals could have been in the lead. For example, this result is also within the poll's margin of error:

Liberals: 34.9, NDP: 33.1, PCs: 31.9.

(Note: For the sake of greater clarity, I've taken the Green Party's marginal support out of these calculations.)

Looking at the poll more critically

But wait. Another simple math calculation casts even more doubt on what this poll shows about the possible outcome of the Nova Scotia election. CRA reports that 30 per cent or nearly a third of the 627 voters surveyed, were undecided, did not plan to vote, or refused to state a preference. That means that 188 of the voters in the sample didn't state a preference for any one of the three main parties. [627x.30=188.1] Therefore, the poll results are actually based on the opinions of only 439 voters. [627-188]

In the statistical science of polling, margins of error rise as sample sizes shrink. It only stands to reason that the fewer people in the poll sample, the greater the uncertainty about overall party support. This uncertainty is reflected in the margin of error. For a survey of 439 voters, for example, the margin of error is plus or minus 4.6 percentage points, 95 times out of 100. (Information on calculating margins of error for different sample sizes can be found in the endnote of Understanding Polling Methodology by Matthew Mendelsohn and Jason Brent.)

Here's how the main political parties stack up using this new, more accurate, margin of error based on the sample size of 439 decided voters:

NDP: 41.6--37--32.4
Libs: 35.6--31--26.4
PCs: 32.6--28--23.4

Now, there's an even-greater potential overlap in party support with the PCs at the top of their range slightly ahead of the NDP at the bottom of theirs. In the following scenario, the Liberals would be in the lead:

Libs: 35.6--PCs: 32.6--NDP: 32.4

Drawing conclusions from polls

Yes, caution is the watchword when interpreting opinion polls, especially when sample sizes are small and margins of error are wide. Any way you slice it, a sample size of 439 decided voters is skimpy at best. (Even 627 is nothing to write home to Mom about.) I'm not saying that the CRA poll is necessarily wrong. On June 9th, the NDP may win --- or it may not. Scientifically, this poll shows that almost anything could happen.

Wark's fearless prediction

Based on the poll, here's my fearless prediction. The NDP will win anywhere between 41.6 and 32.4 per cent of the overall vote; the Liberals will win anywhere between 35.6 and 26.4 per cent; the PCs will win anywhere between 32.6 and 23.4 per cent while the Greens will win anywhere between 7.6 and 0 per cent. So, based on the CRA poll, I'm predicting the NDP will win the election, unless either the Tories or Liberals do. Anyone want to bet a crisp, Mulroney $1,000 bill that I'm wrong?