Yesterday, Forum Research released the results of a survey of eligible voters in Toronto Centre along with the three other by-elections being held on November 25.
The survey found that Liberal candidate Chrystia Freeland has an 11-point lead over NDP candidate Linda McQuaig with Freeland at 46% and McQuaig at 35%. Since Forum’s earlier survey, the Liberals are up one point while McQuaig is up five. This looks to be a safe seat for the Liberals.
I suggest we should take this poll with an especially large grain of salt.
First, I have mixed feelings about IVR surveys having used them during the 2012 Alberta election. I think those who answer automated surveys are different from the general population. But I haven’t done enough research to test their comparability with other methods but my experience tells me we should be cautious with the results, especially when we can’t assess how the data was weighted.
Second, conducting a survey in a riding like Toronto Centre is very difficult because it’s likely that much of the population do not have land lines. The population in the riding is much younger than the rest of Canada made up of students, young professionals, and low income residents who likely opt for a cell phone only instead of paying the extra $20-$30/month for a land line. The problem with this is how expensive it is to find those cell-only households in a small geographic location like Toronto Centre.
Polling firms cannot buy cell-phone sample with a geographic location attached to the telephone number. Therefore, if we want to find cell-only respondents in Toronto Centre, we would have to screen out residents by calling random 416/647 area codes. That’s a very expensive and time consuming process – so I think it’s fair to assume that the Forum Research poll only includes those with a land line and excludes those eligible voters who only have a cell phone.
In some ridings or regions of the country, excluding cell-only households is not a problem. In Nova Scotia for example, we did not include cell-phones in our sample because I felt it would not have improved our forecasting of that election.
But in Toronto Centre, I think if you exclude cell-only households, you are missing a large and important group of eligible voters.
Consider the demographics of the riding.
According to the 2011 Census, the age distribution of those 18 and over in Toronto Centre is:
18 to 34 = 38%
35 to 44 = 18%
45 to 54 = 17%
55 to 64 = 13%
65+ = 14%
This is a very “young” electoral district compared to others across the country. Nationally, Canada’s population by age 18 and over is:
18 to 34 = 28%
35 to 44 = 20%
45 to 54 = 20%
55 to 64 = 15%
65+ = 17%
Forum Research’s unweighted count by age group is:
18 to 34 = 7%
35 to 44 = 12%
45 to 54 = 19%
55 to 64 = 25%
65+ = 38%
Clearly, it’s sample is heavily biased towards older voters. Statistical weighting can help solve this problem. But since they don’t release their weighted counts, I can’t confidently recreate their weighting formula.
Re-weighting Forum’s Data by Age
If we weight their data according to the age distribution based on Census figures, I get the following vote intention:
NDP = 39.7%
Liberal = 39.3%
Conservative = 16.1%
This is a much closer race than the 46% to 35% reported in the Forum Poll’s release.
Now, this assumes that younger voters will make up the same proportion of the electorate as they do in the population of Toronto Centre. Evidence from the 2011 election turnout estimates by Elections Canada underlines why that is a bad assumption. It is highly unlikely that 38% of the electorate that votes in the by-election on November 25 will be made up of 18 to 34 year olds.
But what if turnout rates by age match those from the 2011 election? This all assumes of course that Forum’s vote preference estimates by age group are accurate as well considering that they do not likely include cell-phones in their sample.
If we weighted the Forum Poll results by estimated turnout by age from the 2011 election, the vote preference results are:
Liberal = 42%
NDP = 38%
CPC = 16%
GPC = 4%
If you’re wondering how I did this, you can see here.
What this means?
This is simply a statistical excerise that assumes that Forum’s vote preferences by age group are accurate, but our calculations suggest that the race in Toronto Centre could be closer than the Forum Poll suggests – that is if electorate that turns out to vote on November 25 or in the advanced polls more closely matches the age distribution in the electoral district. Lots of unknowns and the difficulty of measuring voter preferences in an urban, youthful electoral district, makes this riding still too close to call in my opinion.