What the BC Election Outcome Means to Abacus Data
What the BC Election Outcome Means to Abacus Data – “We are up for the next polling challenge”
The results of the BC election were a shock to most in the polling industry. However, with constant refinement and improvement we are up to the challenge of accurately predicting the next election.
While Abacus didn’t conduct any polling in the province after the leaders’ debate on April 29, I was confident that what other firms were finding in the final week of the campaign was an accurate reflection of what was going to happen on election day.
The failure of the final polls to anticipate a BC Liberal win presents an opportunity for us to learn from what happened in BC and improve our methods as an industry.
I don’t agree with the commentary that online research cannot measure public opinion as accurately as live telephone interviews.
- Our own record using internet research in the last Ontario and Federal elections is excellent. In Alberta, we didn’t use online research (we used IVR) and we stopped polling Thursday, four days before election day.
- Internet researchers were among the most accurate in forecasting the popular vote in the last U.S. Presidential Election
But as an industry, we are judged by our last election and so it is time to reflect on what we can do better.
I personally have learned a few things from the BC election that will change the way we do election polls in the future.
1. There’s no such thing as a decided voter until they have voted.
Most pollsters only report decided voters; those respondents who have a clear vote choice or are leaning to one party. Those undecideds are then dumped into the recycling bin.
In the past, we reported vote intention results among decided voters and among all respondents but the undecided voters are reported with those leaning towards a party included.
After polling in Alberta and seeing what happened in BC, I have decided that the term “decided voter” should be tossed aside and all respondents to our survey should be reported in our polling.
Going forward, Abacus Data will only report the vote intention of all respondents and we will not release results for “decided” voters. Doing so ignores an often large group of voters who may lean towards a party but aren’t ready to commit on the first “vote intention” question. Tracking the size of this group within an electorate is important. We will do so during an election and non-election period polling.
Moreover, we will separate out leaning voters from those with a clear first choice so we can understand how other undecided voters might vote if they decide to vote in an election. In BC poll, 22% of respondents said they were undecided on the first vote intention question. In the future, we will pay more attention to who they are and what they think about issues beyond their vote choice.
2. We need to pay more attention to those who do not have a clear vote intention (formerly undecided voters).
What we know from exit polls or post-election surveys in BC and in Alberta is that many voters made up their minds in the final days of the campaign.
If we don’t pay attention to undecided voters, we won’t know some of the underlying attitudes of these voters. While many undecided voters probably won’t vote, some will, and they can be decisive, especially in a volatile political environment with a disliked incumbent but an equally disliked alternative.
Looking specifically at the 225 “undecided” respondents in our survey (before asking the follow up question if they are leaning towards a party) we find that they are probably more likely to vote BC Liberal if they are mobilized and the campaign is effective at setting a ballot box that favours the incumbent Liberals.
Here are some examples of what these undecided voters told us:
- Direction of the Province – right direction 24%, wrong direction 28%, unsure 48%
- Would never vote BC Liberal – 16% / Would never vote BC NDP – 18%
- Impression of Christy Clark – Favourable 11%, Neutral 44%, Unfavourable 36%, Unsure 9%
- Impression of Adrian Dix – Favourable 8%, Neutral 48%, Unfavourable 33%, Unsure 11%
- Views on Northern Gateway – 26% completely opposed, 57% support pipeline with conditions, 18% completely support pipeline
- 2009 Provincial Vote – BC Liberal 22%, BC NDP 14%, Other parties 3%, Did not vote/Can’t remember 61%
- 2011 Federal vote – CPC 20%, LPC 13%, NDP 9%, GPC 3%, Can’t remember 29%, Did not vote 27%
- Evaluation of current economy – Very good/good 17%, Ok 56%, Poor/very poor 27%
- Trust most to manage BC economy – Liberal 16%, NDP 3%, Conservative 3%, Other 1%, Unsure 78%
During the BC election we also released a report that profiled the BC electorate according to their views on a number of issues. You can read the full report here.
The segmentation produced five unique voter groups that we named based on their issue clustering. Like some of the data points raised above, these undecided voters leaned to the right of the political spectrum. Undecided voters were distributed into these voter clusters in this way:
- Pro-business liberals – 37%
- Conservatives – 16%
- Secular pacificists – 14%
- Populist social democrats – 21%
- Progressives – 11%
We should have paid more attention to these voters and we had the data to better understand how they might vote if they were activated or decided to turnout. The majority of them had views that aligned more closely with the BC Liberals and many of them had voted for them in the past. This should have been a warning sign.
Our lack of attention to undecided or volatile voters should also make us question the value of the traditional vote intention question itself.
True, the ballot question measures vote intention at the moment of the survey and is simple and direct. But it does not measure the likelihood that a person’s vote intention is soft, temporary, or volatile.
During the 2011 Ontario provincial election we were attacked by our competitors for using a new question to gauge vote intention. Instead of asking which party the respondent planned to vote for we asked them to rate the likelihood that they would vote for each of the main parties contesting the election.
The objective was to use a measure I thought would capture party choice but also the intensity of support. This type of question is best used in online surveys since the respondent can visually and directly compare their vote choice and rationalize it. On the telephone this is very difficult to do.
Using the intensity of support measure, our first public poll of the 2011 Ontario election found that the Progressive Conservatives were leading the Liberals by six points among all respondents (PC 30%, OLP 24%, NDP 15%, Green 4%, Undecided 22%). This result was determined if a respondent rated the likelihood of voting for one party higher than another. Removing those that had not made up their mind yet, the ballot results would have been PC 41%, OLP 32%, NDP 20%.
Our final poll using the same methodology (but not released publicly) had vote intention at OLP 29%, PC 26%, NDP 18%, and GPO 2% with 25% of respondents not preferring one party or the next. When we remove those undecided voters we got OLP 38%, PC 34%, NDP 24%, GPO 3%, and other at 1% – almost exact to the actual results of the election. In Ontario, it seems that the 25% of our respondents who did not have a clear preference either stayed home or if they did vote, voted in similar proportion to those who did have a clear preference, reinforcing the need to understand these “undecided” voters.
We didn’t use this method during the BC Election but I really wish we did now.
3. We need to separate polling results by population and projected electorates, especially during elections.
Eric Grenier at threehundredeight.com did a very good job explaining this point on a recent post. For our mid-campaign poll, we (like Ipsos, Angus Reid, and others) also weighted according to the BC population and not the expected electorate that would show on election day.
Our poll for Sun News found that voters aged 60 and over were much more likely to vote BC Liberal than those younger than 60 but we still had a slight lead for the NDP among those older voters. We need to keep in mind that this was still early in the campaign.
Some firms that developed likely voter models did not come closer to forecasting the correct final results indicating that their model was incorrect or something was happening with their data collection.
There are two ways you can model turnout: (1) Develop a series of questions that identifies the likelihood a voter will vote based on past behaviour, knowledge about the election, and even where their voting place is. Or, (2) you can weight your data based on known demographics of voters from the past election. Ipsos did that for its exit poll of BC voters.
In future election poll, we will release results for a number of voter turnout scenarios.
4. Pay more attention to other variables that match up with the themes/issues of the campaign.
The BC Election also taught me to listen to my gut a little more and pay attention to other variables apart from the standard vote intention question.
Our mid-campaign poll found that the NDP had a 10-point lead over the Liberals among those who had a clear voting intention at the time.
But some other findings suggested that the NDP lead might be in trouble and the BC Liberals had an opportunity to turn around the election (also note this survey was conducted before the debate).
- The favourability ratings for Christy Clark and Adrian Dix were almost identical. Christy Clark was not personally popular at the time but neither was Adrian Dix (Clark 24% positive, Dix 23% positive).
- Dix was much less liked among those who intended to vote NDP than among those who intended to vote Liberal (Dix among NDP supporters 51% positive, Clark among Liberal supporters 61% positive).
- Evaluations of the provincial economy were not that bad and the numbers were similar to what we saw in other provinces where incumbents were re-elected (current economic evaluation – 26% very good/good, 48% ok, 26% poor/very poor).
- The BC Liberals were trusted the most to manage the economy: 27% BC Liberals vs. 21% BC NDP. More importantly, 43% of respondents were unsure which party they trusted most – a key stat when we consider how the ballot question evolved throughout the campaign from “change” to “it’s about the economy”.
On the Sun News Network we focused on these numbers quite a bit but much of the other media ignored these warning signs that all was not well for the NDP on the road to victory. Voters were still telling us they planned to vote NDP but its support was built on a weak foundation. Dix wasn’t liked personally, economic evaluations were relatively good for the incumbent government, and the Liberals were most trusted on the economy.
5. Don’t stop trying new things
One of the main lessons I learned in all this is not to ignore your instincts by being pushed around by bigger competitors.
I believe we (public opinion researchers) can measure public intentions and opinions accurately.
The BC and Alberta elections taught me many lessons, but the most important lesson I learned was not to not stop trying new things, even when they go against convention wisdom and the “way things are always done.”
Abacus will keep looking for new ways to understand the world around us. Science is about constant refinement and improvement and we are up to the challenge.