Making Polls Relevant: Confessions of a pollster
Today in the Times and Transcript, Alec Bruce argues the news media’s attention to polls has weakened the relevance of political journalism. He writes, “What’s particularly edifying or newsworthy about polls that show Messrs. Harper and Ignatieff enjoy essentially the same level of popular support today that they did last week, last month or even last year? What am I to conclude about findings whose variances fall within their own statistical margins of error?”
He goes on to criticize pundits for trying to break down the numbers, explain away the lack of movement for one party and the sustained support for another. The main thrust of his argument is well taken and as a pollster I’m neither appalled nor do I disagree with his arguments.
We have become obsessed with the top of line ballot numbers. Every pollster releases his or her take on where the federal parties stand; showing marginal changes from week to week. What’s worse, pundits and media “delve” deep into the numbers by looking at correlations between the vote choice numbers and demographic and geographic variables. There are two problems with this: (1) the sample sizes are too small to deliver any meaningful insights and (2) simple crosstabs don’t tell us much about why Canadians are leaning one way or another.
As a new pollster trying to get noticed, I admit the seduction to enter the fray and release our own ballot numbers is tantalizingly strong. While I admit, as a political junkie, it’s interesting to see who’s up and down week to week, I think most publicly available opinion research in Canada isn’t answering the fundamental questions that help us to understand what the heck is really going on in Canada. What are Canadians feelings? Are we mad as hell or just cautiously optimistic? Are we worried our savings are drying up or do we see the light at the end of the tunnel?
It’s one thing to say that support for the federal parties is and has been stagnant for the past year or so or that we aren’t that excited about any of the current party leaders. But it’s quite another to ask questions that probe at our country’s mood and use methods that really explain what is driving those attitudes.
When I decided to leave academia and enter the private sector I promised myself I wouldn’t be an average pollster.
I’m a young number cruncher in a older pollster’s world promising to do things a little differently. I won’t be a pundit and I’ll only speak about something when I have numbers to back it up. I think that’s what my clients expect and what the Canadians are looking for in a new generation of public opinion researchers.
There’s a need in the public opinion and marketing research world for a fresh perspective that not only asks questions in a different way but sees different things in the answers. When others describe “young” people as “uninterested”, “disengaged”, or “entitled”, I say we are “optimistic”, “restless”, and “looking for inspiration”. The difference is how you look at the situation.
So to Alec Bruce and other frustrated Canadians I say: throw down the gauntlet, demand more from our news media and our pollsters. But don’t discount the importance or the value of high-quality research and analysis. Polls help us to understand where we collectively stand relative to one another. We just need to ask different questions and look at your answers in different ways.