A key highlight included a major evidence-based global study into the accuracy of polling that concluded that done well, polls overwhelmingly continue to correctly predict election outcomes. A recent analysis by Jon Puleston for ESOMAR of more than 31,000 polls from 473 voting events across 40 countries spanning 1936 – 2017, found that at a global level, the average error of polls conducted within seven days before an election is +/-2.5%. The study included polls from the last four Canadian federal elections and found that Canadian pollsters performed well with average errors below the global average. While there has been some speculation about the potential decline in the accuracy of polling, the ESOMAR study concluded that when examining polling error data over the history of polling “there is no evidence of any long term decline in the accuracy of polling.”

“Because election polls are a litmus test for survey research overall, decision-makers from all sectors need to understand that surveys — when done well —remain an important source of information in decision-making,” said Barry Watson, Chair of CRIC.

The question of how to address the challenges facing polling was a predominant theme in the conference panel discussions, particularly in light of high-profile polling “miss calls” where polls failed to predict the outcome of the events they were measuring. Conference panel members agreed that done poorly, bad or misleading election polls misinform voters and therefore hinders democracy.

While pollsters have a responsibility to produce accurate and unbiased data, industry associations such as WAPOR, CRIC and ESOMAR also owe it to the public and their members to develop world-leading standards for good research and to ensure that these are being adhered to.

“Pollsters have a professional and ethical duty to ensure that the polls they publish follow the most rigorous standards and are supported by best practice methodologies,” stated Marita Carballo, President of WAPOR. “If they don’t uphold the highest standards, then it’s harmful to give them any public attention.”

A stark example of this was presented by Christopher Adams, rector of St. Paul’s College at the University of Manitoba, and co-author of a study into problematic polling conducted during the 2017 Calgary election, where the differences in reported polls was so large that the polls overshadowed the election campaigns themselves and became the election story. A review of those polls by the study’s independent expert panel found “significant methodological flaws.”

Claire Durand of the University of Montreal also presented on the polling miss of the 2018 Quebec provincial elections, in which the Coalition Avenir Québec (CAQ) won decisively (12 points ahead of the next party) despite polls generally showing a close race between the two major parties. In her analysis, Professor Durand found that the polling miss was due to a late campaign swing towards CAQ, a tendency of “discrete” respondents to finally vote in greater proportion for CAQ and a small effect of differential participation. At the conference, Ms. Durand called on the necessity to find ways to anticipate these factors in order to prevent other polling misses to occur.

A consistent recommendation cited throughout the session was the importance of pollsters to be transparent in their reporting. Polling companies are encouraged to share details of their methodologies and be upfront about potential risks to their predictions. Journalists are encouraged to request these details. Additional resources for publishing and reporting on surveys including a free online course on Understanding and Interpreting Polls is included at the bottom of this release. CRIC will have a further statement on our transparency standards and their enforcement as part of future communications in the run up to this fall’s election.

“Election polling is more than theatre; it is about sound democracy and providing the electorate with the ability to make an informed decision,” said Joaquim Bretcha, ESOMAR President. “In today’s era of misinformation, fake news, click bait and foreign interference, fact checking outlier or bad polls is more important than ever.”