May 27, 2019 – TORONTO: The world’s leading research organizations — WAPOR, CRIC and ESOMAR — held a joint conference in Toronto last Tuesday to present and discuss the accuracy of electoral polling globally and in the Canadian context.
A key highlight included a major evidence-based global study into the accuracy of polling that concluded that done well, polls overwhelmingly continue to correctly predict election outcomes. A recent analysis by Jon Puleston for ESOMAR of more than 31,000 polls from 473 voting events across 40 countries spanning 1936 – 2017, found that at a global level, the average error of polls conducted within seven days before an election is +/-2.5%. The study included polls from the last four Canadian federal elections and found that Canadian pollsters performed well with average errors below the global average. While there has been some speculation about the potential decline in the accuracy of polling, the ESOMAR study concluded that when examining polling error data over the history of polling “there is no evidence of any long term decline in the accuracy of polling.”
“Because election polls are a litmus test for survey research overall, decision-makers from all sectors need to understand that surveys — when done well —remain an important source of information in decision-making as they ever were,” said Barry Watson, Chair of CRIC.
The question of how to address the challenges facing polling was a predominant theme in the conference panel discussions, particularly in light of high-profile polling “miss calls” where polls failed to predict the outcome of the events they were measuring. Conference panel members agreed that done poorly, bad or misleading election polls misinform voters and therefore hinders democracy.
While pollsters have a responsibility to produce accurate and unbiased data, industry associations such as WAPOR, CRIC and ESOMAR also owe it to the public and their members to develop world-leading standards for good research and to ensure that these are being adhered to.
“Pollsters have a professional and ethical duty to ensure that the polls they publish follow the most rigorous standards and are supported by best practice methodologies,” stated Marita Carballo, President of WAPOR. “If they don’t uphold the highest standards, then it’s harmful to give them any public attention.”
A stark example of this was presented by Christopher Adams, rector of St. Paul’s College at the University of Manitoba, and co-author of a study into problematic polling conducted during the 2017 Calgary election, where the differences in reported polls was so large that the polls overshadowed the election campaigns themselves and became the election story. A review of those polls by the study’s independent expert panel found “significant methodological flaws.”
Claire Durand of the University of Montreal also presented on the polling miss of the 2018 Quebec provincial elections, in which the Coalition Avenir Québec (CAQ) won decisively (12 points ahead of the next party) despite polls generally showing a close race between the two major parties. In her analysis, Professor Durand found that the polling miss was due to a late campaign swing towards CAQ, a tendency of “discrete”respondents to finally vote in greater proportion for CAQ and a small effect of differential participation. At the conference, Ms. Durand called on the necessity to find ways to anticipate these factors in order to prevent other polling misses to occur.
A consistent recommendation cited throughout the session was the importance of pollsters to be transparent in their reporting. Polling companies are encouraged to share details of their methodologies and be upfront about potential risks to their predictions. Journalists are encouraged to request these details. Additional resources for publishing and reporting on surveys including a free online course on Understanding and Interpreting Polls is included at the bottom of this release. CRIC will have a further statement on our transparency standards and their enforcement as part of future communications in the run up to this fall’s election.
“Election polling is more than theatre; it is about sound democracy and providing the electorate with the ability to make an informed decision,” said Joaquim Bretcha, ESOMAR President. “In today’s era of misinformation, fake news, click bait and foreign interference, fact checking outlier or bad polls is more important than ever.”
About CRIC: CRIC’s mandate is to develop and approve market research standards, best practices and support programs; provide effective promotion and advocacy for the market research and insights industry; serve as a source of information for the industry; and be a forum for collective industry action.For more information, visit www.canadianresearchinsightscouncil.ca or contact John Tabone at email@example.com.
About ESOMAR: ESOMAR is the global voice of the data analytics, research and insights community, speaking on behalf of over 6000 individual professionals and more than 600 companies who provide or commission data analytics and research in more than 130 countries, all of whom agree to uphold the ICC/ESOMAR International Code. Together with national and international research associations, ESOMAR promotes professional standards and self-regulation for the insights sector and encourages, advances and advocates for the role and value of data analytics, market and opinion research in illuminating real issues and bringing about effective decision-making. The ICC/ESOMAR Code and all ESOMAR guidelines are available on www.esomar.org.
ABOUT WAPOR: Since its foundation in 1947, the World Association for Public Opinion has promoted the highest professional standards, ethics and techniques for public opinion research around the world. WAPOR’s international membership, with more than 400 academics and researchers represents the industry’s most respected names in the survey and public opinion research field. Through publications, seminars, meetings and educational initiatives WAPOR engages in a rich ongoing conversation about best practices for public opinion research, not just in advanced democracies, but also in emerging democracies. For more information about WAPOR, publications and events, visit wapor.org.
#CRIC #ESOMAR #WAPOR #MRX #Polling
Resources for Publishing and Reporting on Surveys: