Werner Krause and Christina Gahn argue that we need to pay more attention to how the media communicates the results of opinion polls to the public. Reporting methodological details, such as margins of error, can alter citizens’ vote choices on election day. This has important implications for elections around the world
2024 is a big election year across the globe, when public opinion polls will come into sharp media focus. The US elections in 2016 and 2020, and the Brexit referendum in 2016, triggered debates about the reliability and potentially 'harmful' power of polls. Recent media reports are again covering the potential pitfalls of misinterpreting polling scores.
These discussions frequently revolve around the assumption that election polls not only mirror the preferences of voters but can also significantly influence them. Hence, while polls can serve as a valuable tool to help citizens make well-informed voting decisions, there is a concurrent risk that these decisions may be based on imprecise or faulty information.
Election polls not only mirror the preferences of voters but can also significantly influence them
Elections are often tightly contested. Since the year 2000, a margin of less than 2.5 percentage points has decided more than a quarter of parliamentary elections in the European Union.
Whether and how polls influence election results has long been subject to academic analysis. This has identified a wide array of potential influences described variously as bandwagon, momentum, Titanic, or underdog effects. Additionally, as the political landscape diversifies and the number of political parties increases, strategic voting decisions become more crucial.
The effectiveness of these voting strategies, however, hinges on the quality of polling estimates, which is challenging for voters to assess. Key questions arise regarding the recruitment of survey participants, whether they represent the voting population, and how many people come out to vote on election day. Moreover, what methods do polling institutes employ to correct potential biases in their samples?
Against this backdrop, polling values reported in the media often show more precision than is appropriate. To address some of these uncertainties, margins of error can illustrate the range within which polling estimates are most likely to fall. Although margins of error do not provide a complete picture of a poll’s reliability, they underscore the need for caution when interpreting polling estimates.
Margins of error underscore the need for caution when interpreting polling estimates
Despite their significance, margins of error usually receive little attention in media reporting. For instance, a study spanning 1980 to 2002 found that only 2% of polling results reported in German media included corresponding margins of error. However, there is a positive trend evident from 2005 to 2017, which shows this figure doubling. According to our data, German media mentioned margins of error in 23% of reports leading up to the 2021 election.
While this data reflects an improvement, the media rarely presents or discusses margins explicitly. Margins are often mentioned only in small text below the polls or at the end of news reports.
We recently published a study in the European Journal for Political Research. Our study, conducted in Germany, delved into the significance of debates surrounding election polls. We employed a survey experiment to examine whether citizens' voting intentions shift when they are shown the uncertainties inherent in polls, particularly when the margins of error are explicit.
We hypothesised that if a political contest is perceived as open, more voters would be inclined to support one of the two largest parties. Such a scenario not only impacts the election outcome but also influences the subsequent government formation process. Our experiment presented participants with various polling estimates. We made the margins of error explicit for some respondents and omitted them for others.
Our results revealed that citizens make different voting decisions based on the presentation of margins of error. When displayed in a closely contested race, more citizens cast their votes for one of the two largest parties. Importantly, these differences in voting decisions are linked to the interpretation of the margins of error. This was evident when the polls were accompanied by an assessment emphasising the openness of the election outcome in relation to the margins of error.
The crucial role of polls in shaping political competition is evident in various additional examples. Former Austrian ÖVP Chancellor Sebastian Kurz and two pollsters are currently facing accusations of financing and publishing manipulated surveys from 2017. Research also indicates that political parties align their campaign strategies with the fluctuations in survey values. Furthermore, polls play a role in influencing the media coverage of political parties during election campaigns.
Research indicates that political parties align their campaign strategies with the fluctuations in survey values
But how should the media present polling values to the public? One approach is to prohibit altogether the publication of polling results immediately before elections. Indeed, according to the European Society for Opinion and Market Research, 46% of countries worldwide already observe this practice. However, such attempts are increasingly ineffective, as we see in the case of the French presidential elections, during which results were published in neighbouring countries, despite a ban. A publication ban also raises concerns related to press freedom.
An alternative solution involves a responsible and transparent presentation of polls by polling institutes and the media. Simply mentioning margins of error is insufficient; instead, we should consider graphical representations of margins of error and provide additional crucial information, such as the number of undecided voters. Reports that delve into and illustrate the uncertainties associated with polls can also contribute positively to public understanding.
Despite ongoing improvements in survey methodologies by polling institutes, statistical challenges persist in every poll. Transparent communication about these issues is paramount, given the multitude of factors influencing poll quality. Our goal should be to empower voters with all the information they need to make their choice. This includes communicating clearly that polls are snapshots of citizens' voting preferences, accompanied by inherent statistical uncertainty.
