Polls and the Crisis of Confidence

      2 Comments on Polls and the Crisis of Confidence

By Jonathan McQuarrie

Why do newspapers support the public-opinion polls?…Not only do the modern polls, based on a small, carefully selected cross section, provide more accurate measurements; they can be applied to give continuous and rapid measurements of public opinion at all times. -George Gallup and Saul Forbes Rae, The Pulse of Democracy, 1940, 119.

So called ‘pollsters’ should hang their heads in shame. It’s time to quit whoring out the profession and get out of the media polling game. –Allan Gregg, tweet on 12 June 2014.

What a difference sixty years makes. On the night of the Ontario election, Allan Gregg, the well-known pollster and political pundit, made his displeasure with the inaccurate polling results perfectly clear. Tweets, of course, must always be taken with a several grains of salt. A more measured critique would certainly avoid use of ‘whoring,’ an ugly verb laden with gendered, moralizing judgments, especially following debate over the Conservatives’ highly controversial Bill C-36. However, Gregg has a long track record of being increasingly critical of his former profession. Following Christie Clark’s surprise victory in British Columbia’s 2013 election, Gregg told Canadian Press that the errors “should not happen.” Gregg is not the only commentator who has been sharply critical of polling, but he is the most prolific.

As Daniel J. Robinson’s book The Measure of Democracy: Polling, Market Research, and Public Life 1930-1945 (Toronto, 1999) makes clear, George Gallup and Saul Forbes Rae’s spirited defence of polling as serving public interests needs to be taken with as much salt as Gregg’s tweet. Early pollsters like Gallup’s American Institute of Public Opinion (AIPO) — and its Canadian branch — were soon shaped more by commercial research and committed to market metaphors than by any idealistic commitment to representing the voice of a broad public opinion. In that sense, Gregg’s position is a more honest representation of polling firms and their major function, which is market research.

Dewey Defeats Truman?, 1948. Source: Wisconsin Historical Society

Dewey Defeats Truman?, 1948. Source: Wisconsin Historical Society

Not long after Gallup and the AIPO confidently asserted the case for their scientific samples, pollsters made their most famous error, confidently picking Republican candidate Thomas Dewey over the actual victor, Harry Truman in the 1948 US presidential election. However, despite the scandal and temporary setbacks, pollsters rallied. According to the political scientist Amy Fried, the APIO overcame the embarrassment by establishing the Social Science Research Council’s Committee on Analysis of Pre-Election Polls, which received funding from the Rockefeller Foundation. Pollsters forged relationships with business and social scientists alike to repair their reputation.

Similarly in Canada, the Canadian Institute of Public Opinion struggled to predict elections in Canada during the 1940s, and yet polling gained credence as the best way to measure public opinion and shape debates. Brooke Claxton, the McGill professor turned Liberal cabinet minister, fostered and entrenched the alliance between polling, social sciences, and the Liberal party, ensuring that polling continued to play a key role in Canadian politics.  Despite the setbacks of the 1940s, polling enjoyed an enhanced reputation on both sides of the border.

This is all to suggest that polling has a long-standing tension between the seeming precision of its estimates and its actual results. However, there is reason to suspect that polling may not recover as neatly from this current crisis as it did during the 1940s. The 1948 debacle took place in a different context, when the state and universities were expanding, and social scientists enjoyed unprecedented influence. This was the age of the technocrat, and numbers were among their most prized tools. Gallup’s argument that polls reflected facts about public opinion was welcome in a period that sought to quantify and rationalize. This is not to suggest that the technocratic era was one of unmitigated success for the social scientist and expert; of course, there were limits, conflicts, failures. However, Gallup and other polling firms had a large number of academics, journalists, and businessmen come to support them, even after their largest failure. Polls did not have to be entirely accurate in order to be defended.

Such support does not seem as forthcoming in our more skeptical times. In sharp contrast to the 1940s and 1950s, experts in a variety of fields seem to be on the defensive. The gulf between understandings about global warming and action is staggering. Universities increasingly orientate towards vocational training in an effort to remain relevant, while disciplines with more muddled career paths decline. The CBC gives more time to a blustering, defiantly evidence-allergic Kevin O’Leary while it cuts funding for investigative reporting. The political scientist Brendan Nyhan has produced “depressing” studies indicating how little impact evidence had on changing minds about the effectiveness of vaccines. To use a bit of jargon (another tool of the expert), expertise is becoming contested terrain. Skepticism over polling is a symptom of this.

While the reputation of polling might decline alongside other forms of social scientific knowledge, such a decline is not entirely a bad thing. Technocracy is often predicated on power hierarchies based on gender, class, and race. More specifically, the power of polling was such that they created considerable anxiety over whether they in fact curtailed democracy. For instance, one can argue that rationalizing technologies that seek to quantify and aggregate opinion into narrowly defined categories cause political alienation. In weighing the impact of the rise of polling in the book Numbered Voices: How Opinion Polling Has Shaped American Politics (Chicago, 1993), Susan Herbst noted, “it is likely that rationalized techniques flourish where ideology and interest in politics are dormant.” (156). Polls also had the danger of lending superficially scientific credence to ideological positions, to the point where ideological and instrumental meaning of polls became intertwined. Polls were never the passive measuring tools of public opinion conceived by Gallup; they could shape as they measured.

Now, many pollsters challenge that there is a crisis. Éric Grenier, a Globe and Mail contributor and operator of threehundredeight.com, a website that aggregates, and assesses data from multiple polling firms, has offered a partial defense of the polling predictions. In a ‘post mortem’ of the Ontario poll results, Grenier noted that the major polling problem for this election were the efforts to assess the ‘likely’ voters, noting that those polls particularly overestimated Progressive Conservative support. He also noted that his riding model only called ten seats incorrectly. Grenier’s efforts to make his projections transparent by writing at length about his methodology and presenting ranges of possibilities with assessments of their likelihood are highly commendable. It helps to bridge the distance between pollsters and the public they seek to analyze.

Still, it seems that Gregg’s disgust and skepticism about the field captures wider sentiment, while Gallup and Rae’s book increasingly reads like a relic of another era.  The dominant skepticism and rise of opinions-as-facts undermines any project that purports an ability to measure public opinion in a neutral and factual manner. Such skepticism has negative consequences, including the decline of evidence-based argument. However, it also makes Gregg’s criticisms more potent, as it invites consideration of the long links between polling and marketing and the role of polling in our politics.

Jonathan McQuarrie is a PhD Candidate at the Department of History, University of Toronto. He tweets about things historical and not at @jrmcquarrie. 

2 thoughts on “Polls and the Crisis of Confidence

  1. Pingback: Are the Repeated failure of Polls to Predict Canadian Elections due to a form of Observation Bias? | David Zylberberg

  2. Richard White

    While I agree that expertise has indeed become contested terrain (I would not say “is becoming”) I do not think this is what underlies skepticism about polling. To me our doubts about polling are actually quite evidence-based. For years, without knowing much about how pollsters obtained their data or were paid for their work, most of us rather uncritically accepted that they knew what they were talking about. In other words we accepted their expertise. But in the last year, with pollsters miscalling three major provincial elections, we are beginning to realize that they do not. The evidence is causing us to doubt their expertise, not the reverse. If we had such unequivocal evidence linking autism and vaccination, the anti-vaccination movement might be easier to understand.

Leave a Reply