Poll-inspired cockiness and its hazards

In the buttoned-down field of survey research, Warren Mitofsky is remembered as something of a legend. In the 1960s, he developed the exit poll for CBS News, a technique in which voters are interviewed as they leave polling places. Mitofsky, who died in 2006, also helped initiate random-digit dialing by telephone, a randomized sampling method that revolutionized survey-taking and was polling’s gold standard for years. 

More so than his colleagues, Mitofsky was a student of the history of survey research. In speeches, he was known to refer to such pioneers of the field as George Gallup and Elmo Roper. Mitofsky kept displayed on a wall the famous photograph of President Harry Truman in victory, holding aloft the front page of the Chicago Tribune that erroneously declared he’d lost the 1948 presidential election to Republican Thomas E. Dewey.  

“There’s still a lot of room for humility in polling,” Mitofsky said in an interview at the 50th anniversary of the 1948 polling debacle. “Every time you get cocky, you lose.” 

It’s an admonition that rings true across generations. And it speaks to an intermittent affliction that can be called “poll-inspired cockiness” — the extreme confidence in election outcomes as signaled by opinion polls. 

The periodic flaring of poll-inspired cockiness is a cautionary reminder for journalists and analysts this year, given the sizable leads former Vice President Joe Biden has built in national surveys. After all, polling’s history is studded with cases of elections that defied poll-based expectations.

Poll-inspired cockiness certainly helps explain the deep shock that accompanied Donald Trump’s election four years ago. The cockiest poll-related moment of the 2016 campaign centered around a vow by Samuel Wang, the neuroscientist who runs the Princeton Election Consortium, a polling-analysis and forecasting site. In mid-October 2016, Wang declared in a tweet that the race was “totally over,” adding: “If Trump wins more than 240 electoral votes, I will eat a bug.”

Trump won 304 electoral votes.

Wang and other poll-based forecasters placed considerable confidence in state-level surveys, which went awry in key places such as Wisconsin, Michigan and Pennsylvania. Polling in those states pointed to Hillary Clinton’s likely, even easy, victory. By narrow margins, Trump won them all, and with them the presidency. 

Wang went on CNN five days after the election to redeem his “eat a bug” promise. He brought with him a can of gourmet crickets. “A lot of people were wrong,” Wang said about the presidential election, “but nobody else made the promise I did.” He looked a bit sheepish as he scooped a sample from the can and popped it into his mouth. 

Wang was scarcely alone in his misplaced confidence about the polls and what they were indicating in 2016. The Reuters/Ipsos poll estimated Clinton had a 90 percent chance of winning. Huffington Post’s polls-based forecast model set Clinton’s win probability at 98.2 percent, declaring that Trump had “essentially no path to an Electoral College victory.” Clinton, the forecast said, “should fairly easily hold onto Michigan, Wisconsin and Pennsylvania.”

Natalie Jackson, then senior polling editor for Huffington Post, said afterward that she had “placed way too much faith in polls. I assumed they would be right. … I saw no reason to question that the polls would be accurate overall.” 

The punctured certitude was reminiscent of the 1948 election — another memorable episode in poll-inspired cockiness. 

On the eve of the election, the Democratic National Committee chair warned Gallup that his polling was understating Truman’s chances. In reply, Gallup said that on Election Day, “the whole world will be able to see down to the last percentage point how good we are.” Gallup predicted Dewey would defeat Truman by 5 percentage points and roll up a substantial electoral vote majority.

Truman won the election by 4.5 percent points and received 303 electoral votes.

Roper, a rival pollster, was so sure of Dewey’s victory that he stopped releasing polling data two months before the election, saying the race was as good as decided. Roper figured that presidential elections were settled by Labor Day and that campaigns were so much sound and fury. 

After Truman’s unexpected victory, Roper acknowledged that pollsters “had gotten pretty smug, and I was one of the smuggest of the lot.” 

Evidence of poll-inspired cockiness can be found even at the dawn of modern survey research in 1936. That was when Gallup and Roper introduced quasi-scientific polling techniques to presidential races. It also was the year of the Literary Digest’s stunning polling error in the race between President Franklin D. Roosevelt and Republican Alf Landon.

The Digest was a venerable weekly magazine that had conducted mail-in polls of presidential campaigns since 1924 — and had never failed to identify the winner. 

The Digest revealed more than a soupçon of cockiness in announcing the launch of its 1936 mail-in poll, saying: “Who will win — Roosevelt or Landon? …  Today, nobody knows. But the Digest is seeking the answer — in the same way that has enabled it, time after time, to tell the country exactly what was going to happen when the voters went to the polls.” 

The Digest mailed 10 million postcard ballots, 2.3 million of which were completed and returned from around the country. Based on that huge sample, the magazine projected Landon’s comfortable victory.

Landon lost by 24 percentage points. For the Digest, it was a crushing polling failure. 

So what accounts for these outbreaks of poll-inspired cockiness? Why does the syndrome flare from time to time, ensnaring polling analysts, journalists, pundits and others? 

At least part of the answer lies in the sense, or illusion, of precision that often attaches to polling data. The data can have an almost mesmerizing effect in setting and affirming campaign narratives about who is ahead and who is lagging. Although many pollsters insist that surveys are only snapshots of a moment, treating polls as election oracles is a tendency rooted in American politics.

Groupthink — a broad unwillingness to challenge or look beyond dominant narratives — offers another explanation. The impression of certainty projected by polling data can reinforce tendencies toward groupthink, which some analysts say afflicted journalists at leading news-gathering organizations in 2016. To many journalists, Trump’s winning the presidency was unthinkable. An alternate outcome was not widely or seriously contemplated, which suggests that willingness to consider contrary and unpopular views could be an antidote to poll-inspired cockiness. 

W. Joseph Campbell is a professor of communication at American University in Washington. He is the author of seven books including, most recently, “Lost in a Gallup: Polling Failure in U.S. Presidential Elections” (University of California Press, 2020). Follow him on Twitter @wjosephcampbell.