Are we facing another big polling miss? Probably

As we head into crunch time for the midterms, it’s important to remember that one of the great political mysteries of the past eight years remains unsolved: the undercounting of Republican or overcounting of Democratic support in polls.

Over the years, several theories have been offered on why this has occurred, ranging from non-response issues, i.e., shy Trump/Republican voters, problems with likely voter models, and a lack of polling respondents without a college degree.

The American Association for Public Opinion Research (AAPOR) released their “Task Force on 2020 Pre-Election Polling: An Evaluation of the 2020 General Election Polls” and could not determine why public polling, on average, favored democratic candidates by 4 to 5 points in 2020.

The problem with the polls is simple and is a known issue in the insights and opinion research industry.

Over the past decade, the number of fictitious accounts, bots, and click farms employing large-scale automation to answer surveys and polls has rapidly grown.

As opinion researchers have made the necessary transition to online from phone data collection, we’ve become dependent on panel companies recruiting compensated respondents to complete our surveys and polls. Panelists earn money by completing surveys on topics ranging from their planned auto purchases, interest in a new product, or on their voting intentions in the next election.

While survey panel suppliers claim they are taking steps to curb fraudulent respondents from our polls, the industry knows bots and click farm problems are rampant.

Industry research conducted in 2021 on behalf of the Insights Association found that 15 percent to 25 percent or more of completed online non-political surveys have quality issues due to:

1. Individuals/organizations using various fictitious accounts
2. Bots and click farms employing large-scale automation to complete many surveys
3. Professional survey takers who are completing up to 25 surveys per day

These results align with other research presentations I’ve heard at industry conferences for years and are not a problem unique to opinion research.

Spotify is plagued by click farms and fake listeners inflating the listens of a song. App stores for Apple and Google are fighting fake reviews boosting sketchy apps. Amazon’s problems with phony reviews are legendary, and advertisers have long fought click fraud. Fraud is a universal problem across industries, including the opinion research industry.

Over the past two years, my firm’s research found that nearly a third of online poll respondents from 2020 to 2021 were not real average people. We now regularly exclude 20 percent to 30 percent of respondents from our online survey and polls via the same tools online retailers use to detect fraudulent transactions. We no longer rely upon survey panel companies to curb abusive respondents.

The abusive or fraudulent poll responses we found in our polls in 2020 were more likely to indicate support for the Democratic candidate by about 5 to 7 points depending on the state.

In Arizona, abusive responses were more likely to favor Trump in the presidential race and now Sen. Mark Kelly in the 2020 Senate race. In Michigan, Pennsylvania, Minnesota, and Florida, abusive respondents favored the Democratic candidates by 7 percentage points.

Abusive respondents are more likely to indicate they are white, male, aged 25 to 44, and to describe themselves as independents or Democrats.

Male panelists are a valuable demographic. Panel companies have trouble recruiting male respondents of any age group, so there are a lot of incentives not to exclude them from panels when researchers routinely ask for 48 percent of their respondents to be male.

I’ve heard stories of panel providers buying fraud detection software to satisfy our industry organizations but not using them — or doing the bare minimum of quality checks — since removing questionable survey respondents would severely dent their revenue.

How much revenue? In a billion-dollar industry currently awash in venture capital and private equity money, that adds up to $200 to $300 million in fraud each year. There are many incentives to look the other way since it’s too late for many of these companies to develop a conscience after their latest round of funding.    

Fortunately, many tools and resources exist to help determine if a survey response is real, and I encourage pollsters to implement their own fraud detection tools.

Some polls use verified voter samples, which will help. Still, I would urge my fellow researchers to implement additional measures since most of the fraud we see involves proxy servers that could mimic the address of a genuine registered voter.

I’m genuinely worried about this fall and the aftermath of another miss in the industry. The odds that many of the current public polls are undercounting Republican support are pretty high. Our data indicate that the number of abusive respondents is growing, occurs at a higher rate among political surveys, and exists across the dozens of panel suppliers we’ve tested.

During the current era of extreme polarization and declining trust in institutions, delivering authentic and accurate data has never been more critical. Another miss will only create more distrust and cries of “fake polls,” which is something our industry will be challenged to overcome.

Randy Ellison, president of Targoz Market Research, is a public opinion and market research consultant. For more than 15 years, he’s advised clients on consumer and voter attitudes, product development opportunities, communications, and brand strategies.