Mark Mellman: Exclude the excluders

It’s long past time the press stopped publishing polls that do not sample cellphones. 

If an enterprising pollster offered a survey that, say, excluded everyone ages 30-44 or those over age 65, no one would pay any attention to it — and rightly so. Yet news outlets regularly publish polls that arbitrarily ignore a similar proportion of the population. A recent study found 39 percent of adults live in households with cellphones only, and no landline. 

{mosads}Unfortunately, turnout rates being what they are, adults and voters are two distinctly different groups. 

Exit polls help bridge the gap. In 2010, they found one in five voters living in a cell-only household, and in 2012 that number increased to one in three.

Demography accounts for some of the discrepancies: those who only have cellphones are much more likely to be younger, poorer and renters — all segments that are less likely to vote. Age differences are particularly striking. Sixty-six percent of those ages 25-29 are in wireless-only households compared to just 14 percent of those ages 65 and older. There is also considerable variation by state. More than half of Idahoans (52 percent) live in cell-only households, compared to 19 percent in New Jersey.

In addition, some have a landline, but don’t answer it. Another 16 percent of the total public is categorized as “wireless mostly,” according to a Centers for Disease Control and Prevention survey. 

None of this would matter much if wireless-only (or mostly) voters behaved like landliners. They don’t. In 2008, President Obama won wireless-only voters by 23 points, while tying John McCain among those with both landline and mobile service. In 2010, Senate Leader Harry Reid led by 19 among those we reached on cellphones, but was tied among those who responded on landlines.

Indeed, this issue goes to the statistical heart of why polls work. Polls can only be justified as representative of a population if they are a random sample — that is, if everyone has an equal probability of being contacted. If one arbitrarily excludes 20 percent or 30 percent of the population, the principle of randomness is violated and there is no reason to believe the survey will prove accurate. That is not to say it will be wrong, but it does mean there is no scientific reason to believe it will be right.

The indictment of polls that fail to include cellphones is strong. But some readers who keep up with such things are no doubt asking why I am taking on this issue again now. After all, many, though not all, of the robo pollsters — the pollsters most likely to excise wireless voters from their samples — have recognized the error of their ways and done penance by altering their sampling procedures. 

Just how have they changed? Many are now adding Internet interviews from (non-random) panels to their sample. The problem with this approach is simple: there seems to be no evidence that Internet panels represent in any way the wireless-only population the polls are excluding. Indeed, what evidence there is suggests they don’t.

For example, Internet panelists tend to be higher income, while cell-only households are poorer. Cell-only voters are more likely to be minority, while online panels are whiter. More relevant, there is no evidence that the political attitudes of online respondents reflect those of cell-only voters. Indeed, David Johnson of Discovery Research Group reported “online access panel users were found to be more like landline respondents than cell phone … respondents.” 

So it turns out that the solution is just another problem. It’s akin to saying, “we’re excluding seniors, but don’t worry, we are getting people who make over 50K instead.” It just doesn’t help.

Until those who don’t reach wireless voters develop a real solution, let’s discourage them by paying no attention. 

Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.