What went wrong with the survey? Some early theories

Another symptom of a deeper failure of this year’s survey is the results of the senior citizen survey. Unlike 2016, polls consistently show Mr. Biden winning by comfortable margins among voters 65 and over. The final NBC / WSJ poll showed Mr. Biden a 23 point rise among the group; the latest Times / Siena poll showed this by 10. In the final account, there will be no reason to believe this was all real.

This is a deeper mistake than those of 2016. It suggests a fundamental misunderstanding of the attitudes of a large demographic, and not just an underestimation of its share of the electorate. In other words, the underlying raw survey data has worsened over the past four years, reversing the changes pollers made to fix what was wrong in 2016.

This helps to explain why the national surveys were worse than in 2016; they made the cut by education four years ago and have made little or no change since. It also helps to explain why the error is so closely correlated with what happened in 2016: it focuses on the same demographic, even though the underlying source of the error within the group is quite different. .

The polls clearly present serious challenges. The industry has always relied on statistical adjustments to ensure that each group, such as white voters without a degree, represents its fair share of the sample. But that only helps if the respondents you reach are representative of those you don’t. In 2016, they appeared to be representative enough for many purposes. In 2020, they were not.

So how have the polls gotten worse over the past four years? This is mostly speculation, but just consider a few possibilities:

The president (and the polls) hurt the polls. There was no real indication of a “Hidden Trump” vote in 2016. But maybe there was one in 2020. For years, the President has attacked the news media and the polls , among other institutions. The polls themselves lost some credibility in 2016.

It’s hard not to wonder if the president’s supporters have become less likely to respond to polls as their skepticism of institutions mounted, leaving polls worse off than they were ago. four years.

“We now need to take a version of the Shy Trump hypothesis seriously,” said Patrick Ruffini, a Republican pollster for Echelon Insights. That would be “the problem with polls that just don’t reach large elements of the Trump coalition, causing them to underestimate Republicans in all areas when he’s on the ballot.”