Polls Missed The Mark In 2016. But Experts Say Things Are Different In 2020.

On election night in 2016, when Republican Donald Trump captured the White House, a lot of people were surprised.

For months, state and national political polls had shown Democrat Hillary Clinton winning the presidency. Forecasters who used those polls to make Election Day projections pegged Clinton’s chances at 99 percent (from the Princeton Election Consortium), 85 percent (from the New York Times), and — on the low end — 72 percent (from the data-crunching site FiveThirtyEight).

“Clinton’s chance of losing is about the same as the probability that an N.F.L. kicker misses a 37-yard field goal,” read the New York Times story.

But it turns out the polls at the root of all those projections were misleading at best, and flat-out wrong at worst.

Charles Franklin, the director of Wisconsin’s most well-known political poll, the Marquette University Law School poll, says he “feels the pain” of people who felt burned by polling in 2016. The last Marquette poll that year showed Clinton winning Wisconsin by 6 points.

“Errors can occur,” Franklin said. “It’s a very good lesson to us all not to exaggerate the accuracy of polling.”

But Franklin and many of his peers have spent the last four years examining and trying to explain what happened in 2016.

They argue 2020 will be different.

What Pollsters Have Changed Since 2016

First of all, it’s important to note something many pollsters are keen to mention whenever you bring up 2016: Nationwide polls were largely correct that year.

And that is true. A RealClearPolitics average of national polls just before Election Day showed Clinton winning the national popular vote by about 3 percent. She ended up winning it by 2 percent.

Of course, the national popular vote doesn’t translate to winning the White House. Clinton’s vote totals weren’t enough in many states to win their Electoral College votes, thereby dooming her bid for the presidency.

President-elect Donald Trump gives his acceptance speech during his election night rally, Wednesday, Nov. 9, 2016, in New York. John Locher/AP Photo

State-level polls in 2016 were a different story. Over the past four years, experts reached an academic consensus about those polls’ biggest flaw: a failure by many to “weight for education.”

“Weighting for education” is an attempt to compensate for the fact that college-educated people are both more likely to respond to polls and more likely to be Democrats. A poll that’s weighted for education takes responses from people without college degrees and gives them more “weight,” so the poll more accurately represents the demographics of the place it’s trying to survey.

For example, here’s how a Wisconsin poll that’s weighted for education would play out:

Pollsters call thousands of people, but many don’t answer their phones. Of those who do answer and take the survey, 60 percent have a college degree. But here’s the problem: only about 30 percent of Wisconsin’s population has a college degree, according to the U.S. Census.

If the poll published those results without any adjustments, it would disproportionately represent the opinions of college-educated voters, who tend to be more liberal than their neighbors without college educations.

So, the poll takes the responses from the people without college degrees and gives them more “weight,” until the demographic ratios in the poll align with the demographics of the state.

According to Ashley Kirzinger, associate director of public opinion and survey research at the Kaiser Family Foundation and a member of the American Association for Public Opinion Research’s poll transparency project, a number of state-level polls didn’t weight for education in 2016.

But she said many have now course-corrected.

“If polls get it wrong this year, it’s not going to be because they weren’t weighting for education,” Kirzinger said.

According to an analysis of 2020 state-level polls by the New York TImes earlier this year, 46 percent of the more than 30 state-level polls released between March and May weighted for education, compared to 20 percent in 2016.

The Monmouth Poll, a survey based out of Monmouth University in New Jersey that does state-level polls in closely watched states like Pennsylvania, Florida and Arizona, along with a number of others, was one poll that made the switch.

Patrick Murray, director of the poll, said Monmouth hadn’t weighted for education before because it didn’t believe there was accurate data available about education levels in certain states. Murray also said in an email interview that, “prior to 2016 there was no significant difference between how people with and without a college degree voted.”

“The latter condition changed dramatically in 2016,” he said.

Now, Monmouth weights for education using self-reported data from respondents, along with U.S. Census data.

According to Franklin, the Marquette University Law School poll has always weighted for education (among other things, like age). He says if his poll hadn’t weighted for education in 2016, its final October 2016 prediction of a 6-point Clinton win would have been even further off.

Charles Franklin is the director of the Marquette Law School Poll. Angela Major/WPR

“It helped us be less wrong than we would have been,” Franklin said.

He said, as it turns out, the Marquette poll — and many others — had other major factors to contend with in 2016.

Complicating Factors In 2016: Undecided Voters And Turnout Changes

In the final Marquette poll of 2016, which was done about a week before Election Day, Clinton led by 6 points among likely voters. But 7 percent of likely voters said they were undecided — that includes people who said they wouldn’t vote for any of the candidates provided by the pollster and those who refused to answer.

Ultimately, those undecided voters broke heavily for Trump, according to exit polls on Election Day and a number of post-election studies, including a comprehensive review of 2016 polling by the American Association for Public Opinion Research.

In a press conference about the AAPOR study, Courtney Kennedy, director of survey research at Pew Research Center, said undecided voters typically split down the middle between the two major party candidates. In 2016, that wasn’t the case.

“In 2016, they broke decidedly for Donald Trump,” she said.

Some people have attributed the late surge of undecided voters to Trump in 2016 to the existence of so-called “shy” Trump voters — people who planned to vote for Trump but didn’t want to tell a pollster because of his controversial public image.

However, a number of studies have refuted the “shy Trump voter” theory. Many experts argue if shy Trump voters existed, polls conducted online, which feel more anonymous, would have shown different results than polls done over the phone.

“If there was a single methodological flaw (with polls), like shy Trump voters, we should have seen the electronic polls were right and others were wrong,” Franklin said.

Aside from the Trump-heavy swing of undecided voters, unpredictable voter turnout is considered the second notable complicating factor for some state-level polls in 2016.

“(Turnout is) not the largest of the reasons, but it was definitely a factor,” said Andrew Mercer, a senior research methodologist at the Pew Research Center.