After Upset Election, PA Pollsters Mull What Went Wrong
On Election Day, many Pennsylvania voters from both sides of the aisle watched in surprise, and in some cases, shock, as Donald Trump won the commonwealth and was elected the next president of the United States.
They weren't the only ones. Lots of political analysts in the state were also taken aback.
Most major polls never showed Trump winning Pennsylvania, let alone the election. The state was thought to be part of opponent Hillary Clinton's so-called "blue wall."
So what went wrong?
State pollsters are still trying to figure that out for themselves, and meanwhile, they have to answer to a disgruntled public.
Muhlenberg College Poll Director Chris Borick said he's been on the receiving end of some angry emails lately.
He said this happens after every presidential election to some degree, but this year it's been worse.
"People are, are upset, and are angry either that the polls gave them confidence that was unwarranted, or that they thought the polls were biased because of some kind of nefarious reason," he said.
And Terry Madonna, director of the Franklin and Marshall College poll, said his past week has been similar.
Madonna and Borick have been in the Pennsylvania polling business for years, and they provide political analysis and commentary on the side.
F&M and Muhlenberg are well-rated surveys, which means they tend to be accurate. But both pollsters say this year, they've never been further off. And many other polls--across the country--are in the same exact boat.
Borick said that's largely because on a fundamental level, this year was different.
Over and over, pollsters underestimated the enthusiasm in Trump's base. Borick said white, working-class voters in Pennsylvania turned out in almost double the numbers they did for Mitt Romney in 2012.
"I was truly skeptical about the idea of perhaps a silent Trump vote, that people were not telling pollsters, including us, that they were going to vote for Trump," he said.
Throughout the campaign, Trump himself repeatedly insisted his large, raucous rallies were a sign of a populist movement.
"I hear we set a new record for this building, and by the way I didn't have to bring J-Lo or Jay-Z, the only way (Clinton) gets anybody," Trump said at one Pennsylvania rally late in his campaign. "I'm here all by myself."
But Madonna said he mostly discounted that.
"Trump was saying look at the size of the crowds, look at the yard signs!" he said. "And I would kind of joke and say yeah, there's hundreds of yard signs, but yard signs don't vote."
Now, of course, he's rethinking that approach.
"After the election I also joke, maybe yard signs did vote," he said. "You know, we didn't pick up that."
Trump ultimately won the state by 1 percent of the vote, which is not far off from the state poll average, which had Clinton up about two points right before the election.
Still, it's a discrepancy.
Madonna said a big reason he and others were off was because of "non-response bias." Basically, those silent Trump voters Borick was so skeptical of? They were real.
In F&M's case, one problem was methodology. This year, Madonna was experimenting with new ways of reaching voters.
Traditionally, polls are conducted via landline. But now, Madonna said, more and more people have cell phones, or just don't answer polling calls, particularly young people. So he tried his luck at reaching them in other ways.
"Telephone interviewing in five to ten years will probably be extinct--at least that's what some pollsters think," he said. "So this year we included online polls, so about 40 percent of the people we interviewed conducted the poll online."
Madonna said he still thinks it was a valid idea. But it did throw off results.
"There is no doubt that that skewed the poll towards the Democrats, and when I released it I pointed that out to everybody so they would know that," he said.
"I'm not making an excuse," he added. "I'm just saying that we went into this fully aware that there had been problems developing over the years."
Borick said there was also another, perhaps trickier problem -- pollsters and analysts interpreted their own results incorrectly.
"That narrative that was building in the media and being woven in with the polling results might have blocked some legitimate questioning of some of the predictions," he said.
So where to pollsters go from here?
Madonna said he still trusts his model overall, but he's going to keep tweaking it.
"We're going to continue to experiment with finding a balance between surveys online and telephone interviewees--that's going to take some more complicated statistical adjustment," he said. "The other problem was the regional breakdown...so we have to do a better job when we adjust the polls by region."
And Borick is still poring over his results, looking for ways to fix the flaws that became apparent this year.
"You know, could it be an exceptional year?" he asked. "I mean that's the question. And I've drawn no firm conclusions. At least we'll have some time before we go back and do this again."
He and Madonna agreed that at the end of the day, polls can't predict the future. For better or worse, it all comes down to the data.