Can You Believe the Polls?

In a word, no, but they tell you something useful if you have help.

Over the past couple of weeks, and for the next couple, we’re going to be seeing a lot of polls. Back in August/September, they were reflecting some revival for the Democrats, even though the state of the country was and is, not to put too fine a point on it, awful. More recently, Republicans have pulled ahead. The question is: Are the polls reliable? The answer is no, but they’re useful nonetheless. You just need to have a handle on the extent of their bias.

The help needed is provided by a sharp, data-driven outfit I recently discovered. It’s run by Spencer Abraham, son of former Energy Secretary and US Senator Spence Abraham of Michigan. Its name is The Polling Monitor, and it’s a great resource. Here’s what it does:

The Polling Monitor’s system is straightforward. We use each polling entities’ historical performance to determine how to interpret new polls they release. Our…database contains the results of each pollsters’ surveys over the past three election cycles in [current] battleground states. Our analysis compares a pollster’s final pre-election polls in all Presidential, Senate and Gubernatorial contests in those elections to the ACTUAL election result to determine how accurate each pollster has been in each state they have surveyed. Our database provides readers with the ability to see what the results of a new poll would be if we “adjust” those results by the historic performance of the pollster.

Using the historic performance of pollsters to assess 2022 surveys indicates that various races are tighter than the public polls indicate. For example, recent surveys in Pennsylvania and Arizona showing solid Democrat leads are, when adjusted by the pollsters’ track records, much tighter. In Pennsylvania, USA Today/Suffolk’s latest poll shows Republican Mehmet Oz down by 6 points. But over the last three cycles in Pennsylvania, USA Today/Suffolk’s polls have overweighed Democrat candidates on average by 5.8 points. Adjusting for that historic bias suggests a very tight contest. Similarly in Arizona, CBS News/YouGov has Democrat Mark Kelly up by 3 points over Republican Blake Masters. But in Arizona, CBS News/YouGov has overweighed Democrats by an average of 4.2 points since 2016, also suggesting a much closer contest.

Yesterday’s Polling Monitor analysis in particular is going to provide some good cheer for Mitch McConnell. In Pennsylvania, a key race if Republicans are to gain a Senate majority, and where Democrat Fetterman has been leading from the outset, the most recent CNN poll has Fetterman up by five points, 51-46 over Oz.

But what is CNN’s historical performance in Pennsylvania 2016 – 2020?

Well goodness gracious, it has a Democratic bias of 8.8%. (I know that’s a shock).

So what’s the adjusted margin? Oz by 3.8%

Similarly in Wisconsin, where Republican Ron Johnson is trying to hold his seat against Democrat Mandela Barnes, the most recent CNN poll (10/13 - 10/17) shows Johnson with a slim one point lead. But what’s the CNN polling bias in Wisconsin over the last three cycles? Unsurprisingly, Polling Monitor tells us that it’s about what it is in Michigan — 7.3%. So what’s the adjusted margin? It would appear Johnson has a much healthier lead, 8.3%.

In Michigan, there is no Senate contest, but there’s an important gubernatorial race between Republican Tudor Dixon and Democrat Nurse Ratched Gretchen Whitmer. According to CNN, Whitmer is ahead 52% - 46%. But what’s the historical bias? TPM finds that it’s a whooping 9.8% in Michigan, meaning that Dixon is probably ahead by 3.8%.

Of course it’s not like pro-Democrat polling bias is new. Ann Coulter gives us a blast from the past, in particular Ronald Reagan’s one-sided victory over Jimmy Carter in 1980 (which also saw Republicans win a net gain of 13 Senate seats). Ann’s blog, Unsafe, examines “Why a Red Wave Is Suddenly Possible.” It reminds us of polling results of yore:

Thus, for example, after being hectored for most of 1980 that Ronald Reagan was headed for another Goldwater-style fiasco, here's the sort of thing a teenager would have read in The New York Times weeks before he won a landslide victory against President Jimmy Carter, taking 489 electoral college votes to Carter's 49.

     -- Sept. 15, 1980: "Reagan and Carter Even In Washington Post Poll"

     -- Sept. 21, 1980 "Allowing for the margin of error, the polls indicate a virtual dead heat between Mr.Carter and Mr. Reagan"

     -- Oct. 23, 1980: "Poll Shows President Has Pulled To Even Position With Reagan"

In mid-September, the Times' Anthony Lewis painted a vivid picture of Reagan's coming annihilation, citing a bunch of state polls:

     -- "A recent New York Times poll of registered likely voters [in New York] showed Carter leading Reagan, 44 to 38."

     ACTUAL RESULT: REAGAN: 47; CARTER: 44

     -- In Washington state, "a poll for the Carter campaign put the president ahead by 3 points against Reagan."

     ACTUAL RESULT: REAGAN: 50; CARTER: 37

     -- In Illinois, a "poll for Carter's campaign put him ahead by 5 points."

     ACTUAL RESULT: REAGAN: 50; CARTER 48.

My guess is that, if the pro-Democratic bias is shown once again two weeks from now, we’re going to hear the usual shopworn excuses: Polling respondents weren’t candid, there was a late break for the Republicans, cell phones make things tricky, our sample did not adequately reflect new voters, blah, blah, blah. The one thing we’re guaranteed not to hear is the thing closer to the truth: The polls, like so much of the rest of “journalism,” are simply stories put together by the Democrats. They’re planted to suppress Republican enthusiasm and goose Democratic fundraising. They’re wrong, not because they’re mistaken (always in the same direction, year after year), but because — guess what! — they’re dishonest.