The below article first appeared in David Corn’s newsletter, Our Land. The newsletter comes out twice a week (most of the time) and provides behind-the-scenes stories and articles about politics, media, and culture. Subscribing costs just $5 a month—but you can sign up for a free 30-day trial.
Walking my dog. On the Metro. In line at a sandwich shop. People keep coming up and asking me about “the polls.” What do the numbers mean? Should they be worried about the election? If a set of swing state polls is released, the odds are by the end of the day I will have been asked by a friend, a relative, a neighbor, or a stranger, or several, “Did you see that poll in Nevada? Why was there a shift of three points since the last one? How could Pennsylvania be going in a different direction? And North Carolina, really? Do you think that’s accurate?” If they start referencing Nate Silver, Nate Cohn, or any of the other pollster celebs…I want to scream.
Polls, to be hyperbolic about it, have ruined American politics. Okay, a lot has ruined American politics. But polls have certainly made American politics less enjoyable. Many of those who follow politics—and not enough citizens do—have become slaves of polling, overly obsessed with these surveys and palpitating over the slightest changes. I’m not unsympathetic. This election is prompting more anxiety than most. The oft-repeated mantra that the 2024 race could determine whether the United States remains an imperfect democracy or slips toward a more authoritarian form of governance is true. Thus, every iota of data related to the face-off between Donald Trump and Kamala Harris appears loaded with relevance and consequence. Still, the hyperfixation on polls is unwarranted and distracts us from other important aspects of this most important election.
Polls don’t matter. Or maybe they do. It depends on your definition of “matters.” By all measurements, this is a close race. What else do you need to know? The candidates are within a few points of each other in the national polls and the swing state polls. But the difference is usually within the reported margin of error. That means the poll that has just caused you heartburn may not have any value in terms of telling us what will happen on Election Day.
And get this: That margin of error may not even be accurate.
What was most interesting in this article, though, was what it said about the margin of error: “The real margin of error is often about double the one reported.”
While doing a little (but not much) research for this rant, I came across a useful article from the Pew Research Center, which does a lot of polling. It was published this summer and called “Key things to know about U.S. election polling in 2024.” The piece made the usual points. In 2016 and 2020, polling underestimated Trump’s performance. (Polls on average overestimated Hillary Clinton’s strength by 1.3 percent and Joe Biden’s by 3.9 percent.) The 2022 nonpartisan polls—meaning those taken by the media and research centers and not by campaigns and political groups—were more accurate than people may have assumed after the mythical “red wave” did not materialize. Polling methodologies have shifted to keep current with changes (such as the decrease in the use of landlines and a low response rate). Pollsters have improved how they weigh demographic variables to obtain representative samplings.
What was most interesting in this article, though, was what it said about the margin of error: “The real margin of error is often about double the one reported.” What? Read that again. Double the margin of error. “A typical election poll sample of about 1,000 people,” Pew tells us, “has a margin of sampling error that’s about plus or minus 3 percentage points.” That’s usually the number you see associated with a poll. Three percent. That doesn’t seem so bad.
But there are other errors. If you must know, they are called noncoverage error, nonresponse error, and measurement error. I’m not going to go into the technical details here. But this is the bottom line from Pew: “The problem is that sampling error is not the only kind of error that affects a poll. Those other kinds of error, in fact, can be as large or larger than sampling error. Consequently, the reported margin of error can lead people to think that polls are more accurate than they really are…Several recent studies show that the average total error in a poll estimate may be closer to twice as large as that implied by a typical margin of sampling error. This hidden error underscores the fact that polls may not be precise enough to call the winner in a close election.”
So are you really going to pull your hair out over a poll with a margin of error of 6 points? C’mon. Get a grip.
“Turning the press into pollsters has made American political culture Trumpian: frantic, volatile, shortsighted, sales-driven, and anti-democratic,” wrote historian Jill Lepore.
It’s easy to be a polling Grinch. If you want to dive into such territory, I commend two well-researched articles. In 2015, historian Jill Lepore wrote a lengthy and fascinating piece in the New Yorker on the history of polling that took a dim view of this practice and decried its impact on US politics. She explored the decades-old debate among social scientists as to whether there really is such a thing as “public opinion,” questioning whether polling measures it or creates it. George Gallup, who helped invent the polling industry, believed it did exist and could be quantified for edification and profit. But Lepore offered the case that whoever was correct about this, polling and the media addiction to it is not beneficial for democracy. After citing the Gallup Poll’s former managing editor David Moore’s remark that “media polls give us distorted readings of the electoral climate, manufacture a false public consensus on policy issues, and in the process undermine American democracy,” Lepore added her own observation: “Polls don’t take the pulse of democracy; they raise it.”
Referencing Trump’s 2016 campaign, she concluded, “Donald Trump is a creature of the polls. He is his numbers. But he is only a sign of the times. Turning the press into pollsters has made American political culture Trumpian: frantic, volatile, shortsighted, sales-driven, and anti-democratic.”
Lepore might have been unduly pessimistic about pollsters meeting the technical challenges of the day, but on the Big Idea she was prescient. Picking up where she left off is Samuel Earle, a PhD candidate at Columbia Journalism School, who published a long essay on polling in the recent issue of the New York Review of Books. (His piece is ostensibly a review of Strength in Numbers: How Polls Work and Why We Need Them by G. Elliot Morris, the editorial director of data analytics at ABC News and FiveThirtyEight, its polling review outlet.) Earle, too, wonders about the nature of public opinion and the ability to capture it. He presents a harsh history of the polling biz, noting that Gallup once said of polling, “If it works for toothpaste, why not for politics?” And he applies the Heisenberg observer effect to polling:
[E]very attempt to study how people think and act has the potential to influence how they think and act, thus changing what is being recorded, either in self-fulfilling or self-negating ways. The results of any poll on a particular issue are liable to change how people think about that issue, just as any poll showing a candidate’s popularity is liable to influence that candidate’s popularity.
Polls are shortcuts to understanding a rather complicated matter: how millions of Americans, each operating on different levels of engagement with different levels of information, will make a specific decision. In a way, polls may be comforting, providing the fantasy of certainty (or possible certainty) in a sea of unknowing. But they can enhance anxiety and smother more substantive discussions of an election. They definitely are useful for campaigns, as the political professionals strive to find the best messages and plot out how most effectively to use their resources. Which states should we spend money on? Where should we send the candidate? What themes and ideas seem to be resonating? Let’s look at the numbers.
The fascination with polls also reflects the data-fication of society and popular culture.
Earle acknowledges the benefits of polling for the pros. But he’s right when he observes, “[P]olls saturate election coverage, turn politics into a spectator sport, and provide an illusion of control over complex, unpredictable, and fundamentally fickle social forces.”
The fascination with polls also reflects the data-fication of society and popular culture. Here’s one crude analogy. For many years, only Hollywood insiders pored over the opening-weekend box office returns for movies. But at some point—I can’t recall when—seemingly everyone began talking about that first weekend take. The question was no longer, Is this movie good? It became, How did it do?
I’m sure we can chart how polls came to dominate political coverage. In the mid-1970s, according to Lepore, media outlets, which had previously relied on Gallup’s firm and other polling outfits, began conducting their own polls. “[W]e’ve been off to the races ever since,” she wrote. And now coverage of polls crowds out other elements of the race. When someone (like me) complains about horse-race political journalism, this is often what they have in mind.
Here’s a recent example. When Harris earlier this month proposed expanding Medicare to include home health care, the New York Times placed its story on this plan on page A12. On the front page, the top story was a report on the new swing state polls the newspaper had conducted with Siena College. The Times was promoting its own polling and—with other outlets picking up these findings—creating a news cycle. Yet Harris’ proposal could affect millions of Americans. It was arguably more consequential than the polls of the moment. Adhering to its basic precepts of politics coverage, the editors of the Times deemed those surveys more important.
There’s plenty more to say about polls. Political pros and amateurs love debating which ones are more accurate and how they are used or abused. (Some libs have recently been complaining that Republicans are producing junk polling that shows Trump in a better position in order to rig the national averages of polls in his favor.) Politicos assess how to recalibrate this year’s surveys according to various factors. (What if the current polls are wrong in the way the 2020 polls were wrong? What if they are wrong in the way some of the 2022 polls were wrong?) Polling is a cottage industry. Dissecting polls is one as well. Or perhaps a hobby. Like fantasy football. (At least in fantasy football you pick and manage your team and possess some agency.)
You will note that I’ve managed to get through this diatribe without declaring that a poll is just a snapshot in time and that the only poll that counts is on Election Day. More to the point, polls are the sugar high and empty calories of politics. And they make for lazy—or, at least, easy—journalism. I’d rather see reporters dig into other stuff. The ties between right-wing extremism and the GOP, the dirty deeds being perpetuated by billionaire-funded super-PACs, the role of dark money and disinformation in this campaign, the how-this-affects-you implications of the candidates’ positions. I bet that if you asked voters and news consumers, a majority would agree. Let’s poll that.
