On Tuesday afternoon, Kathleen Weldon, director of data operations and communications at the Roper Center for Public Opinion Research, hosted a panel on the relationship between journalism and public polling. The panelists were Ariel Edwards-Levy of CNN; Philip Bump, formerly of The Washington Post and now with Hearst; and Nathaniel Rakich, managing editor of VoteBeat and formerly with FiveThirtyEight. Topics included:
- Why polls matter to journalists
- What makes a poll credible
- Communicating uncertainty
- And how to build better relationships with journalists.
Why Polls Matter to Journalists
Philip framed polling simply: it is a way of asking the public about things at scale. You get a statistically representative sample and hear what people actually care about, untainted by the biases of a few individuals.
Nathaniel put it even more directly: his biggest pet peeve is journalists who prefer talking to a handful of “real people” over doing the same thing systematically with thousands. “The best way to do that is a poll.”
Ariel added that polling is “a scientific method which is not perfect, but better than asking your Uber driver.” It trumps anecdotal data.
What Makes a Poll Credible
When asked what they look for first when evaluating a new set of numbers, the panelists were consistent. They want transparency.
Ariel wants a full methodology statement that answers: Who did you ask? How representative are they? How did you find them? What exactly did you ask? How were the data weighted? “Any pollster worth their salt is delighted to share their methodology and won’t shut up about it!”
Communicating Uncertainty
One of the most useful discussions for anyone who produces or pitches poll results was how to communicate uncertainty without losing the audience.
Philip dislikes polls that report decimal places (“47.5% vs. 47.2%”), implying a precision not warranted given the margin of error.
Ariel called polling “a snapshot, not a prediction… and the snapshot is blurry.” But multiple snapshots from different angles, taken together, produce something useful, she said. The challenge is helping general audiences understand this without losing them in statistical jargon. Margin of error is itself a jargony term, the panelists agreed; the concepts behind it are accessible to anyone, but the vocabulary can be a barrier.
Nathaniel recalled a useful approach from Monmouth University, which released multiple models of the electorate, with different weights: one for a high-turnout environment, one for a low-turnout environment. It frustrated audiences who wanted one definitive figure, but it forced a more honest reckoning with uncertainty.
How to Build Better Relationships with Journalists
The panelists offered practical advice for researchers and communications professionals who want journalists to take their surveys more seriously.
Nathaniel recommended a grassroots approach: reach out for coffee or lunch and attend journalism conferences.
Philip’s advice was to learn what a journalist actually writes about before you pitch them. “Nothing will ingratiate you more than being able to say ‘here are crosstabs on X,’” when “X” is a topic they cover regularly.
Ariel’s advice was not to overestimate what journalists already know about your subject. “You have written these results a hundred times, but others will find them interesting. Present it in a conversational way. Things that strike you will strike other people.” When you share data, make the full methodology and topline results easy to find. Don’t make journalists chase you for the basics.
Author Notes:
Jeffrey Henning
