Thursday, May 06, 2021

On Politics: Polling’s prognosis: wary conservatives and eager liberals

A new report points to another problem with polls: liberals who are almost too willing to chat.
Ruru Kuo

Political pollsters know they're having trouble reaching entire swaths of the Trump-supporting electorate, and they're working on ways to address it.

But some of them fear that might not even be the extent of the problem, and a report released today by Data for Progress, a left-leaning think tank and polling firm, bolsters that view.

Data for Progress researchers have gotten used to being told to take a hike — and worse — by Trump supporters they try to poll, and the report detailed their difficulties in reaching that group.

But perhaps most significantly, it also found that they were having trouble reaching a representative sample of Democrats, as well as Republicans — partly because liberals with strong political views were almost too willing to be polled.

Data for Progress's results have been on the more accurate side among its peers, though it relies entirely on so-called nonprobability methods — namely reaching people by email and text message — that haven't gained full acceptance as an industry standard for political polls, at least not yet. (That's partly because these methods involve using an already-limited universe — say, people who have cellphone numbers attached to their names on publicly accessible voter files, which accounts for only about three in every five voters — to draw one's sample.)

The report by Data for Progress amounts to an earnest recognition that it has encountered issues drawing a representative sample across the board. "We want this to be out in the open so that we can all help to identify the root causes," Johannes Fischer, a lead methodologist at Data for Progress, said in an interview via Zoom.

ADVERTISEMENT

With Democratic respondents, unlike Trump supporters, the report found that the most common problem was over-participation: "Liberal activist"-type voters, the report said, were so eager to express their views that they could quickly fill up much of the poll's Democratic sample, unless proper adjustments were made.

So-called response bias is a big part of the reason innovative methods like text-to-web surveys have yet to win the full faith and confidence of the polling establishment. Still, it's safe to say the report's findings speak to the broader issue of who is and who isn't willing to respond to surveys, across modes. And as more and more polling is done via the web and text message, these techniques demand the same scrutiny as traditional phone surveys.

'Activist' overrepresentation

By using voter files attached to people's email addresses, researchers can piece together profiles of respondents based on publicly available data, including things like whether they've ever volunteered for a political campaign.

Using an algorithm that takes into account a user's entire profile, methodologists will create a model that uses the set of people who have volunteered for campaigns to identify other people with similar interests and lifestyles — the kind of "activist" voters mentioned in the report.

ADVERTISEMENT

Going forward, Fischer said, the team is working on ways to prune its sample, investigating ways "that we can down-weight people that we're relatively confident are activists," so that other respondents identified as Democrats can have their perspectives represented in polls.

Education isn't everything

After the 2016 election, when pollsters underestimated Donald Trump's support in many key states, post-mortem reports indicated that firms had often failed to account for the differences between white voters with and without college degrees.

"Weighting" data by education — that is, making sure that a survey had the right level of representation across education levels, particularly within racial groups — was cemented as standard operating procedure.

But the Data for Progress team has found that this may not be enough. Its report showed significant, consistent differences within education groups that had an effect on whom they reached. Not all white conservative voters with college degrees will vote similarly — and not all of them will be equally likely to respond to a survey.

ADVERTISEMENT

A particularly big factor was where people lived: The team found that a respondent's social environment — who their neighbors are — may matter more than their educational level.

Ultimately, said Colin McAuliffe, a co-founder of Data for Progress, the team found that among white voters, when it comes to candidate choice, "the percentage of voters in their ZIP code with a college degree was more predictive than their own education level."

In particular, Fischer said: "There are substantial differences among white conservatives that we aren't picking up on. A college-educated white Republican in a rural, consistently Trump-supporting ZIP code is different from one in an urban or Biden-supporting ZIP code."

"We're hearing from more of the Biden-supporting ZIP codes, and less of the non-Biden-supporting ones," he added. "And because of that, we can't just weight by education and expect that to be solved."

Finding fixes

Over the course of the 2020 campaign, Data for Progress researchers said they got profane responses from thousands of people they texted — often topped off with a "MAGA" or "Trump 2020" reference. This was all of the evidence they needed to support social scientists' frequent observation that support for Trump correlates to suspicion of institutions, which can have a big impact on surveys.

So as they conduct polls on political races and issues this year, Data for Progress researchers have begun sending out different types of invitations via text — all leading to the same survey. "We're experimenting with the language that we use when we text somebody," Fischer said. "Can we use language that elicits more authoritarian responses, or keys into language used more often on the right, to engage those voters?"

And by specifically factoring in the social and political climate of their respondents, as well as the respondents' own personal and political profiles, Fischer said that Data for Progress had cut its measured error in half.

When building a representative sample now, "we care about the neighbors," Fischer said, "more or at least as much as we care about the respondent themselves."

Could 'Young Rock' be Dwayne Johnson's 'Apprentice'?

In The Times Magazine today, Jeremy Gordon explored "Young Rock," an NBC comedy based on the life of Dwayne Johnson, the former pro wrestler and star of the "Fast and Furious" movie franchise.

Jeremy writes:

With "Young Rock," Johnson may very well be trying to find out if this alchemy can be performed for real: if a fiction can be created in front of an audience and then imposed on reality. The framing device for the show, the reason we're learning about Young Rock's life, is that Johnson is on the campaign trail for the 2032 presidential race, where he has a real shot to win. Like all coming-of-age stories — and most instantly remaindered political memoirs — "Young Rock" purports to trace how Johnson's upbringing turned him into the man he is today: wrestling champion, the highest-paid actor on the planet, maybe a future president. Roll your eyes, but accept the possibility. Ever since Donald Trump was elected, plenty of charismatic celebrities have been floated as potential candidates. More than the other contenders — Oprah, Mark Cuban — Johnson has gained real traction, even going so far as to publicly state that he wouldn't run in 2020 but that it was something he "seriously considered."

He goes on:

Johnson passes every cosmetic test: handsome, tall, voice like a strong handshake. He's the star of several film franchises that future voters will have grown up watching. And while a different show might play all of this for laughs, "Young Rock" frequently lapses into what messaging for Johnson's actual campaign might sound like. It's never specified whether he's running as a Democrat or a Republican; he presents as a third-way politician who just wants America to push past its divisions. Candidate Rock is a little like Michael Bloomberg, but with more convincing platitudes and even better delts.

Finally, he writes:

It's not Trump's "The Apprentice," which was a genuine hit for a decade. But Johnson has many other concurrent efforts to expand his fame across American life: A new "Fast & Furious" movie comes out in June; his relaunch of the much-maligned X.F.L., which he purchased last year, is still in the works; there are rumors that he'll return to the W.W.E. for a final match. Nobody has ever taken this path to the Oval Office, but you could have said that about Trump, who also understood the importance of committing to character. When your supporters want to believe what you're saying, there's no limit to how far the gimmick can go.

NEW YORK TIMES PODCASTS

Sway: A top G.O.P. pollster on Trump and Facebook

On today's episode, Kara Swisher was joined by Frank Luntz, the veteran Republican pollster and strategist who shot to fame with his Fox News focus groups.

In the wake of Facebook's continued ban of Donald Trump, Luntz says the former president has no one but himself to blame for losing his social media status and the White House as well as the Republican majority in Congress.

He's got some predictions for Mitt Romney and Liz Cheney, some frustrations about the term "vaccine passports" — and some thoughts for Tucker Carlson.

SUBSCRIBE TODAY

If you've found this newsletter helpful, please consider subscribing to The New York Times — with this special offer. Your support makes our work possible.

Were you forwarded this newsletter? Sign up here to get it delivered to your inbox.
Is there anything you think we're missing? Anything you want to see more of? We'd love to hear from you. Email us at onpolitics@nytimes.com.

Need help? Review our newsletter help page or contact us for assistance.

You received this email because you signed up for On Politics With Lisa Lerer from The New York Times.

To stop receiving these emails, unsubscribe or manage your email preferences.

Subscribe to The Times

Connect with us on:

facebooktwitterinstagram

Change Your EmailPrivacy PolicyContact UsCalifornia Notices

LiveIntent LogoAdChoices Logo

The New York Times Company. 620 Eighth Avenue New York, NY 10018

0 Comments:

Post a Comment

<< Home