Nov 02 2008

WaPo Comes Clean On Pollster Turnout Models

Published by at 10:08 am under 2008 Elections,All General Discussions

Another leading polling expert, this time for the Washington Post, comes clean about the uncertainties in the voter turnout models which have produced Obama leads of +2 (a tie) to +15 in the same week from different pollsters. A must read for Americans so they understand why polls usually get the race wrong (only 1 or 2 an election season get close):

Simply put, we may be wrong about who is likely to vote on Tuesday. One of the trickiest parts of political polling is determining which of the people interviewed in pre-election surveys will really vote. It’s relatively easy for us to identify such sharply delineated groups as the population of all adults living in the United States or even all registered voters, but the pool of actual voters is a group that exists at a single point in time, on Election Day (plus those casting ballots early and by mail).

Even a few days away from an election, that group remains an unknown population. Not everyone who says that they will vote will actually do so, in part because when asked about their intentions, people want to sound like good citizens. So pollsters develop models to whittle down their samples to account for people’s tendency to overstate these things.

In Washington Post-ABC polling, we ask a series of questions about whether and how people plan to vote, whether they have voted before and basic knowledge about the voting process. We then feed all this information into a range of models, corresponding to different levels of turnout. We report a single model, but only after assessing the quality and impact of all of them. Likely voter modeling is a craft, bolstered by science.

Emphasis mine. As we see with hurricane storm tracks there is actually a cone of possible trajectories, the pollsters only let people see the mode the poll owner (or pollster) wants them to see. And trust me, all those models have 95% confidence levels. Can they all be right?  Nope.

So is this a statistical tie or an Obama blow out? recent state poll data and independent measures (like the volume of early polling) tell me it could very much be a tie right now. Bottom line – I don’t think anyone really knows for sure.

Update: And what if McCain-Palin supporters are so fed up with the entire Political Industrial Complex (PIC) that they are abstaining from the polls? This bothers the WaPo:

I worry more about a basic concern: whether we are getting a truly random sample of opinion. Pollsters bank on the fundamental notion that the people who answer our calls are similar to those who don’t, and we have reams of data justifying those assumptions. But what if the people who pause to take a pollster’s question are significantly different from those who don’t?

After all, fewer and fewer people have been taking our calls over the years. The Pew Research Center, which has done extensive research into declining survey-response rates, has found that poorer, less educated whites — who tend to hold somewhat less positive views toward African Americans — are also harder to get on the phone than those who have higher incomes and more formal education. My fellow pollsters and I give this a pretty academic name, “differential nonresponse,” but it’s a live, practical concern.

If there is a problem with the polls it probably lies in who is responding and who is not. Polls are the rare statistical measurement which requires the sample to engage in the measurement. If the sample refuses to a large degree, the poll becomes less and less accurate.

3 responses so far

3 Responses to “WaPo Comes Clean On Pollster Turnout Models”

  1. […] The Strata-Sphere » WaPo Comes Clean On Pollster Turnout Models […]

  2. robert c verdi says:

    I do wonder if Pollsters often look at what other might be doing simply to be safe. Essential act like the group, so if it all goes wrong you have cover, we shall see.

  3. […] Diabolically disorienting stuff. UPDATE: See AJ Strata’s post on why the polls make no sense and should not be trusted. Obama’s Plan to Bankrupt the Coal Industry « The Political Page pinged back with […]