Oct 06 2008
More Poll Posting – Obama May Be In Real Trouble If Polls Weight Samples Wrong
With Barack Obama pulling ahead in the polls, it seems a lot of us on the right are concerned about the validity of the message we are seeing. I am still of the opinion something is wrong with the polls and have shown why record crowd levels at Palin events, record viewership of the Biden-Palin debate (this is due to Biden???) and record RNC donation levels all paint a very different picture than a runaway race for Obama-Biden.Â
I did find one VERY interesting set of data on polls and the electorate which confirms my suspicion. But first I want to send a “shout out” to DJ Drummon and his correct statement about today’s less than professional polls:
And I have to warn you that the polls will not reflect an immediate valid effect – if they show a sudden jump for either side it is far more likely to mean that the polls have – once again – changed the party affiliation weights, a practice as dishonest as putting one foot on the floor while you weigh yourself. The reason is because while people do change their minds, historically it has always been a few people at a time and in small steps. One proof of that is that as popular as Presidents Reagan and Clinton were, neither saw a major shift in party identification during his Administration.
Emphasis mine. Note that the two post popular GOP and Democrat Presidents of modern times did not see large changes in party affiliation. And I also want to note Party Affiliation is a poor indicator of policy preferences. And now we get to some intriguing data from The Wall Street Journal. The story is all about the large pool of undecided voters, but it includes the following graph of party affiliation vs policy preferences (Liberal vs. Conservative)
Note the D-R-I split on “all voters” is 43-36-28. Most polls are weighting their sample by party affiliation, not policy preference. With the Democrat Presidential Primary drawing huge crowds and lots of GOP cross-overs to fight off Obama I would not want to put my reputation on party ID being the definitive weighting factor.Â
Look at the policy preference break down of Lib-Con-Mod, it is split  23-37-36 – almost the REVERSE of the party ID break down. That means if you re-run the polls using policy preference as the weighting factor you could see the polls almost flip 180°!  Democrats are up by 7%, but Cons are up by 14% – twice the advantage!
Even more interesting is the undecided poll. Â In the Party ID view undecideds are split equally between Dem-Rep-Ind at 28-26-43. This would lead one to allocate undecideds or leaners basically down the middle between Obama and McCain since the difference left and right is a statistically insignificant 2%.
But in the policy world view the picture is dramatically different. The Lib-Con-Mod mix is heavily tilted away from Obama’s base with 18-32-47. Here we see very little opportunity for Obama to take the undecided voters. Now the imbalance is 2:1 in favor of McCain, which holds a statistically huge 14% advantage.
I have said all year this was a vote against partisanship. This year the nation would vote on policy, not party affiliation. Hyper partisans from each party are to blame for the current mood of the country. If I am right and these polls have been weighted using the wrong driver (historically it has been party driven) then Obama is not in the lead and has no hope of catching up. Go back to what DJ Drummond pointed out about Clinton and Reagan, they won on policy positions not party ID. Normally I don’t think there is such a schism between the two, but this year there probably is.
I would like to see some pollsters like Rasmussen run a result that had policy position the driving weighting factor, not party ID. I would like to see those results as the other possible outcome, the other end of the ‘storm track’. Remember, these are statistical models just like Hurrican tracks. There is a cone of possible trajectories and a likely path. It is time pollsters starting showing the actual cone and how it shifts based on simple assumptions.
AJ, I hope and pray you are right.
I’m not sure what the market decline will do to the election…the Dow is down several hundred today, indicating that perhaps the “desperately needed” bailout was not so desperately needed.
And both Obama and McCain are throwing the Big Bombs at each other. This is going to get VERY ugly. The good news is that Obama has more skeletons in his closet, and it’s my understanding that the total funds available to McCain (his campaign and the RNC) are nearly twice Obama’s budget. The bad news is that Obama has the propaganda press providing free publicity for him.
In minor, unrelated news which has no connection at all to politics, world stock markets crashed and the second great depression began. Brazil and Russia have already halted trading as the contagion spreads.
Not nearly as important a story as the latest polls, though.
Hey AJ
You wrote: Note the D-R-I split on “all voters†is 43-36-28
That 28 is actually 20.
I share your poll dissonance AJ. I was wondering whether the lead up to the 1994 conservative takeover of the House and Senate might provide an historical barometer for analysis. There are of course substantial differences in the specifics of the two times but I believe the general level of frustration towards government as both being quite high.
I don’t recall poll results going into the 1994 elections but I do recall the MSM at the time in a state of shock following the outcome. This leads me to wonder whether the poll results at that time may have similarly provided false hope to Democrats back in 1994. If this is true, then it may provide insight into how and why the same could be true today.
AJ:
I read the story in the WSJ today. Interesting that the conservatives are 37% of the poll and Republicans are only 36% of the poll. It would seem that party affiliation is way off also because the “I” is only 20%. I think the “I” should be btw 25-30%. Rasmussen only has the “D” at 37%. The poll is suspect because it may take leaners into account and then took too many democratic leaners.
On the Liberal-Moderate-Conservative split, it is quite interesting that the 23-36-37 adds to 96. What? 4% are on the moon. Since moderates are the ones on the fence, lets add the 4% to the moderate column. That makes it 23-40-37.
Now the problem for a pollster is which way does the 40% lean. You have to choose the proper center/left to center/right split in your poll to be accurate. Now I know that Survey/USA does publish the policy preferences of its sample. However you need a five bucket survey to be accurate. Liberal, center/left middle center/right conservative. I would be happy with a four way survey since I believe the true middle is lightly populated (less than 5%). I would think that the split is not 20/20 but more like 25/15 left. BTW, if you plug in those numbers you get 52-48. That looks like the 2004 election.
What I have always wondered about how they change so quickly, 3 weeks ago Obama was behind. I think Obama is gaining, but I am not sure how much.
WWS:
I don’t think anyone is forgetting the markets, in fact I think Obama is cheering on the collapse. It was really kind of lucky for him.
AJ
“Note the D-R-I split on “all voters†is 43-36-28. ”
It is generally less confusing to your audience to match the graphic to the text in that the WSJ graphic shows the sequence of D-I-R and you reordered it to D-R-I.
thanks for the interesting analysis
[…] Strata has an interesting analysis of recent polls. Note the D-R-I split on “all voters†is 43-36-28. Most polls are weighting their sample by […]
[…] noted in a previous post that the polls seem to be weighted on Party ID and not Policy Preference, where conservatives and […]
[…] Over at strata-sphere.com is an interesting observation about polling. Using a graphic from a WSJ piece, the author says that […]
[…] convinced that they had gone under the hood and found that the numbers were all wrong (here is one such example). Indeed, the one recent poll that was decent news for Democrats was immediately assailed from the […]