« BACK  |  PRINT

RS

FRONT PAGE CONTRIBUTOR

PPP’s polls were rigged all along

New York Magazine was trying to be sympathetic to the popular polling figures on its own side of the political, but let out a secret in the process: Public Policy Polling cooked the books all along.

The key paragraph:

When I talked to Tom Jensen, PPP’s director, this morning, he was understandably in the mood to gloat. “These supposed polling experts on the conservative side are morons,” Jensen crowed. “Jay Cost” — the Weekly Standard’s polling expert who’d waged a number-crunching war against PPP — “is an idiot.” But Jensen conceded that the secret to PPP’s success was what boiled down to a well informed but still not entirely empirical hunch. “We just projected that African-American, Hispanic, and young voter turnout would be as high in 2012 as it was in 2008, and we weighted our polls accordingly,” he explained. “When you look at polls that succeeded and those that failed that was the difference.” Given the methodological challenges currently confronting pollsters, those hunches are only going to prove more important. “The art part of polling, as opposed to the science part,” Jensen said, “is becoming a bigger and bigger part of the equation in having accurate polls.”

Note the doublethink here: Jensen and New York Magazine are both so very pleased with the election’s results – as well as eager to take digs at Scott Rasmussen and Jay Cost – that they’re both perfectly willing to overlook the fact that Cost was right. In fact, the truth was much further along than Cost, I, or most any other commentator was ever willing to go.

I don’t remember anyone willing to say PPP was actively rigging the polls to reach chosen results, but there it is in black and white. Jensen decided in advance what he wanted the electorate to look like, and so tweaked the numbers until he got what he wanted. This isn’t a whole lot different from what Research 2000 admitted to doing, folks.

In science, it’s not just that you got the answer you wanted. It’s the process that matters. PPP, Nate Silver, and the New York Times – the heroes in New York Magazine’s story – are not practicing science. They’re taking their own beliefs and wrapping them up in a cargo cult.

Maybe that’s why Silver didn’t even try to project the House of Representatives this time. I didn’t have time to do any projections this cycle because I had to take a full time job. Nate Silver is a paid professional with the New York Times. What excuse is there for him?

Crossposted from Unlikely Voter

COMMENTS

  • supa

    Nate Silver isn’t a pollster… Nor is he affiliated with PPP. Not sure why you bring him up.

  • marya136

    Nate Silver’s probably going be named Man of the Year. Bow down, oh, lowly one. You are in the presence of The Master.

  • sbradsha

    And the point is that the pollsters apply their secret sauce to account for sampling bias ?

    Surely if the poll is shown to be accurate in retrospect that is what matters ?

  • MiamiDave

    So, PPP took a lucky guess as to what the electorate would look like (based, apparently, on what they wanted it to look like), and now they expect us to kowtow to them because they guessed right? PPP is the hack magician who wants their audience to “pick a card, any card” and is astounded– and happily surprised– when their audience picks the right one. Their credibility is about nil.

  • earlgrey

    I am confused (perhaps it is still the shock of this election hitting me square in the face), are you suggesting that PPP simply shaped the race via a Field of Dreams type approach. If you build it they will come? If you wish it, the votes will follow.

  • http://www.hakubi.us/ Neil Stevens

    They created propaganda for their party.

  • Kyle-MI

    But Silver’s calculations are all based on the rigged polls. He guessed right because they guessed right. There was no objective information that turnout would be what it was. They all got lucky. That is definitely not science.

  • http://www.hakubi.us/ Neil Stevens

    He was also in the story I linked, which is why I brought him up.

  • Kyle-MI

    It is impossible to know how much it shaped the race. Did it make a difference in the outcome? It was close in the three swing states we lost, FL, VA, and OH. Who knows?

  • supa

    I guess so. We had the same thing from Rove etc. To be honest I find it a bit infuriating in retrospect. But I suppose it did keep morale up for the most part.

    I don’t buy that the public in general was inundated with positive polling news from the MSM. Nate Silver yes. But whenever I watched CNN (for as long as I could stand Wolf Blitzer) they were always hyping how close the race was, not how BHO was in the lead, that’s precisely why Nate became so popular to the left. He was saying, “don’t worry guys, Obama is very likely to win”. I didn’t believe it. I still don’t want to. But he was right. One poll may be rigged against Romney, but all of them taken together were not.

  • kmtierney

    They were EV…. bingo. No real sense trying to tell a different story. Many thought things were one way. They just weren’t.

  • lar0311

    Nate Silver based his calculations on a number of national and state polls and applied it to his system. He was 49/50 in 2008 and 50/50 in 2012. It’s probability and therefore science. It’s not an exact science but he has outperformed pretty much everyone else.

  • jimmyg

    Isn’t that the gist of the controversy this year with all the pollsters, each pollster had to estimate, or guess, what the turnout would be for each individual party. In this case PPP guessed correctly, in 2008 Rasmuessen guessed correctly. That is the secret sauce. Otherwise a pollster would randomly grab a 1000 people off the street, ask them who they are going to vote for, and from only that information make a prediction of who will win and who would lose.

  • Jim_Riggs

    Why wouldn’t they assume turnout would be about the same as the last time?

  • dpmaine

    > What excuse is there for him?

    Other polling analysts did try this cycle, and had mixed results.

    The main reason is, that, uh, constructive redistricting makes it hard to correlate voter patterns to a win. Florida’s topline totals are very closed to tied – 50/50 between Pres. Obama and Gov. Romney.

    Yet, the Congressional delegation is something like 19-7 in favor of the GOP. Based on the concept of the House of Representatives, being a fairly democratic body, you would expect it to be divided roughly equally.

    The same is in true in Democratic states – California, New York, Illinois, Massachusetts.

    The name of the game is gerrymandering – you cut up your opponents districts in a way to concentrate their voting bloc into the smallest number of districts possible, and then you dilute your voting blocks just enough to safely create districts that are 1.5-to-1 in your favor. That way you don’t have to really run much of a re-election campaign, and your opponent’s can’t really do anything about it. 10 years of easy re-elections.

    Trying to capture this in a polling model is very hard, or impossible. Also, polling individual Congressional seats is pretty difficult.

  • iowaguy

    Students rig their data to come up with the correct result, when they KNOW the result ahead of time. Nobody knew this result. These arguments are killing us folks. Rove needs to retire. What a disaster.

  • http://travismonitor.blogspot.com Freedoms Truth

    Huh? Is that news? PPP was ‘rigging’ the polls to the 2008 model? I thought that was known, every time we cracked open polls to see D +8. Or is it that they were doing weightings in there.

    We conservatives argued against this being a 2008 repeat, a lot of evidence mitigated against it … and … oops, they were right. First, you cant claim PPP was ‘rigging’ to be dishonest since, well, it gave a good result. And this was a Demographic and not political aligned weighting (although the 3 groups mentioned it boils down to much the same thing).

    BUT YOU MISS THE REAL POINT. The REAL STORY. How did Romney get fewer votes than McCain?!? How did that happen? If Romney had achieved the McCain level, it would be a telling the whole 2008 turnout model might be thrown out.

    So perhaps the question is less “How did youth/black/hispanic vote stay up?” It is “How did all the other turnouts go down like it did?”

  • iowaguy

    Thank you.

  • http://www.hakubi.us/ Neil Stevens

    The difference between weighting and rigging is the process. Read the posts. Study science instead of lapdogging the lefty activists here.

  • Kyle-MI

    Why should you trust the averages? You ever hear of garbage in, garbage out? Bad polls will add to systematic error in the average and affect accuracy.

  • Kyle-MI

    You cannot determine accuracy by one data point.

  • http://travismonitor.blogspot.com Freedoms Truth

    “Not just one. Taken together they seem to be pretty accurate.”

    Bingo. Respect RCP. The poll averages, even if they have some absurd pollaganda polls, give you a sense of things. Obama led in sept in RCP. Romney led in RCP in oct. Then late oct it was even and then Obama moved up. RCP picked up on Romney’s lead after debate 1, and also had Obama’s late post-Sandy surge that put him back in front.

    I do think Nate Silver was being bogus when he kept Obama with better chances even while Romney started leading in RCP poll averages, but his excuse was the state polls (even though he was using older polls).

  • iowaguy

    HA… you can’t retroactively call a poll rigged… Now it’s ppp (whom I thought would be colossally wrong). Whoever was right, you’d call it rigged? This isn’t “science” this is math. PPP guessed, and I do say GUESSED, right. To call it rigged is very bizarre. Every campaign cherry picks the polls to sway public opinion. I really think this is barking up the wrong tree.

  • Kyle-MI

    No, you don’t need to know the results in advance. You can also rig a poll to give the answers that you want. Weighting would be based on some objective independent piece of evidence. PPP did not base their weighting on any reasonable objective criteria. They used the 2008 turnout because it gave them what they wanted.

  • http://travismonitor.blogspot.com Freedoms Truth

    You get a lot MORE info from EV, as you can tell who voted.
    It changes the whole GOTV game. Its silly to say EV spells the end of getting election day info, it tells you a wealth of information!

    Had you had a local person look into it, they could have told you if EV trends were better than 2008 or not or more like 2010. One thing that frustrated me was the GOP side talking about “Hey, we are doing better than 2008 on this or that.” when comparing against a 7 point win was the wrong metric! Yes, we cut from 7 pts to 2pts, but it didnt cut it did it.

    The more I think about it, the more I conclude that this is more an issue with GOTV then about polls. Any given poll has a 3% MOE. Any given GOTV effort can get you 3% delta in turnout. WE GOT THE 2008 TURNOUT MODEL BECAUSE THEY RAN AS GOOD A TURNOUT / GOTV OPERATION AS IN 2008 AND WE RAN AS BAD A TURNOUT / GOTV OPERATION IN 2008!

  • MoeLane

    Yes, and in 20*10* he was notably incorrect about the House and got at least one Senate race (NV) wrong. For that matter: in 2012 he was off on the final Senate totals by two and a half, and didn’t do a House prediction at all.

    This may prove relevant in the 2014 election cycle.

  • MoeLane

    Gerrymandering is a nice excuse that both parties like to draw on, but when you have a turnout disparity at the level that we’re talking about it’s insufficient. Take a look at the swing districts; take a look at places like Wisconsin, on the state level-races; and take a look at the swing state results.

  • Kyle-MI

    You keep using that word (science) but I don’t think it means what you think it means. Rasmussen did not rig their data. They weighted the data using an objective criteria. In light of the comparison with the known results, they need to reevaluate that criteria, but it wasn’t rigged. The PPP demographic turnout was neither objective nor reasonable.

  • iowaguy

    Kyle, that’s weighting.(2008) Unless you’re claiming they made up their results. If so, they were very accurate in fake data. The death of a pollster would be to be wrong. Why would they try to fake it?

  • Kyle-MI

    But they lost 10 million votes and we lost 2 million. Both parties had worse GOTV then in 2008. The Dems were even worse then the GOP, but it wasn’t bad enough to loose the election. We could do better with GOTV, but, looking at those number, I am not sure we want to follow the Dem model.

  • Kyle-MI

    No its not. It is not reasonable and objective. You can do a back test on known data to show that it is not. For example, take the turnout from 2004 and use it to adjust the polls in 2008. It will not give you the correct answer. You can do the same with 2000 and 2004 and get the same incorrect answer. The only time it works is between 2008 and 2012. That is not accurate.

  • Kyle-MI

    But that algorithm doesn’t work for any other recent election cycle. It won’t work between 2004 and 2008. It is not reasonable and not accurate.

  • iowaguy

    Seriously? Let’s blame the polls? I’ve been trying to argue my point below, but this is laughable. Come on Neil, it’s not the MSM polls you should be upset about — or any polls. Romney failed to convince the American public about his vision for the future. Period.

  • MoeLane

    No, Neil is saying that Jensen admitted that he eyeballed the proportions until it came out the way that he wanted them. Which is neither ‘science’ nor ‘math:’ it’s ‘guessing.’

    This is a fairly obvious point, and you’re becoming boring in your disinclination to see it. Stop doing that.

  • TravisMonitor

    I have my own diary on this now.

    http://www.redstate.com/wosg/2012/11/09/it-was-the-gotv-all-along/

  • TravisMonitor

    ” I didn’t believe it. I still don’t want to. But he was right.”
    No, maybe he was right to say it 2 days pre-election, but he was NOT right to be saying that the whole time. During most of October, Romney was leading RCP and state polls in many swing states. Romney would have won had the election been oct 15. Only after sandy was there about a 2 pt+ bump that carried into election day. So nate silver called it right, but he would have been wrong had not support moved Obama’s way in the last week.

  • deltawing

    This article is just flat out wrong. PPP didn’t “rig” their polls. Like every other poll, they weighted the results to reflect a certain demographic breakdown. There is nothing dishonest about that.

  • SirGladiator

    We can go back and forth all day about ‘they got lucky’ vs ‘they were smarter than us’ but the bottom line is they got it right, and we tricked ourselves into getting it wrong because we wanted to believe we were winning. Maybe it was all a lucky guess on their part, but you know what? We’d have done ourselves quite the service if we’d believed it, even if it had been wrong. Romney would’ve had a much stronger close to the campaign if he believed he was behind (as it turned out he was) than if he believed (as he did) that he was about to win and could just sit on the lead. Libya, Obamacare, Life, Marriage, winning issue after winning issue left completely off the table, all because ‘we’re winning, lets play safe and run out the clock’. Next time when the liberals tell us they’re winning, let’s just believe them and go out there and get some more votes.

  • themoderateman

    PPP projected turnout based on the information gathered during the calling. I really don’t understand this diary. PPP was accurate. Can’t fix a problem without defining it first.

  • themoderateman

    I don’t think you understand polling. The turnout models are based on what the respondents are saying when you call them. When PPP and other pollsters called the electorate during the 2012 election, the electorate was D+6. That was a data point.

  • themoderateman

    The objective data was created by how the respondents self defined. This was D+6. Romney/Rove pushed back on this data because they could not convince the media or the money backers they had a fighting chance if they bought into reality. This was a sad case of intentional self dellusion.

  • themoderateman

    Mormon.

  • themoderateman

    Maybe we will realize climate science is accurate and well defined as well? Evolution next? The anti-science, anti-intellectual movement in the GOP is ridiculous. There is nothing snobbish about an education. Nothing.

  • renl57

    They haven’t finished counting the votes yet!

    People don’t get this, but this has happened in other close elections before.

    The final vote totals won’t be in for weeks yet.

    I’ll bet many of these “missing votes” will show up as more overseas ballots and so on.

  • renl57

    No.

    Obama wasn’t on the ballot.

    Blacks and young voters are much more likely to turn out for a black President than for some white Congressman.

  • renl57

    Not true.

    Google for something called the Central Limit Theorem.

    The average of polls turns out to be closer to the actual population being sampled. In fact the standard deviation of that average of polls turns out to be quite small.

    The Central Limit Theorem suggests that the RCP average of polls will be more accurate than any single poll. And it was.

  • renl57

    So did every pundit that our side trusted: Michael Barone, for example.

    Why aren’t we having diaries criticizing him???
    Nearly every conservative pundit said Romney was going to win. They were obviously telling us what we wanted to hear, because conservative journals pay them to tell us that.

    The next election that comes up, I will listen to Nate Silver, not to Michael Barone.

    I care only about final results. I don’t care if they involve some guesswork.

  • MoeLane

    We ban religious bigots on sight.

  • Nelson

    I think the conservative echo chamber, which includes the blogosphere, has succumbed to the same measure of success as reality stars as to what is relevant. The more audacious the claim (without pushing too much the limits of absurdity) the more publicity it garners. Publicity, in reality television, my friends is its own reward.

    Cases in point:

    Dick Morris -Romney will win in landslide. Any self-respecting numbers guy and more importantly, one who is living in the real world, would know that such claim is preposterous for either one of the following two reasons: there has not been a landslide election since 1964 (Johnson/Goldwater). Since the 1980s, the electorate has become much divided and the candidates less distinguishable (all running to the right of center). Fertile ground for a non-landslide.

    Donald Trump -Need I say more? No I shouldn’t. I’d feel more cheapened than I already do talking about Dick Morris.

    Fox News -Need I say more?

    John Sununu -What he said about Colin Powell was red meat to the conservative base, but effective in alienating the minorities and the youth whose sense of race is a bit more evolved than us old timers.

    Neil Stevens -Blaming those who got it right (PPP, Nate Silver, etc.) for making a more plausible assumption garners him one of his most discussed diaries. To many of his readers’ credit, they see the foolishness in his non-approach.

    The accountability of reality stars is only to themselves. They are as good as the publicity they attain, good or bad. This, non-surprisingly, fits the conservative dogmas of you’re on your own, and the end justifies the means. If you attain success, regardless of the damage you cause to your environment and fellow humans, you have nothing to apologize for.

    If we were such individual beings, we don’t need society and in extension we don’t need politics. After all, politics is the game of who gets what and how much. We have finite resources and we use politics to duke it out as to who gets what and how much. As such, we’re inherently interconnected. The less greedy and self-serving we’re the healthier the societies in which we spend our short lives.

  • leftylurker

    But it occurred…there is something to that

  • lapert

    How is a target party ID mix any more objective that a target demographic mix? Seriously, you are being ridiculous – they were both weighting it based on subjective opinions of turnout and it turns that averaging everyone’s subjective views produces a better prediction – everything else produces denial.

  • billcor

    Neil, how would the most accurate polls be “rigged”? They made assumptions about the electorate and those turned out to be correct!

    Nate Silver just took an AVERAGE of the state polls, and got 50 of 50 states right. How is that “poll rigging”?

    It was so disappointing of our side to get our hopes up with all this “unskewing” nonsense. We need to respect the data! Many commenters here just ignored state polls, and followed overpaid consultants “hunches” like Dick Morris and Karl Rove. If we had come to grips with the data earlier, we could have given Gov. Romney positive feedback on his campaign.

    I am sick and tired of our side getting our butts kicked because people don’t pay attention to numbers.

  • Dan Middleton

    Nate Silver is engineering his own obsolescence. As Sean Davis pointed out a week or two ago at The Daily Caller, setting aside the whimsical secret sauce of the way Silver “weights” polls according to his personal judgments (or biases), what Silver does can be done by anybody with the time to feed poll toplines into Microsoft Excel with a Monte Carlo plug-in. You need to be a better mathematician (or at least have more patience) than a humanities major like me to do this stuff, but that’s hardly saying much.

    Even more fundamentally though, Silver’s premise is: polls are trustworthy, and the guy who is ahead in the most state polls in key states is going to win. Where would we be without Nate Silver to deliver us this insight? I watched him have a little Twitter-spat with Matt Drudge about the polls in 2008. The ’08 polls were “dead-on,” Silver insisted. Fortunate for Silver, since he built his reputation on interpreting them. If polls are dead-on and a reliable predictor of outcomes, what, exactly, do we need Silver for?

    I won’t give you a percentage figure on the probability, but this moment is likely the peak of Silver’s popularity and influence. Conservatives like Cost raised his profile through their back-and-forth with him, and now the election results have given him vindication. His book is flying out of Amazon warehouses.

    But where does he go from here? Obviously the first election where his predictions happen to be substantially contradicted by the actual results, his reputation as a soothsayer will take a substantial hit. He will have explanations or excuses, and the rest of the media will cover for him, and he will survive, but he will be damaged nonetheless. Others have pointed out that he didn’t even make an attempt to predict house races this year, and this was a shrewd move on his part; his methods clearly don’t work nearly as well in that arena.

    But even assuming the best-case scenario for Mr. Silver, he has nowhere to go but down. The novelty of his methods will wear off. Other people will copy them, with minor variations and modifications, and start producing similar statistical probability forecasts every election cycle. The pedestrian simplicity of what Silver actually does, the lack of any truly revelatory innovation or insight in it which I alluded to earlier, will start to become more and more apparent.

    The career forecast for Nate Silver is to become either useless or superfluous.

    Now that said, it won’t stop him from enjoying a long career as a talking head and newspaper and magazine columnist alongside the host of other poll wonks and prognosticators, most of whom are useless or superfluous themselves. And aside from the exasperation caused by the smug, casually-informed liberal internet commentators who treat Silver like a guru, I don’t begrudge that kind of career to a baseball starts nerd made good. But I hope he enjoys this peak while it lasts, because the product he sells is one that tends to lose a lot of its shine, sooner or later, one way or another.

  • zipzap

    This is a weird post. All pollsters take their raw data and massage it to achieve what they believe to be the most likely outcomes. This is why people pay them: to extrapolate from what data they gather. People like Nate Silver then aggregate that, weighting for past reliability, etc.

    It seems like you’re railing against the whole concept of polling unless the pollsters only share raw data. All of the pollsters (Rasmussen, PPP, Gallup) actively ‘rig’ the polls using their best-guess information bias.

    Stop trying to blame the polls and data-heads and READ the data instead.

  • http://travismonitor.blogspot.com Freedoms Truth

    “Where was our GOTV?”

    RIGHT. That is what I concluded after looking at the exit polls and the results. There were millions of soft Republican voters who didnt show up.

    http://www.redstate.com/wosg/2012/11/09/it-was-the-gotv-all-along/

    Had Romney done better GOTV he might have confounded those polls.

  • http://travismonitor.blogspot.com Freedoms Truth

    Obama had a huge number of disappointed voters, because he broke promises, screwed up the economy, and was/is a poor leader. He was at 43% approval earlier in the year.

    Given that, it was hard for him to win… but he did, mainly by catering to his base rather than chasing the moderate voters.