« BACK  |  PRINT

RS

FRONT PAGE CONTRIBUTOR

Jon Ralston and Ed Morrissey call out CNN and Time

By: Neil Stevens (Diary)  |  October 29th, 2010 at 12:00 PM  | 

RESIZE: AAA

CNN

Now that Nate Silver has declared PPP to have a Republican house effect, all eyes turn to CNN to see what kind of house effect they must have.

Has the whole polling world gone Republican? Why does CNN get the results it does? Jon Ralston and Ed Morrissey have questions.

I mentioned yesterday that transparency, including the publication of crosstabs, was more important to a pollster’s credibility than raw accuracy is. This new controversy confirms that, I think. The more data that comes out there, the more people can dig in to look for anything “funny.”

And it is funny that a state as large as California would have nobody at all, none, in the 18-34 age range be polled by CNN and Time. Or worse, nobody at all in the 18-49 range in Colorado. And no urban voters at all in Pennsylvania? What happened to Philly and Pittsburgh? The Langoliers?

Something’s just not right with these polls, and we’d never have noticed without the sufficient transparency in the data that CNN and Time did provide.

From Unlikely Voter

Tags: , , ,

COMMENTS

POSTING RULES  |  LOGIN TO COMMENT  |  REGISTER TODAY
  • http://itsaboutfreedom.proboards.com IronDioPriest

    Are you saying that the pollsters are deliberately trying to make it look like the GOP wave is larger than it is?

    What is the “House Effect”?

  • The_Rebel

    This column he wrote two weeks ago tells me all I need to know about this moonbat:

    http://www.lasvegassun.com/news/2010/oct/13/blind-hate-financing-angles-run/

  • hungarianfalcon

    is an inherent (systemic) bias. Note, bias does not automatically mean intent. You can have bias that are deliberate and some that aren’t.

    What I think is totally comical is Silver diving who has the house effect 5 days before an election. Why not keep you yap shut until midnight on 11/2 when the real House actually says it’s peace? It could end up that the purported house effect polls were actually right and the “correct” polls had the house effect.

    I think this election is the riskiest one in a while for pollsters. There will be kings made this election among the pollsters and the unknowns are so significant that we’re going to see some real surprises, hopefully in the direction of the red team but conceivably in both directions. My money is against Silver strictly because of the bad karma of the NYT. I hope he got a big paycheck for getting in bed with those clowns.

    Silver does a good job of walking a fine line between being a democratic hack pollster and objectively calling things. You can tell a true objective person because their conclusions should be evenly and randomly distributed for both Blue and Red. Silver is certainly on the Blue side but not by enough to be beligerant. Can’t remember the last time he was wrong on the red side w/any signficance.

    HF

  • proudmarinemom

    There have been a number of comments on other sites suggesting that a lack of landlines among the 18-34 age group is having an effect. That seems to be the age group that would be harder to reach via landline telephone calls. My kids are surgically attached to their cell phones and Facebook accounts and never, ever use a landline phone for anything.

    Not defending these sloppy polls, but would like to read your thoughts on this issue.

  • hungarianfalcon

    nt

  • http://www.hakubi.us/ Neil Stevens
  • proudmarinemom

    .

  • congressworksforus

    There are two other groups that combined outweigh this polling myth.

    1. Older, rural people are now also heavily cellphone (and rural doesn’t necessarily mean BFE…)

    2. Middle-aged business people (think Smartphone).

    You couldn’t get me on a landline either!

  • http://www.hakubi.us/ Neil Stevens

    proudmarinemom refers to the polling debate surrounding people who HAVE no landline, not people who have one but can’t be reached on it.

  • Obilisk18

    Morrissey and Ralston both mis-read the CNN polls. It’s not that CNN didn’t poll anyone in these categories. They did. If you look at the Nevada poll, for instance, you’ll see that while they don’t have any numbers for 18-34 or 35-49, they do have numbers for under 50 and above 50. If you multiply out the percentages (algebra and whatnot) you get around 38-39% in the first category (under 50) and 61 to 62% in the second category (over 50). Which is pretty much compareable, if you look at the exit polls, to the age breakdown in the last mid-term election Nevada (62% of voters were over age 45). They didn’t include the specific cross-tabs in the 18-34 or 35-49 categories because there weren’t enough respondents to make it statistically meaningful. Even in the under 50 category they had an 8% margin of error. Once you start getting higher than that it’s not really worth reporting the data. I sent this explanation to Ed and he said, you know, that he’d never heard of a poll that didn’t have enough respondents for a statistically meaningful sample. But really, most polling just include all the data they have, even if it’s not statistically meaningful. Because people want to see it. You’ll notice that most polling companies don’t include margins of error for subgroups.

    This explains why, for instance, polls of really small subgroups, say minorities, tend to be sort of all over the place. In the Florida Senate race we’ve seen polls showing Rubio winning anywhere from 20% to 70% of the Hispanic vote. It’s not that there’s that much volatility in the Hispanic vote- it’s just that, you know, when you’re talking to 60 respondents it’s not very hard to go badly wrong.

    That said, I don’t know what’s going on with that Philly data…I mean, surely Philly’s a large enough portion of the electorate to draw some conclusions from? Maybe they rolled it into the “Philly Suburbs” number for some reason (maybe to make that smaller category statistically significant)?

  • mikeinnyc

    You are completely wrong. For all pollsters, (like Nate Silver) being right in retrospect either makes or breaks your career. There is NO WAY that he would distort his results to favor one political party or result over another. Being wrong would only discredit him in the next election and make his services worth less money.

  • http://www.hakubi.us/ Neil Stevens
  • Adjoran

    Much of this work is done by temporary workers who won’t have a job next week. They probably just didn’t do the detailed publishing because the office party had already begun.

  • distantvoter

    He specifically acknowledges that “house effect” is comparing one pollster to the general body of pollsters, and that just because he is adjusting for house effect doesn’t mean the pollster is actually wrong. It is just a guess.

    Silver, Cook, Sabato, RCP, all the pollsters, everyone is guessing. We simply don’t know what turnout will be. The consensus guess of all these experts is somewhere around 55-60 seats, but they all admit that the polls may be wrong, and that it could be much, much more. Silver in a recent post said that his model gives something like a 30% chance it will be more than 70 seats.

    I have no problem with his model at all. I don’t think it can handle a mega-wave, and I think there’s a good chance that is what hits on Tuesday.

    He uses past performance of each pollster to weight their results in this cycle, to get a weighted average of polls. So if SUSA was right on the money last cycle, they would be weighted very highly. If PPP overrated Ds by 4 points, they would be weighted lower. That seems very reasonable.

    Then, he compares each pollster to the weighted average for this cycle to find out if they seem to be biased one way or another compared to the rest of the polls — for this cycle. If they are, he makes a small adjustment to their results — house effect.

    If you are going to use an average of polls, rather than just trust a single pollster, I can’t think of any better way to do it.

  • http://www.hakubi.us/ Neil Stevens

    Nice cop out, but nothing to back it up.

  • distantvoter

    Is going to struggle to reflect Tea Party impacts, for instance. There are highly motivated voters who may not have voted in 2006, and so don’t get through the likely voter screens.

    Backup? Even Gallup doesn’t know, which is why for the first time ever, they are using two likely voter models. That would say they don’t know, and there’s a large measure of guess work here.

    Cook is guessing, that’s why he gives a range, says it could be higher than that, but says he’ll be washing dishes if the D’s hold the House. He doesn’t know. He knows it’s going to be a stomping, but he doesn’t know how bad it will be.

    Even the best pollsters don’t know what the electorate is really going to look like, because the likely voter screen is so hard to get right this time. Jay Cost has made this point repeatedly. Did you see his “Hulk” post? He doesn’t know what the electorate will be like. He’s guessing a R+8 generic, but hoping for and suspicious that it will be R+12, based on Gallup’s lower turnout model.

    Michael Barone has a huge amount of uncertainty, talking about 1894 vs 1994.

    All these experts have tremendous uncertainty. They know the Ds are going to take a world of hurt, but they are simply guessing at how bad it will be, and you can see it in what they say.

    How many predicted O’Donnell would win the primary three weeks before she did it? Not very many. Who predicted Paladino would win in NY? How many people really thought Mollohan would lose his primary in WV? How many expected a poll yesterday showing Pingree behind in Maine, or that Bawney Fwank would be in trouble, or that Dingell would have to campaign hard, or that Oberstar would be in serious trouble?

    This is the most volatile election in our lifetimes, and no one knows what will happen. All the experts are guessing 55-60 seats in the House, but they all have a LOT more that are competitive than that, and chances are very high that there’s going to be some shockers that none of them anticipated.

    When Silver predicts 53 seats but says there is a 30% chance that he has underestimated by 17 or more, he’s saying as clearly as can be, “I’m guessing. I really don’t know how bad this is going to be for the Democrats.” And all the others are saying similar things.

    You KNOW, with a reasonable level of certainty, how it will come out if the swing is +8, or whatever. But no one knows or has any level of certainty, what that swing will really be, and there are so many seats in play that a small change in the swing can mean 10 or 15 more takeovers.

  • http://www.hakubi.us/ Neil Stevens
  • hungarianfalcon

    I think you’re missing the nuances of a standard deviation, random/gaussian distributions, counting statistics and all that. Not to mention my comment about bias not necessarily being deliberate. Take a look at the nature of his followers even before he got in bed with the NYT. Do you think he magically got a left wing distribution because he was a paragon of virtue? Me thinks not. Part of that fan base was earned by telling them what they wanted to hear.

    In the game Silver plays, 1% drift in the generic ballot is a lot, particularly in the house where polling is far less comprehensive than in the senate. Yet that 1% is still well within the margin of error the pollsters claim. So one can easily have their own bias (again, deliberate OR otherwise) and have it fall within the noise, as one would say in my field of expertise (analytical chemistry). Falling within the noise would mean that his results were not discernible from the results of the general field of polling prognosticators.

    Again, I ask, when was the last time a Silver prediction ended up being significantly on the red side of the result. I don’t stalk him so I can’t say there hasn’t ever been one, but there hasn’t ever been one that I’ve seen.

    HF

  • bluerose75

    Quite simply someone like Neil is a numbers cruncher. He is no different then the pollsters. He would be lost without poll this and poll that. He thinks History will tell you this and history tells you that. Quite frankly he has no idea who is going to win and I would not put an once in faith in anything he says. He has no more insight then anyone else here. He will defend himself to the bitter end and dismiss you with one liners becasue he is truly guessing himself. He makes me laugh.

    I will guarantee you this Neil will not have the right count on election night and like all pollsters and people like him they all use the same excuse. Polling is not perfect, this was missed, that was undercounted, that was overcounted and so on. They all have excuses. We should all ignore them because they deserve no more credence than you or I.

    Election in 2004, 2006 and 2008…who cares. There is no poll that will be right this election cycle. So many of us want to have some tangible sign to have faith and confidence in the outcome. That simply does not happen. You have to vote, get others to vote and become involved. Nate Silver, Sabato and Cook just guess. And they get paid for it…LOL…Look at ESPN, WEATHER CHANNEL and others how often are they right? My goodness on NFL Countdown the entire cast of ESPN (Berman, Ditka, Carter and Jackson) are lucky if they are right 50 to 60 percent. Remember these are the experts…like Neil here! Please!

    This election cycle will have many surprises…NOT CAUGHT IN POLLS!! There is no way anyone can measure the result. It is guessing….yes I know they have their “Likely Voters Screen” and other garbage.

    Vote and ecnourage others to vote that is the best way to gaurantee victory. Not sitting on a Poll Throne like Neil acting like he has more insight then you do. He is a number cruncher pure and simple. Like many others in “HISTORY”

  • distantvoter

    And that turns “wild guesses” into “informed guesses”. But there is no real certainty in this environment.

  • distantvoter

    I agree with your basic premise. But polls and number-crunching are valuable in guiding where to put resources/effort.

    If I have limited dollars to give, I would rather give to beat Barney Frank (where there’s at least a chance) than Pelosi, where there is no chance.

    Number crunching helps us to evaluate polls, and polls tell us something.

    And I would not be quick to say that there is no poll that will be right this election cycle. In fact, it is almost certain that there will be a poll that is right. The difficulty is knowing which one, and the tremendously high level of uncertainty that we have. We don’t know which poll is right. We can guess based on biases and history, but we don’t know. And because of the wild cards in this election, we have to admit the possibility that all of them will be wrong. There is a huge level of uncertainty.

    Cost, Silver, and to a lesser extent Rothenberg, Cook, and Sabato are also number crunchers. There is nothing wrong with that. They acknowledge the uncertainty.

    Neil knows a lot more about polls and number crunching than most of us, and it is definitely valuable, especially in the Senate where there are a lot of polls on each race.

    But you are right that the best thing at this point is simply to vote and encourage others to do so. That’s the only thing left we can do, now.

  • distantvoter

    That Cost, Barone, Silver, and Cook all read polls, too. In fact, some of them may have access to private polls that we haven’t seen. The fact that they have polls doesn’t mean that they feel any more certain about what is going to happen.

    Silver says there is a 15% chance the Dems hold the House, and a 30% chance they lose more than 70 seats. He has polls, too, and is describing wild uncertainty.

  • http://www.hakubi.us/ Neil Stevens