On the first sets of numbers (comparing the last digit distributions):

You’ve conveniently dropped the fact that he’s comparing two sources of polling data and noting the differences, not simply comparing one data source against an assumed baseline. The methodology used to actually choose the numbers to compare is insignificant as long as two conditions hold:

1) The two data sets are of the same nature
2) The same methodology is used on both

In particular, note the comparison with Quinnipiac, which he chose because of the highly similar nature of the data being compared (and he explains his methodology here). There is no question that the two data sources are of a very different nature, despite the fact that they’re measuring the same thing, so visibly so that the need for a statistical test like chi^2 is unnecessary (it would merely confirm the obvious, although for reference chi^2 for these two gives confidence <0.0001 that they’re the same). Crying foul play because the obvious isn’t beaten like a dead horse seems like something of a stretch on your part.

On Benford’s law:

It’s well established that Benford’s law can be extended to later digits (http://en.wikipedia.org/wiki/Benford’s_law#Generalization_to_digits_beyond_the_first). The application to polling data may be a stretch, but note how he uses it: “This would be consistent with a distribution that at least partially observes Benford’s Law, in which smaller digits are more likely to occur.” He does not state that it *does* observe Benson’s law, only that if it did these results are consistent.

On the Oklahoma student poll:

I appreciate that this received only a one line comment, in a seemingly obvious attempt to gloss over the fact a full fledged argument would hold no water. Comparing to artificially simulated data is a completely valid thing to do when you’re attempting to demonstrate that the data you’re analyzing appears to have been artificially generated. Noting that the maximum difference between an artificially generated CDF and the actual CDF is only ~0.025 and then considering the incredibly simple model being compared against has to at least raise some questions (FYI, with my sample size of 1000 artificial students this gave a confidence of 93.5%. that the two distributions are the same, using the Kolmogorov-Smirnov test). As a final sniff test, and answer this honestly (if only to yourself), do you really, truly believe that of 1000 high school students, none could get more than 8 correct? Take the quiz yourself and see how you do and think about whether that is even remotely plausible.

I apologize for exceeding the expected level of statistical sophistication on this site, enough so that you feel compelled to accuse me of cutting and pasting. For your benefit I’ll present some possible alternative explanations for the lack of formatting:

1) I wrote my entire comment in this nice box, without too much concern for detailed formatting, so long as my points are made clearly (I believe they are)
2) I copied my work out into notepad to make sure it wasn’t lost while I waited 24 hours for my registration to go through and then pasted back into the browser.

Interestingly, both are true. If you have any evidence to support your claim that this is plagiarized, please do present it (perhaps you’ve seen this elsewhere, which I would find truly amazing). Or, in fact, if you can even point out a place where some formatting is obviously missing from my post, or even something in my comment that isn’t a direct response to the article, I would appreciate it. Otherwise, please refrain from making incorrect accusations based on a gut feeling.

Now say that you?re sorry for being so sloppy and try again. (yeah I cut and pasted that from your post)

In particular, note the comparison with Quinnipiac, which he chose because of the highly similar nature of the data being compared (and he explains his methodology here

Sorry, the “here” used was meant to indicate “at this point in his article”, as in “here in his article he describes his methodology, as opposed to elsewhere when he might not have”, not a missing hyperlink.

PS: There?s no way for you to link to this directly without also making visible my observation that Nate Silver never, ever predicts a race where the outcome isn?t already conventional wisdom. I mention this because you did wait a day to post, and I?m a fair-minded man.

I wasn’t aware you had made such an observation (at least not until you mentioned it here), so I had no reason to avoid validating it, but I must say I disagree with it now that I’m aware of it. In fact, I think it was his ability to predict the electoral outcome of the 2008 election better than anyone else (even conventional wisdom) that won him so much attention in the first place.

That aside, I’m not sure how this relates the objections raised in my post, which have nothing to do with his ability or willingness to predict outcomes, but rather are a direct criticism of the faulty logic applied in this article’s critique of the Strategic Vision analysis. To those criticisms I gladly welcome debate and discussion, as I find the whole controversy quite interesting (whichever side you attribute the fraud to), but find the level of discourse used in this article to be somewhat underwhelming. In particular, it relies largely on straw man and tangential attacks (i.e. the whole Benford’s law approach, which also happens to be just plain wrong), ad hominem (“pretend math”, “mathematical illiterate”, “willful incompetence”, even when it is well established that Nate knows his mathematics), and flat out misrepresentation of the original material. For that last one in particular, here are some examples:

• “He attempts to provide a thin veneer of justification for his work by citing Benford?s Law. ” Yes, I’m bringing up the Benford’s law thing again because it seems to be the most fleshed out criticism of the argument. A quick reading of Nate’s article will make it quite clear that Benford’s law isn’t used as a justification for anything, but is simply brought up as a potential characterization of the data, the bulk of his justification relies on (1) the incredibly obvious differences between the data sources (yes, not just magical eyeballing, please don’t pretend the differences aren’t obvious), which a chi^2 test can verify and (2) the wide array of issues covered by Strategic Vision polls. Which brings me to my next misrepresentation:
• “All it would take for Strategic Vision to get a distribution like he shows, is to have a bunch of polls that show something like, oh, R 48 D 49 Other 1 Undecided 2. But we don?t see that because, guess what, Silver stripped out the Others and Undecideds!” As described in the original article, “Strategic Vision’s polls cover a wide array of topics: Presidential horse race numbers in any of a dozen or so states, senate and gubernatorial polling, primary polling, approval ratings of various kinds, polling on issues like the war in Iraq, and more abstract questions such as whether voters think that ‘experience’ or ‘change’ is the more important quality in a Presidential candidate. No one type of question, in no one state, represents more than a relatively small fraction of the sample. Under those circumstances, I can’t think of any reason why the trailing digit wouldn’t approach being random — although there absolutely might be reasons that I haven’t thought of.” So it seems pretty clear that the potential counter to the wonky distribution is based on incorrect information. Furthermore, any such biases should be equally present in a variety of polling sources, and it’s quite clear from his comparisons that they are not.
• “He asserts out of thin air that the distribution of last digits should be uniform. How is this the case?” First, see the quote from the original article above detailing Nate’s rationale (which is there, and not pulled from thin air), and furthermore note that it is, in fact, well established that last digits tend to follow a uniform distribution, even in election results (see, for example, this paper)

I was simply attempting to raise the level of discourse by providing justification for my responses. A few more responses, which I’m sure will be dismissed equally quickly.

a) The parts I?ve skimmed have shown you to be snotty, and sputtering at the mere suggestion that Emperor Nate has no clothes

I have made no claim that Nate is right, only that your reasoning is insufficiently precise and, in fact, insufficiently correct to warrant your conclusions. I’m also fairly confident that the perceived snottiness comes from (1) a tendency on my part to be overly blunt when responding to arguments I feel are insufficiently supported and (2) a tendency for human nature to characterize opposition as snotty/biased/angsty/etc as a sort of knee-jerk defense mechanism.

b) You?re a Democrat, right? You?re not my audience. Heck, I even spelled out in my piece that I expected you guys to be trying to drown me out.

Actually you did not spell that out, or even mention any potential opposition to your claims. Further, pieces like this, which are a critique of methodology, conclusions, etc, are fundamentally scientific in nature and as such have no ideologically aligned target audience. Facts are facts, correct approaches are correct, and scientifically drawn conclusions are valid regardless of who draws them. However, for reference, my political ideology is libertarian and, although I did vote Democrat in the most recent election, I would describe myself as more of an independent.

c) I just skimmed a little more, and you?re cheating. This isn?t Weekend at Bernie?s. You can?t defend Nate from the charge of not doing his homework, by doing his homework for him.

This is also untrue. I did the following things:

• Defended his methodology, as I believe it follows sound practices
• Described how carrying out a chi^2 test on the last digit distributions was unnecessary because such a huge visual disparity makes such a test unnecessary, but then carried it out anyway to demonstrate how overwhelmingly conclusive it was
• Pointed out your misrepresentation of his Benford’s law argument
• Ran a statistical test on the Oklahoma student polling comparison to once again make the point that analysis based on visual similarity is sufficient in cases like these.
• Provided a direct quote from his original article that refutes your claim that he pulls the uniform distribution assumption from thin air, and provided further reading in case you were actually curious about that assumption.

So yeah, I?m just gonna pat you on the head, say ?Way to go, sport,? and move on.

Had I been looking for your approval I’m sure I would feel great right now. However, since I was honestly hoping for a well reasoned response to (what I feel are) sound objections to your article, I’m instead just a bit disappointed.

And I’ve also noticed that you’ve still not responded to any of my points. I can probably even guess why:

1) It doesn’t matter what I say here, because it’s your blog and most readers will side with you regardless of what happens.
2) You don’t think it’s worth the effort, since I’ll probably still disagree with you.
3) It’s more fun to taunt dissenters than to address them.

It’s too bad there isn’t more of a commitment to open discussion and respect for those who disagree with you.