Nate Silver Exposes RCP for Skewing Results Toward McCain

Денг Гордон · Oct 2, 2008

Last week, I discussed Real Clear Politics' decision to exclude the Research 2000 daily tracking poll from their national averages. After a lengthy discussion with RCP founder John McIntyre, I decided to defend them, pointing out that while RCP may have a debatable framework for deciding which polls they do or do not include, they at least appeared to have applied this framework consistently.

My gut-level takeaway from my conversation with McIntyre was somewhat different from what I represented, however. My gut-level feeling was that RCP was in fact cherry-picking those results that were to its liking, and then coming up with post-facto rationalizations to justify its decisions.

RCP made a decision yesterday that convinces me I should have trusted my gut. Specifically, it was their decision to re-include polls from American Research Group (ARG) in their state-by-state averages.

ARG had been effectively "banned" from RCP for several months now, going back to the Democratic primaries. Sometimes RCP listed ARG polls with an asterisk and did not include them in their averages; more often they didn't list them at all.

Why didn't RCP include ARG's polls in its averages? Presumably because they had an exceptionally erratic performance during this year's primaries, a performance that -- when coupled with ARG's relative lack of disclosure about its methodology -- had led RCP to conclude that their polls were not credible.

As you may know, I am not much of a fan of ARG myself. However, while ARG ranks toward the back end of our pollster ratings, so do a lot of other pollsters that RCP has no problems with.

What I do know, however, is that whatever RCP's reasons were for excluding ARG from its averages, none of those reasons should have changed over the past 48 hours. Did ARG suddenly improve its level of disclosure? No. Did their polls suddenly become more accurate? How could they, when we haven't had any elections to evaluate them.

No. The only thing that changed is that ARG released a set of three polls yesterday that had considerably more favorable numbers for John McCain than other contemporaneous polls of those states. In each of those states -- North Carolina, Virginia, Nevada -- ARG is presently the outlier on the John McCain side (tied with Mason-Dixon in the case of Virginia). RCP did not feel any compulsion to include ARG's numbers when ARG cycled through all 50 states a couple of weeks ago (including many where there has been very little polling). Only when ARG released this gravity-defying set of polling in Virginia and North Carolina and Nevada did RCP suddenly have a change of heart.

This is not RCP's sole problem with consistency. Take a look, for example, at which polls they presently include in their averages for Virginia:

You'll see three polls -- SurveyUSA, Mason-Dixon, and ABC/Post -- that were conducted at essentially the same time. The ABC/Post poll was in the field from the 18th to the 21st, Mason-Dixon from the 17th to the 22nd, and SurveyUSA from the 19th to the 21st. And yet, the Mason-Dixon and ABC/Post polls are included in its average whereas SurveyUSA is not. Guess which one had the strongest numbers for Obama?

A similar example from Minnesota. The Star-Tribune conducted polling from September 10th through September 12th; SurveyUSA conducted polling from September 10th to September 11th. The Star-Tribune poll, which shows a tie, is included; the SurveyUSA poll, which showed Obama ahead by 2, is not.

As long as we're having fun with this, how about Alaska? Anchorage-based pollster Ivan Moore had released a poll in July, then showing just a 3-point race between McCain and Obama. We included it, Pollster.com included it, and RCP did not. But then last month, when the same firm released a fresh poll showing the profound effect that Sarah Palin had on the race (McCain +19), RCP decided they were a worthy pollster.

Or how about North Carolina Senate? RCP presently includes two weeks-old polls from SurveyUSA and Research 2000, whch had shown Elizabeth Dole with a solid lead in her race against Kay Hagan. But they didn't bother to include more recent polls from Elon and Civitas, which show the race essentially tied.

Why is one of Mitch McConnell's internal polls included in their Kentucky Senate averages?

Why, a week or so ago, did RCP include a poll from the Marketing Resource Group of Lansing, Michigan, which the Detroit News describes as a "Republican consulting firm", while excluding a Nevada poll that came out on the same day from Project New West, a Democratic strategy firm? Might it have had anything to do with the fact that the Michigan poll showed McCain ahead -- and the Nevada poll showed him behind?
Click to expand...

http://www.fivethirtyeight.com/2008/09/real-credibility-problems.html

Log in or Sign up

Nate Silver Exposes RCP for Skewing Results Toward McCain

Денг Гордон Member

Share This Page

Log in or Sign up

Nate Silver Exposes RCP for Skewing Results Toward McCain

Moderators

Денг Гордон Member

Share This Page

Useful Searches