Wednesday, July 23, 2008

The Deal with Those Rasmussen "Leaners" [Update]

Yeah, what is the deal with that switch? On its simplest level, the chart below looks at the difference in the Rasmussen poll margins based on whether leaners were included in the data. That distinction seems to have been instituted following July 4. The polls that started coming out on July 9 began to use the language "with leaners" to describe the alternate numbers to what was in the headline. "With leaners" data gives McCain about a one point bump when we look at how the margin between McCain and Obama is affected based on whether leaners are included. It has been rare that Obama has gained once "leaners" are included with those who directly answer the candidate preference question. Interestingly, the two polls today gave the Illnois senator a boost once leaners are included.

Rasmussen Polls Since w/Leaners Distinction was Added (7/9/08)*
Statew/o Leaners
Undecideds Drop
Minnesota (7/10)
Minnesota (7/22)
New Jersey
North Carolina
North Dakota
South Dakota
Avg. Change+1.08-5.4
*The "with leaners" distinction was added to reports that were released beginning on 7/9/08. The date on which these polls were conducted (The ones that these releases were based on) stretches back to 7/7/08.
**Rasmussen has only conducted one poll in these states. Therefore, the difference was taken from between the with and without leaner numbers within the same poll in these cases.

The problem here is not one of the changes in the margins though. It is one of comparison. You can't directly compare the new "leaner" data to past Rasmussen polls that did not include the respondents that meet that description. Obviously if leaners are pushed in any one direction, the number of undecideds decreases. So, if we look at the data concerning undecideds plus those supporting other candidates (not McCain or Obama) in the most recent polls and in the one immediately prior to the inclusion of leaners, we get a better sense of how much the undecided total has dropped. We can look at this within each poll; looking at the with and without leaners numbers, but what we are trying to capture is the problem of comparing the new, with leaners polls with the old, without leaner polls. And what we see is that on average, the percentage of undecideds drops by more than five points per poll when leaners are included in the topline numbers. Now, we expect to see the number of undecideds drop this time of year...naturally. But we don't expect that decrease to be manufactured. And the catch is that everyone (FHQ included) has been using Rasmussen's "with leaners" numbers since the switch. The result is that comparisons and subsequent analyses--whether used for electoral college projections or not--are open to a potential bias.

In our case, here at FHQ, I took the liberty of changing data to reflect the "without leaners" view across all the Rasmussen data. I altered the margins of these 24 polls then, to pull them in line with the pre-switch polling methodology. The effect that had on our state-by-state averages was negligible. The only change was that Ohio slipped back into Obama's column (Due to the new Rasmussen poll in the Buckeye state, Ohio has moved from an Obama toss up to a McCain toss up.). Again though, that isn't the real issue. One poll among many in the average is not all that consequential. However, when we continue to compile "with leaner" polls, they collectively have the potential to skew our examination of the electoral college. And that just so happens to be contrary to what we want to accomplish with this endeavor. So let's just lop off the Rasmussen data and be done with it. Well, that deprives us of a valuable source of data. Since Rasmussen made the switch (post-July 4) there have been 43 new polls. 24 of those polls have been from Rasmussen. That's approaching 60% of the data. I don't then, want to throw Rasmussen out. What we can do is continue what we've begun here: to chart how much of a difference the "with leaners" data has on our electoral college projections. And as we do with the monthly examinations of how the averages have changed from state to state, we can observe these differences periodically as well. After a month or so, we will then be able to see if there is any significant bias attendant to including the leaner data and how large that impact is.

A belated thanks to reader, SarahLawrenceScott, for getting the ball rolling on this examination.

Recent Posts:
The Electoral College Map (7/23/08)

The Electoral College Map (7/20/08) [Update]

The Electoral College Map (7/16/08)

No comments: