Tuesday, October 14, 2008

How Big a Margin is Too Big to Make Up?

In yesterday's New York Times, John Harwood laid out the history of presidential comebacks from late polling deficits, but that isn't really what I'm after here. Reagan is the only candidate who mounted such a turnaround when he unseated Jimmy Carter in 1980. The question I'm asking, though, is one I've been thinking about for the better part of two weeks because of an anonymous comment I got. [Granted, I did the leg work for this a couple of weekends ago, but I'm just now getting to it. It may not be as pertinent now with Obama seemingly comfortably ahead, but I'll press on regardless. More information is better than less in my estimation.] Here's the idea:
"Here are the margins that Bush won these states in 2004:
Ohio 2.1%
Nevada 2.6
Colorado 4.7
Florida 5.0
Virginia 8.2

It's hard for me to believe that Virginia would change 8.3% in one direction(to Obama) therefor don't be surprised if/when McCain starts polling better in Virginia."
I'll have to say, 8.3% does sound like a lot to make up in just four short years; especially in a reliably red state like Virginia. So at first glance, this sounds like an entirely reasonable hypothesis. But it is never good enough for me to simply take someone's word for it. The one question that arises from this is, 8.3% is a lot, but compared to what? What is the typical swing from one party to the other from election to election? Oh, I'm glad you asked. Let's have a look.

First off, I looked at all the swings from 1984-2004. That'll give us a total of five transitions and enough variation in the types of elections to give us a decent idea of what is business as usual in terms of these swings. There are several combinations here but two main hypotheses emerge from the electoral shifts we have witnessed over the last five cycles:
H1: Swings from one party to the other are likely to be larger in elections where the party in the White House changes than in years where the incumbent party is reelected.

H2: Competition matters. A transition from a landslide to a more competitive race is likely to affect a bigger shift than one from a competitive election to another. The biggest shift, though would combine both of these hypotheses. A switch from a landslide to another landslide where there is also a change in power translates into the biggest shifts.
Let's look at each of these five electoral transitions:
1984-1988: Big GOP landslide to Moderate GOP landslide
1988-1992: Moderate GOP landslide to Moderate Democratic landslide
1992-1996: Moderate Democratic landslide to Moderate Democratic landslide
1996-2000: Moderate Democratic landslide to Narrow GOP win
2000-2004: Narrow GOP win to Narrow GOP win

There are several types of transitions across these five cycles and that should allow us something better than a passing glance at what we're after and a decent test of our hypotheses. To start, we have a hypothetical threshold in place, 8.3%. From 1984-2004, how many states not only shifted 8.3%, but also changed hands?

Electoral Transitions (1984-2004)*
YearToward the Dems
Toward the GOP

Average Shift: 11.76% (toward the Democrats)
Total Shifts >8.3%: 35
Number Changing Parties: 9

Average Shift: 12.92% (toward the Democrats)
Total Shifts >8.3%: 39
Number Changing Parties: 22
Average Shift: 2.28% (toward the Democrats)
Total Shifts >8.3%: 10
Number Changing Parties: 0

Average Shift: 10.22% (toward the GOP)
Total Shifts >8.3%: 27
Number Changing Parties: 8
Average Shift: 2.15% (toward the GOP)
Total Shifts >8.3%: 7
Number Changing Parties: 0
* The states in bold and italicized are states that had shifts from one party to the other greater than 8.3% and changed parties from one cycle to the next.

Source: Dave Leip's Atlas of US Presidential Elections

There's a lot to look at in this table, so let's try to take it in bits. Let's tackle our hypotheses first. First of all, swings are both bigger and more frequent in years where power changes hands. The only case where there was no change in the incumbent party in the White House and there were shifts was in 1988. George Bush's victory in 1988 had nowhere to go but toward the Democrats following Reagan's thorough trouncing of Mondale in 1984. While there wasn't a change in the party in the White House after the 1988 election, there was a change in who was in the White House. That may explain some of the change. During both Bill Clinton's and George W. Bush's successful reelection bids neither saw many states change more than 8.3% from their initial election, nor did either see a single state change sides.

Not surprisingly, both of those elections (1996 and 2004) saw the smallest average shifts across all 50 states -- less than 2.5% in each. In the three elections that saw a change in power, whether a partisan change or a change in the occupant of the Oval Office, each witnessed average shifts of over 10%. Now, 2008 isn't following a landslide victory, and as a result the incumbent party is much less likely to see such widespread, large shifts and still hold on to power. If, however, Obama wins the 2008 election, we would be likely to see more and bigger shifts, though perhaps not to the degree of the shift in the change election of 1992. We could see a shift similar to 2000, though. The way things look now, the trend could be the exact opposite between 2004-2008 than it was between 1996-2000. Instead of moving from a moderate landslide to a competitive election, the way things appear now, three weeks out, a move from a competitive election to a moderate landslide isn't that far-fetched.

In a change election -- one where there is a change in power -- a shift of 8.3% or more isn't that uncommon from one election to the next.

Recent Posts:
The Electoral College Map (10/14/08)

The Electoral College Map (10/13/08)

A Follow-Up on ACORN


Jack said...

I'm pretty sure that if the election shits 8.3% then you have to have at least one state switch by that much. And we've seen much bigger switches than that in the past. FiveThirtyEight has electoral history charts since 1948. Discounting extreme examples caused by the Civil Rights Movement, such as Alabama being D+14 in 1960 and R+69 in 1964, it's not difficult to find huge swings. For example, Vermont was R+17 in JFK's 1960 victory and D+33 in LBJ's landslide, a swing of 50 points. Oklahoma went from R+50 in 1972 to R+1 in 1976.

Even consecutive similar elections can produce significant swings. 1952 and 1956 are as similar as they come. Both had the same candidates for both parties and were clear victories for Eisenhower, though not of 1984 proportions. But there were significant swings both ways between 1952 and 1956. Louisiana, for example, was D+6 in 1952, and R+14 in 1956.

Perhaps more interesting is to look at the relative electoral history charts below the "absolute" [for lack of a better term] charts. This allows us to eliminate any "landslide effect" and see how the state changed relative to the rest of the country. We see cases of huge swings relative to the electorate at large. Again ignoring LBJ losing the Deep South, and Carter's 59-point relative gain in GA in 1976, there are plenty of large switches, such as Maine going 14 points more Republican than the country in 1960 and 15 points more Democratic in 1964, a 29 point relative shift. Of course, 1964 numbers are distorted by the huge margins of victory Goldwater won in the Deep South, but even when looking at other years there are a number of 20 point shifts.

The conclusion to be drawn from this rather long-winded comment is that an 8-point shift is not impossible, or unusual, even in similar elections, especially given situations such as the changing demographics in VA and NC.

Anonymous said...

I'm starting to think Virginia can turn blue.

If you look at Real Clear Politics 2004 final NATIONAL poll average, it was Bush 48.9% Kerry 47.4%.

Right now Real Clear Politics has it McCain 42.0% Obama 50.2%.

Lets do the math:
48.9-42.0=6.9% drop republican
50.2-47.4=2.8% increase democrat

6.9%+2.8%= a difference of 9.7%

Now keep in mind that these are NATIONAL averages that could have nothing to do with Virginia, but as of right now I would predict an Obama win in Virginia BARELY.

I still think that of the 5 most watched/competitive states (Nevada Colorado Florida Ohio Virginia) Obama's margin of victory in those states will be in that order, with him winning Nevada with the highest margin and Virginia with the lowest margin, and that is IF Obama wins Virginia.

Nevada and Colorado have increasing Mexican populations that will make Obama cruise in those states.
With the economic colapse/recession the big senior citizen population in florida will want change and therefor vote Obama plus the white population is dwindling in Florida.
Ohio has a lot of Reagan Democrats and a reasonably high white population, but I expect Obama to still win Ohio.
Virginia is a southern state but is changing, the question is, are there enough southerners to keep Obama from winning Virginia, we will see.

The 5 states in order of Obama's margin of victory:
1 Nevada
2 Colorado
3 Florida
4 Ohio
5 Virginia

By the way I'm the Anonymous mentioned in this post.

Anonymous said...

By the way this ACORN thing is very disturbing.
The system is corrupt.
I'm voting Chuck Baldwin.
Both major parties are corrupt and need a message sent to them.

Jack said...

Yep. ACORN is a perfectly good reason to vote for Chuck Baldwin. I'm with you there. I used to support Obama but after all this crap with ACORN I think I'm going to vote for the Gene Amondson/Leroy Pletten ticket.

Anonymous said...

Any idea where I can pick up one of those campaign buttons, Jack?

Jack said...

I guess I let a bit of my inner snark out and owe Anonymous an apology. Sorry, whoever you are, and apologies in advance as this is bound to happen a few more times before the election.

Prohibition Party candidate Amondson is the only candidate I've ever seen whose website has more to do with his other endeavors than his presidential run (compare the website of Foxwoods Casino pit boss and Rhode Island senatorial candidate Bob Tingle (R), whose website has more and bigger pictures of other people than Tingle himself). The Prohibition Party website is no help either. You'd have to go to Cafepress.

Unknown said...

Nevada seems curiously resistant to national trends, and I'm not sure why. If things swing back toward McCain, I might agree with anonymous. If they keep going toward Obama, Nevada may very well be the closest of those.

Anonymous said...

If a national trend is also a Democratic trend, it could be that the oversampling of rural and really red Nevada, may explain the resistance.

We've seen some indication of a jump for Obama in Nevada, but I still don't have a sense of whether pollsters conducting surveys there have stumbled upon the right formula for surveying the state. The increased Obama support there doesn't match what has happened in Virginia or Ohio.

...or even fellow western state, Colorado.

Anonymous said...

My previous comments might be confusing.
What I'm saying is that of the 5 states I mentioned I predict Obama winning Nevada by the biggest margin, followed by Colorado then Florida then Ohio then Virginia by the lowest margin (if Obama even wins Virginia).

If you look at Real Clear Politics 2004 average for Nevada they had it Bush by 6.3% and Bush won Nevada by 2.6%, that is a 3.7% difference towards Bush.

The pollsters have a hard time polling Nevada for some reason.

Rasmussen's final poll for Nevada in 2004 had it Bush by 2%, which is pretty accurate.

Rasmussen was pretty accurate across the board in 2004.

The latest Rasmussen polls for the 5 states I've been mentioning are:
Colorado Obama +6
Florida Obama +5
Nevada Obama +4
Virginia Obama +3
Ohio Obama +2

Anonymous said...

I think Nevada and Colorado are a done deal for Obama.
Even if you take away the 33 electoral votes from Ohio and Virginia on your map Obama still wins with 278 electoral votes.

It's funny and sad that the co-writer of the illegal immigration amnesty bill (McCain) is going to lose the election because of the increasing Mexican population in Nevada and Colorado.

The republican party needs to wake up and represent the white middle class before the party collapses.

I'm a former Republican, now Constitution Party.
I voted for Pat Buchanan in 92 and 96.
I guess I would be considered a right-wing populist.

Unknown said...

Wow, anonymous...racist much?

Anonymous said...


Nope, just concerned about the white middle class and the shrinkage of it.

Jack said...

Nope, closeted racist, with a sprinkling of cultural nationalism, as evidenced by an earlier post of his on another comment. Or something like it.

I don't think NV has been that resistant to national trends - it was polling around M+2 when the race was even and has generally been about D+3 or so lately, even if it's been inconsistent. And the problems inherent in polling Nevada may have something to do with it. Maybe it's a litlte less sensitive than the average state but it's definitely been affected.

Jack said...

I should add that the "right-wing populism" that anonymous mentioned is basically a combination of bashing blacks, Hispanics, Asians, Europeans, poor immigrants, educated people and intellectuals, community organizers, Keith Olbermann, pacifists, socialists, feminists, the UN, gays, different lifestyles, value systems and cultures, liberals, soccer fans, Canadians, the disabled, the media, people who don't own guns, and anything that the average white American does not like.

Anonymous said...

Alright gents. We've veered off topic here.

Let me address some of your initial concerns. Yeah, the averages are certainly vulnerable to extreme outliers. We have two courses of action to deal with this issue.

1. Remove the outliers
2. Use the median

Allow me to take the easy way out for now simply because it is less time-consuming. If we remove the extreme outliers (I'll arbitrarily set that at any swing over 20 percentage points from one election to the next.), we see that some of the effect in the 1984-88, 1988-92 and 1996-2000 election is minimized, but not too much.

The 1984-88 figure drops to 10.57% toward the Democrats.

The 1988-92 number dips to an 11.21% shift toward the Democrats.

And the 1996-2000 average shift changes from 10.22% to 8.87%. That's right around that 8.3 threshold. So, by removing the outliers we see that if we move from a tight election (2004) to perhaps a modest landslide (hypothetically 2008), we might see an overall shift around this 8.3% figure that was originally proposed.

If I have some time tonight prior to the debate, I'll look at the median numbers as well and see what that turns up.

Jack said...

Apologies for being a jerk earlier.

Anonymous said...

No worries, Jack. Just striving for some southern civility here. It's crunch time. People are getting tense.

Robert said...

Obama put McCain in a box on the immigration issue. In his most blatantly misleading (untruthful, whatever you want to call it) ads, he went for the Hispanic vote by grossly distorting McCain's record. McCain mentioned immigration last night, but he can't afford to say much about his position without alienating either his base, Hispanics or both!

Unknown said...

Southern civility? What have you got against Northerners, you little--

Just kidding. :-)

On a more serious note, I noticed something interesting looking at the Relative charts. For the most part, the regions act somewhat uniformly over time: Both South regions (except for West Virginia, which is a border state) have generally moved from blue to red, the Northeast regions and the North Cantral have moved from red to blue, Prairie and Southwest and Big Sky have stayed red, etc.

But then you've got the Rust Belt. There, Indiana has stayed consistently red, Pennylvania has stayed blue (yes, it was red in '48, but this is before the realignment started for the rest of the country so I assume that to be an outlier), Michigan has kinda sorta moved from red to blue, and Ohio has moved from red to purple. I can kind of understand Pennsylvania acting a little differently from the others since it's a border state, but does anyone know why that region acts in a much less unified manner than most of the other regions?