Nate Silver and Sam Wang (S&W) have gained much acclaim for predicting election outcomes. Am happy for them. They are talented academics and scientists. But there is considerably less to their electoral prediction feats than meets the eye.
First of all, S&W are not really doing high level, sophisticated social science forecasting, as the media thinks. Their work is measurement more than forecasting; description more than explanation; and involves polling expertise far more than political insight.
After digging into the minutiae of S&W’s methods, the familiar Edison quote about “genius” came to mind: “Genius,” Thomas Edison said, “is 1% inspiration and 99% perspiration.” S&W are well trained statisticians, but their methods are not anything you wouldn’t find in a decent graduate level stat textbook; and it would be mostly in the less daunting, descriptive stat chapters.
What separates S&W from the crowd is the sweat labor they employed gathering polling data from every village and corner of the country. They amass these polls, well over 4000 separate ones, identify their strengths and weaknesses, and “aggregate” them in ways to get a better picture of the electorate a few weeks or days ahead of the election.
Yes, some of their “weighting” techniques for combining polls is creative, but little that CNN’s John King or your average R or D, cable news station “political strategist” wouldn’t immediately understand. What’s different about S&W, is that other pundits didn’t have the fortitude (or wherewithal) to locate thousands of polls, capture their results, wring proprietary information about them from a few pollsters, and record all of this information in a (relatively) simple stat program or spreadsheet.
Bravo! S&W deserve much credit for that. We’d be shorting S&W by saying they (and now their worker bees) are doing no more than glorified grunt work; but neither are their predictions divinely inspired or the work of genius.
To begin with, the feat of calling all (or nearly all) state presidential or senatorial elections correctly, is not as difficult as it sounds. That is especially true in today’s highly polarized electoral map. As I sarcastically said to a friend the other day, “my cat Percy could predict which party will win/hold about 80 of the 100 Senate seats in November..…simply because that’s how many seats are either SAFE or NOT UP FOR ELECTION. (Only one third of the Senate is up for election every two years). Any barely competent, non partisan electoral handicapper would agree with that.
There are about 20 states among those with Senate elections in November where the election of a D or R would not be considered a black swan event; they are theoretically in play, But of those 20, about 10 are considered to LEAN STRONGLY in one direction or another. See for example Real Clear Politics, Politico or the Cook Report.
Thus, remaining in sarcastic mode, a Sea World Dolphin, being smarter than my Percy the Cat (though not treated as humanely), might do well (if not perfectly) predicting which party will own about 90 of the 100 Senate seats after the November 4th election. The mainstream, non partisan electoral map handicappers widely agree the D/R split on those 90 seats will be between 44/46 and 46/44 on election night.
Now we’re in the home stretch, down to about 10 toss up states. There isn’t perfect agreement on the specific composition of the battle ground seats, but all the mainstream handicappers say there are 9 or 10 states where the election could go either way. I’m going with 10 in this little exercise. (It’s a nice round number),
It’s of course a lot harder from here on. But, we’re dealing now with about 10 difficult calls; not struggling with 100 races to predict, as the fawning media makes it sound when they say Silver or Wang miraculously called the D/R split in the Senate, or the 50 state electoral college outcomes perfectly.
Based on a scan of Real Clear Politics, Politico, Cook Report, and Rasmussen, it is a sure bet the D/R split of the remaining 10 toss up states will be somewhere in the
7-3 to 3-7 range. Las Vegas odds makers would be exceedingly comfortable with that; probably also with 6-4 to 4-6, but I won’t push it.
Thus, without barely lifting a finger, we can confidently say the range of plausible outcomes is between 53/47 and 47/53. Not between 0/100 and 100/0, as you might infer from the gushing and awestruck Rachel Maddow (who knows better) or Wolf Blitzer (who may not). Most people reading this blog, probably already know this. But more regular folks don’t, which is not a failing.
S&W’s celebrity status could wane soon, because their approach is prone to being upended by what social scientists call the Hawthorne Effect: Just the awareness of being closely watched (by a researcher) changes the subjects” behavior. So, don’t you think all those obscure pollsters – the 3,980 other than prominent ones – are changing how they poll, now that they’ve been subject to S&W ‘s proctology exam?
S&W are aware of all that, and trying to stay one step ahead. That’s probably why Silver moved to ESPN (from the NY Times), where he can hedge bets on his career, regressing to measuring baseball performance, or embracing college basketball bracketology. He won’t do nearly as well there because you have to really know something about forecasting and basketball to predict the final four.