Silver and Wang Electoral Predictions: Genius or Glorified Grunt Work?

percy_xmas copy (1)

Percy the Cat, Elite Forecaster

Nate Silver and Sam Wang (S&W) have gained much acclaim for predicting election outcomes. Am happy for them. They are talented academics and scientists. But there is considerably less to their electoral prediction feats than meets the eye.

First of all, S&W are not really doing high level, sophisticated social science forecasting, as the media thinks.  Their work is measurement more than forecasting; description more than explanation; and involves polling expertise far more than political insight.

After digging into the minutiae of S&W’s methods, the familiar Edison quote about “genius” came to mind: “Genius,” Thomas Edison said, “is 1% inspiration and 99% perspiration.”  S&W are well trained statisticians, but their methods are not anything you wouldn’t find in a decent  graduate level stat textbook; and it would be mostly in the less daunting, descriptive stat chapters.

What separates S&W from the crowd is the sweat labor they employed gathering polling data from every village and corner of the country.  They amass these polls, well over 4000 separate ones, identify their strengths and weaknesses, and “aggregate” them in ways to get a better picture of the electorate a few weeks or days ahead of the election.

Yes, some of their “weighting” techniques for combining polls is creative, but little that CNN’s John King or your average R or D, cable news station “political strategist” wouldn’t immediately understand.  What’s different about S&W, is that other pundits didn’t have the fortitude (or wherewithal) to locate thousands of polls, capture their results, wring proprietary information about them from a few pollsters, and record all of this information in a (relatively) simple stat program or spreadsheet.

Bravo!  S&W deserve much credit for that.  We’d be shorting S&W by saying they (and now their worker bees) are doing no more than glorified grunt work;  but neither are their predictions divinely inspired or the work of genius.

To begin with, the feat of calling all (or nearly all) state presidential or senatorial elections correctly, is not as difficult as it sounds. That is especially true in today’s highly polarized electoral map. As I sarcastically said to a friend the other day, “my cat Percy could predict which party will win/hold about 80 of the 100 Senate seats in November..…simply because that’s how many seats are either SAFE or NOT UP FOR ELECTION. (Only one third of the Senate is up for election every two years).   Any barely competent, non partisan electoral handicapper would agree with that.

There are about 20 states among those with Senate elections in November where the election of a D or R would not be considered a black swan event; they are  theoretically in play,    But of those 20, about 10 are considered to LEAN STRONGLY in one direction or another. See for example Real Clear Politics, Politico or the Cook Report.

Thus, remaining in sarcastic mode, a Sea World Dolphin, being smarter than my Percy the Cat (though not treated as humanely), might do well (if not perfectly) predicting which party will own about 90 of the 100 Senate seats after the November 4th election.  The mainstream, non partisan electoral map handicappers widely agree the D/R split on those 90 seats will be between 44/46 and 46/44 on election night.

Now we’re in the home stretch, down to about 10 toss up states. There isn’t perfect agreement on the specific composition of the battle ground seats, but all the mainstream handicappers say there are 9 or 10 states where the election could go either way.  I’m going with 10 in this little exercise. (It’s a nice round number),

It’s of course a lot harder from here on. But, we’re dealing now with about 10 difficult calls; not struggling with 100 races to predict, as the fawning media makes it sound when they say Silver or Wang miraculously called the D/R split in the Senate,  or the 50 state electoral college outcomes perfectly.

Based on a scan of Real Clear Politics, Politico, Cook Report, and Rasmussen, it is a sure bet the D/R split of the remaining 10 toss up states will be somewhere in the
7-3 to 3-7 range.  Las Vegas odds makers would be exceedingly comfortable with that; probably also with 6-4 to 4-6, but I won’t push it.

Thus, without barely lifting a finger, we can confidently say the range of plausible outcomes is between 53/47 and 47/53.  Not between 0/100 and 100/0, as you might infer from the gushing and awestruck Rachel Maddow (who knows better) or Wolf Blitzer (who may not).  Most people reading this blog, probably already know this.  But more regular folks don’t, which is not a failing.

S&W’s celebrity status could wane soon, because their approach is prone to being upended by what social scientists call the Hawthorne Effect: Just the awareness of being closely watched (by a researcher) changes the subjects” behavior.  So, don’t you think all those obscure pollsters – the 3,980 other than prominent ones – are changing how they poll, now that they’ve been subject to S&W ‘s proctology exam?

S&W are aware of all that, and trying to stay one step ahead. That’s probably why Silver moved to ESPN (from the NY Times), where he can hedge bets on his career, regressing to measuring baseball performance, or embracing college basketball bracketology.   He won’t do nearly as well there because you have to really know something about forecasting and  basketball to predict the final four.


6 thoughts on “Silver and Wang Electoral Predictions: Genius or Glorified Grunt Work?

  1. Alex MacLachlan

    It sounds like S&W have decided on quality of life and enjoyment of work, admiring the accomplishments of those whose outcomes they cover/predict. I think many of us would do the same thing if given the choice between politics and sports


    1. Irv Lefberg Post author

      Yeah, I’m admittedly a little jealous of Silver. Would love to have made millions doing my hobbies, which at one time was (also) Sabrmetric baseball stats, and later on political forecasting, More power to him!


  2. Harold Nelson

    Hi Irv,

    I don’t know if you’ve read Silver’s Signal and Noise or not, but you seem to be echoing his own description of what he does and recommends. It’s all in doing simple things relentlessly. There are no magic bullets in academically sophisticated statistical methods. I tend to think like Silver myself.


    1. Irv Lefberg Post author

      I understand, Harold. What you and your all star SAS team did for years with medicaid was brilliant, and elegantly simple. And relentless. Same with Darthmouth Atlas. I have a bias of sorts toward explanatory modeling and forecasting as being more creative. Maybe its not!!! I may be just be a bit jealous of Silver, 🙂


  3. pollins1

    Good points! No deep magic in S&W. Yet Gallup + many other big-dollar Republican pollsters looked much the same data in 2012 and made wildly incorrect forecasts that November, thus misallocating last-minute Romney campaign funds in the process (I know, we’re all so disappointed). Even better (for me) Vegas drank the Karl Rove Kool-Aid and made their lines based on Rove’s predictions. My point is that it is not as easy as it looks to make predictions in the toss-up races. You have to have a way of figuring out who is actually going to show up at the polls, since no one brings ballot boxes to the voters the way that pollsters phone or knock on doors to ask people about their preferences. APSA’s Political Forecasting Group has figured out how to do this, and Silver has poached their techniques (without attribution). And I doubt that the Republicans (and Vegas) will repeat the costly forecasting errors of 2012.


    1. Irv Lefberg Post author

      Thanks, Brian. I really need to go back to 2012 and check out the polls and punditry in the days and weeks leading up to election day. I of course recall the R pollsters and pundits missing it by a lot — I think, due as much to not being able to bear the thought of another Obama victory and living in their echo chamber, as to methods not as thorough or as smart as Silver’s. But, I don;t recall the Gallup big miss. (I of course believe you that it happened), I can recall following the polls in the “battle ground”/toss-up states on the Real Clear Politics website, and also PPP, I thought based just on those (a small subset of Silver’s universe) one could discern a likely electoral college Obama advantage. It appeared a lot closer from the national polling numbers, especially after Obama royally screwed up the first debate with Romney. .As I said to Harold Nelson in my reply here, I have a bias toward explanatory modeling and forecasting as being more creative (sexy, to use a technical term) than (even) the smart and diligent type of work Silver does (which i still don;t think of as “forecasting”). . I may be just be a bit jealous of Silver. I mean, starring in both Sabrmetrics AND political stats! Yikes.

      I really look forward to learning about the APSA forecasting group work. Sure hope the Pennsylvania polls are solid!! 🙂



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s