Estimating the 2016 Hugo Nominations, Part 3
This is where things get messy—perhaps to the point of incoherence. I estimated 5,000 voters in last years Hugos who seemed to not be associated with the Sad or Rabid Puppies: some Hugo voters of past years, some who joined to vote No Award for the Puppies, some who joined just to vote, some who maybe joined to participate in the controversy, and some who joined for unknown reasons. We don’t have much past data on this group, so how can we calculate how they’re likely to vote in 2016?
To be honest, we probably can’t, not with any definite certainty. What I want to achieve is just getting in the ballpark, of producing a low and high estimate that we can compare to the Rabid Puppies estimate. That’ll at least tell us something. You know me: I always like numbers, no matter how rough they are.
What we do have is data from past Hugo years, and if we make the assumption that voting patterns won’t be wildly different from previous years, we’d at least have a place to start. So, first thing I’ll do is take a look at previous voting percentages in the Hugos. This chart shows what percentage of the Vote the #1, #2, etc. novel receives in a typical Hugo year. For these purposes, I average the voting patterns from 2010-2013 (the 4 prior years with no Puppy influence, drawn from the Hugo voting packets).
Table 1: Voting Percentages in Hugo Best Novel Category
This means that the most popular Hugo book, in any given year, gets around 17% of the vote, with a range of 15%-20% in the years I looked at. So if 4,000 people vote in 2016, we might estimate the top book as getting between 600-800 votes. If 3,000 people vote, that drops us down to an estimate of 450-600 votes.
Does this estimate tell us anything, or is it just useless fantasizing? I can see people arguing either way. What this does is narrow the range down to something somewhat sensible. We’re not predicting Ann Leckie is going to get 2000 votes for Best Novel. We’re not predicting she’s going to get 100. I could predict 450-800 and then match that against the 220-440 Rabid Puppies prediction. That would tell me Leckie seems like a likely nominee.
We can go destroy this prediction if we make different assumptions. I could assume that the new voters to the Hugos won’t vote in anything like typical patterns, i.e. that they are complete unknowns. Maybe they’ll vote Leckie at a 75% rate. Maybe they’ll vote her 0%. Those extremes grate against my thought patterns. If you know Chaos Horizon, I tend to chose something in the middle based on last year’s data. That’s a predictive choice I make; you might want to make other ones.
I believe voting patterns will be closer to traditional patterns than totally different from them. You may believe otherwise, and then you’ll need to come up with your own estimate. If I’m off, how far off am I, and in what direction? Too low by 100-200 votes? Too high by 100-200 votes? And if I’m off by only that much, is the outcome of this prediction affected?
So, this says . . . if these 5,000 vote along similar lines to past Hugo voters, and we imagine three turnout scenarios, where do we end up?
Let’s not drop in book titles yet, let’s just multiply table #1 by three different turnout scenarios for our 5,000 2015 voters (40%, 60%, and 80%):
Table 2: Estimated Votes in 2016 Hugo Best Novel Category Based on Prior Voting Patterns
|# of Typical Voters||2000||3000||4000|
What I’m interested is whether or not these numbers beat the Rabid Puppy numbers from last post. Even if we assume robust Rabid Puppy turnout of creating 440 votes, we have 1 novel above that in the 60% scenario and 3 in the 80% scenario. Even if you pump the Rabid Puppy number up to 500, we still have at least 1 novel above in both the 60% and 80% scenario. If we lower the Rabid Puppy vote to a more modest 400, we wind up with 2 in the 60% and 4 in the 80%. This is the piece of data I want: in most turnout scenarios, a few “typical” books beat the Rabid Puppies, but not all. I’d estimate we’re in store for a mixed ballot. Only in very modest turnout scenarios (40%) do the Rabid Puppies sweep Best novel.
We do need to factor the Sad Puppies in (next post), but the numbers don’t suggest to me that the Rabid Puppies will manage 5 picks on the final Hugo ballot. The numbers also don’t suggest that there won’t be 0 Rabid Puppy picks.
Two questions remain for this post: what will the turnout be? I don’t know how many of those 5,000 will vote. I know passions are high so I assume the turnout will be high. Then again, this stage is difficult to vote in: you have to read a bunch of novels, remember when the ballot is due, realize you’re eligible to nominate, etc. To my eye, somewhere between 50-75% seems about right. I’m going to pick a conservative 60% just to have something to work with. Since I included all three bands, you’re free to pick anything according to your tastes and your own sense of what will happen.
Next, what is Novel #1 going to be? Novel #2? Novel #3? I’m just going to utilize my SFF Best of Critics list to drop in here. I definitely think the top of the list (Leckie, Jemisin, Novik) is what this group will be voting for. I’ll preserve ties, so if we go to the Top 12 contenders I’m tracking for this prediction, here’s where they show up:
1. Ancillary Mercy (estimate 17.73% of vote)
1. Uprooted (estimate 17.73% of vote)
3. The Fifth Season (estimate 13.06% of vote)
3. Aurora (estimate 13.06% of vote)
10. Seveneves (estimate 6.54% of vote)
Golden Son and The Aeronaut’s Windlass did show up on the list, so I’m going to give them a 1% of the vote. Something but not a lot. This is in line with Butcher’s past totals from before the Sad Puppies; he was only getting a few votes based on 2009 Nomination Data, where WorldCon showed us how many votes everyone got.
Could those be off significantly? Sure they could. That’s why it’s an estimate! Multiply those out in the scenarios, and you get this chart:
|The Fifth Season||261||392||522|
|The Aeronaut’s Windlass||20||30||40|
|Agent of the Imperium|
|Honor At Stake|
|A Long Time Until Now|
Leckie and Novik get lots of votes, probably enough to beat the Rabid Puppies without any help. Given Leckie pulled in 15.3% of the vote in 2015 and 23.1% in 2014, wouldn’t her vote percentage be somewhere in that ballpark for 2016? The average of those two is 18.15%, and I predicted her at 17.73% in the chart above. Estimating is a different act than logical proof, and one that is ultimately settled by the event—come a few weeks, we’ll known the ballot, and I’ll either be right or wrong. Jemisin and Robinson get votes and will be competitive. Stephenson is down lower, but he also appears on the Rabid and Sad lists, so that jumps him over Jemisin and Robinson’s total and onto the ballot.
I know this will be the most disliked of the predictions. You know my theory at Chaos Horizon: any estimate gives you a place to start thinking. Even if you vehemently disagree with my logic, you now have something to contest. What makes more sense? If you apply those estimates, what ballot do you come up with? So argue away!
Tomorrow the Sad Puppies, and then we combine the three charts to get a bunch of different scenarios. If we see patterns across those scenarios, that’s the prediction.