Predicting the Hugos: Thinking of 2016
It’s getting close to the first of the year, and I’ll have my first 2016 Hugo prediction up soon! Here at Chaos Horizon, we use the stats and data from previous years to predict what will happen going forward. Obviously, that’s a very specific methodology that won’t be to everyone’s taste.
Here’s how it works: I begin with the assumption that what will happen in the 2016 Hugos will follow the patterns of previous years, particularly 2015. Of course, every year is different, but this gives us a starting point to begin a prediction. This assumption is useful in some cases (the Warriors have gone 28-1 games in the NBA so far; should I predict the Warriors to win their next basketball game?) and not useful in other cases (let’s say I rolled three fours in a row with a pair of dice; should I predict the next roll to also be a 4?).
The tricky part for the 2016 Hugos is to make a decent estimate of how much impact the Sad/Rabid Puppies will have this year. Before Correia and Kloos withdrew, the Puppies took 4 out of the top 5 Novel spots in the 2015 Hugos. After the controversy surrounding the slates hit in 2015, there was a huge increase of Hugo voters (5,653 people voted in the Best Novel category, up from 3,137 in 2014). All 5,653 of these voters are eligible to vote in the 2016 Hugo nominations—but how many of them will? And what percentage will follow/be influenced by the Puppies?
I don’t think we’ll exactly know until the nomination stats are released next August, but what we can do is work on some sensible guesses.
First thing, how many people will nominate in 2016? We saw a voting increase between the 2014 and 2015 Final Hugo Ballot of 5653/3137 = 1.8X. If we apply that number to last year’s nomination number, we’d get a 1827 2015 nomination ballots * 1.8 = 3289 nomination ballots in the Best Novel category. The controversy and high emotions surrounding last year’s Hugo could drive that number even higher. Remember, though, that the nomination process doesn’t get near as much ink as the final ballot does.
Next, to predict the nominees for the 2016 Hugos, I’ll begin with some stats from 2015:
Best Novel Nominations 2015 Hugo (1,827 ballots)
387 Skin Game Jim Butcher 21.2%
372 Monster Hunter Nemesis Larry Correia 20.4% *
279 Ancillary Sword Ann Leckie 15.3%
270 Lines of Departure Marko Kloos 14.8% *
263 The Dark Between the Stars Kevin J. Anderson 14.4%
256 The Goblin Emperor Katherine Addison 14.0%
210 The Three Body Problem Liu Cixin 11.5%
199 Trial By Fire Charles E. Gannon 10.9%
196 The Chaplain’s War Brad Torgersen 10.7%
168 Lock In John Scalzi 9.2%
160 City of Stairs Robert Jackson Bennett 8.8%
141 The Martian Andy Weir 7.7%
126 Words of Radiance Brandon Sanderson 6.9%
120 My Real Children Jo Walton 6.6%
112 The Mirror Empire Kameron Hurley 6.1%
92 Lagoon Nnedi Okorafor 5.0%
88 Annihilation Jeff Vandemeer 4.8%
Correia and Kloos turned down their nominations. We need to be aware that something similar could happen again. Also note how close Addison was—she almost beat Anderson (7 votes).
Let’s transform that chart by taking out the author’s names and replacing them with either Sad/Rabid Overlap (appeared on both the Sad + Rabid slates), Sad No Overlap (appeared only on the Sad Puppy slate), Rabid No Overlap (appeared only on the Rabid Puppy slate), or Typical (did not appear on a slate). Here’s what you get:
Best Novel (1,827 ballots)
Spot #1: 387 Sad/Rabid Overlap #1 21.2%
Spot #2: 372 Sad/Rabid Overlap #2 20.4% *
Spot #3: 279 Typical #1 15.3%
Spot #4: 270 Sad/Rabid Overlap #3 14.8% *
Spot #5: 263 Sad/Rabid Overlap #4 14.4%
Spot #6: 256 Typical #2 14.0%
Spot #7: 210 Typical #3 11.5%
Spot #8: 199 Sad No Overlap #1 10.9%
Spot #9: 196 Rabid No Overlap #1 10.7%
Spot #10: 168 Typical #4 9.2%
Spot #11: 160 Typical #5 8.8%
Spot #12: 141 Typical #6 7.7%
Spot #13: 126 Typical #7 6.9%
Spot #14: 120 Typical #8 6.6%
Spot #15: 112 Typical #9 6.1%
Spot #16: 92 Typical #10 5.0%
Spot #17: 88 Typical #11 4.8%
This allows us to see the relative power of the picks. When the Sad and Rabid Puppies overlapped, they were able to generate more votes than anything but the most popular Typical pick. At the top, in Spot #1 and #2, they had a comfortable margin (100 votes). When the Sad and Rabid puppies separated, they fell behind Typical #1, #2, and #3. We can also see that the Sad/Rabid numbers fell off rapidly: Overlaps #1 and #2 generated more votes than less popular Overlaps #3 and #4.
So, if everything stayed the same, or the number of votes generated by the Sad Puppies and Rabid Puppies increased at the same rate as the Typical votes, you’d predict Sad/Rabid Overlap #1 and #2 to make the final ballot, with the most popular Typical #1 book also to make the ballot, and then a dogfight for Spots #4-#5.
But everything isn’t likely to stay the same. Sad Puppies IV is already putting together a crowd-sourced list; with more than 5 suggestions in the Novel category, that could very well dilute the vote across those nominations. I suspect we’ll see something similar to last year: works at the top of the list that are very popular, on the order of Butcher popular, will generate far more votes than less popular works lower down on the list.
We also have no idea whether what I’m calling the “Typical,” “Sad,” and “Rabid” votes will increase at the same rate. Other discernible blocks could also emerge, although you can’t vote against someone in the nomination stage, so nothing like an explicit anti-Puppy vote can occur without generating an opposing slate.
This could also create a situation where the Sad Puppies and the Typical votes overlap, like if the Sad Puppies picked Seveneves or Uprooted, books already strong Hugo contenders. I’ll take a look at the leaders in the Sad Puppy nominations tomorrow.
I think things will be very close in the lower spots. A surge of 100-200 voters in either direction can imagine it, and the kind of predictive work I do at Chaos Horizon is incapable of tracking things that finely, particularly when faced with major change.
So, here’s my initial thoughts, what I’m calling the Overlap Theory: Since the 2016 Hugos nominations are likely to draw such attention, the works that are most likely to get nominations are those that overlap in more than one of the Typical/Sad/Rabid categories. The power of overlapping will usually be more powerful than going alone. So here’s what my initial top of the Ballot might look like:
1. Typical/Sad Overlap #1
2. Sad/Rabid Overlap #1
3. Typical/Sad Overlap #2
4. Sad/Rabid Overlap #2
5. Typical No Overlap #1
6. Sad/Rabid Overlap #3
7. Sad/Rabid Overlap #4
8. Typical No Overlap #2
9. Typical No Overlap #3
10. Sad No Overlap #1
11. Rabid No Overlap #1
That’s assuming no major shifts in percentages in the relative group sizes from last year. If you’ve got any suggestions on how to calculate such shifts, let me know!
So, as we dive into another controversial year, what do you think? Do the 2015 stats provide any meaningful guidance for 2016, or will things be so dynamic/unpredictable that the past is no guide to the future? What impact do you think the Puppies will have on the 2016 nominations? How can we best model that impact here at Chaos Horizon?