2015 Hugo Analysis: Best Novel

The controversial 2015 Hugos have been given, and now it’s time to sift through the stats to see what we can learn. A couple preliminaries before we dive into the Best Novel category:

Here are the stats themselves: 2015HugoStatistics. I believe they are going to release the anonymous ballots at a later data so people can run simulations on them, but I don’t know when or how they’re going to do that.

If you’re not familiar with the tabulating process, this post from Staffer’s Book Review does a good job explaining the basics.

I know emotions are highly charged around the 2015 Hugos, but Chaos Horizon is a stat-driven site, and I try to hit the middle of the road in my estimates. This angers some commentators, who want me to minimize or maximize my estimates to bolster/attack a certain side. Chaos Horizon is not the frontline of the Hugo wars; we’re the room in the back where they do the autopsies. Feel free to question/interrogate the numbers as vigorously as you want, but keep the sniping and arguing about politics to a minimum.

Lastly, the analysis can only analyze behavior, not intent. If someone votes Butcher / Anderson one-two in the Best Novel, that’s behaving like a Sad Puppy. If you follow Vox Day’s suggestions, that’s behaving like a Rabid Puppy. If you vote all the Puppy picks below “No Award,” that’s behaving like a No Awarder. I can’t tell you why an individual voter did those things. I think we all know there’s a range of reasons someone might vote that way, and it’s never fun to get lumped into a group. That’s what voting does, though; it turns the individual into a list of numbers on the page. We’ll never be able to precisely “know” the reasons behind these numbers; the best we can do is have some rough estimates. Depending on your tolerance, you may want to add a “haze” around each of my estimates of 10-25%.

So, let’s get started. Here are my rough estimates from earlier in the week:

Core Rabid Puppies: 550-525
Core Sad Puppies: 500-400
Neutrals: 1400 (voted some Puppies, but not all)
Primarily No Awarders But Considered a Puppy Pick: 1000
Absolute No Awarders: 2500

That “Neutrals” is the most unsatisfying. Let’s see if we cant refine that today.

So let’s look at the initial pass of the Best Novel Results in the Race for Position 1:
1691: Three-Body Problem
1515: Goblin Emperor
1054: Ancillary Sword
874: Skin Game
268: No Award
251: Dark Between the Stars

5653 total votes. A couple immediate facts: 5950 voted for the Hugos in total, meaning 300 people sat out the Best Novel category. 95% of voters voted here. Around 3% voted No Award over everything, which is in-line with past years. That means 8% checked out of this category totally, or around 550 voters. Those are the true neutrals, I guess.

Vox Day, leader of the Rabid Puppies, suggested Three-Body Problem for first place. Is that number 525? Let’s confirm right now. Since Three-Body Problem won, it was eliminated immediately for the race for position #2. That means it was crossed off the ballot, and all of those first place votes went to the second name on the list. Since VD suggest Butcher, we should see an increase of around 500.

What do we see? 1264 Butcher votes. Subtract the first pass Butcher total 874, and we get 390 votes. That’s a little lower than expected, but not horribly so. It may be that not all the RP followed VD’s first place suggestion, or that some didn’t follow his second place selection. Anderson picked up (314-251) 63 votes from the Cixin Liu voters, which puts us back to around 450 Puppies who voted for Liu in round #1, then moved to a Puppy pick in Round #2.

So I think we can estimate that least 450 Puppy voters voted for Liu #1 in the Best Novel category. Since that novel beat Goblin Emperor by 200 for Best Novel, we can reaffirm that it was the Puppy voters who drove Liu to the win.

Now, let’s estimate the total number of Puppy voters. 874 voted for Skin Game #1, followed by 251 for Dark Between the Stars. That totals 1125, which we can think of as a rough estimate of the “maximum Sad Puppy vote,” or the number of people who could look past the controversy to vote a Sad Puppy pick #1. That group will consist of core Sad Puppies (those who voted all the Puppy picks above No Award) and neutrals who liked Anderson/Butcher better than any of the non-Puppy picks. Remember this 1125 doesn’t include the 450 Rabid Puppies who voted for Liu #1.

So we can say that we have a maximum Puppy vote of 1575, including Rabid Puppies, Sad Puppies, and Puppy-leaning neutrals. I think we can confirm that number by looking at some other categories. Take the max number of people in the swept categories (Novella, Short Story, etc.), who had a preference above “No Award.” I find that by looking at the Race for Position #5. Those numbers are 1476 (Novella), 1885 (Short Story), 1527 (Editor Short), 1769 (Editor Long), 1624 (Best Related Work). While there is some variation in those categories, I think 1600 looks like a good central estimate, perhaps with a +/- 200 around it. Again, these are rough, but I think they better reflect the data than my initial estimate.

So what does that mean? It means I should refine my neutral number:

Core Rabid Puppies: 550-525
Core Sad Puppies: 500-400
Sad Puppy leaning Neutrals: 800-400 (capable of voting a Puppy pick #1)
True Neutrals: 1000-600 (may have voted one or two Puppies; didn’t vote in all categories; No Awarded all picks, Puppy and Non-Alike)
Primarily No Awarders But Considered a Puppy Pick above No Award: 1000
Absolute No Awarders: 2500

That looks like a slightly more reasonable and refined range.

So let’s look at what happened after the first pass. Dark Between the Stars was eliminated, yielding the following numbers:
1727 (+36): Three-Body Problem
1544 (+29): Goblin Emperor
1082 (+28): Ancillary Sword
1004 (+130): Skin Game
271 (+3): No Award

The + number in parenthesis is the number of votes added in this pass.

Roughly half of KJA’s vote went to Butcher, and the other half equally split to the three other candidates. I think that’s a great indication that half the Anderson vote was core Sad Puppy, half was neutral.

No Award was eliminated next.

1758 (+31): Three-Body Problem
1565 (+21): Goblin Emperor
1117 (+35): Ancillary Sword
1013 (+9): Skin Game

These are voters who had a ballot that had No Award and then some works listed after that. A lot of these awarders (175) went to No Preference, indicating they didn’t have anything listed after No Award.

No it gets interesting. Skin Game gets cut next, and those 1013 votes are redistributed. Let me add No Preference in as well

2162 (+404): Three-Body Problem
1814 (+249): Goblin Emperor
1240 (+123): Ancillary Sword
437 (+237): No Preference

We had a huge surge to Three-Body Problem from the Skin Game voters, with half of those (the core Sad Puppies) choosing Liu over Leckie or Addison. Don’t forget those 237 who checked out completely, moving from Preference to No-Preference. That’s a pretty compelling stat: 641 Butcher voters wanted Liu or nobody. If you’re looking for a core number on the Puppy side who were “protesting” the more mainstream Hugo picks, is this it?

Next, Leckie goes by the wayside. If my analysis is correct, more voters should go to Addison than Liu.

2649 (+487): Three-Body Problem
2449 (+635): Goblin Emperor
555 (+118): No Preference

That’s the first time Addison has picked up more votes in a round than Liu. This lets us really see how different the tastes are of the Leckie supporters than the Butcher supporters. If those 400+ Rabid Puppy votes weren’t bolstering Liu’s totals, Addison would have won.

We don’t get as much data from the other passes because an author quickly gathered more than 50% of the vote, making a second pass unnecessary. From the Round #2, I can tell you that of The Dark Between the Stars 314 votes, more than half (169) went over to Butcher when he was eliminated. When Skin Games 1448 votes are eliminated, a massive 860 votes go to Goblin Emperor and a mere 226 to Leckie. 860-226 is more than 600, and that might be the Vox Day Rabid Puppy block in action.

In the Race for Position #4, No Award grabbed 2674 votes (very close to my core No Award vote of 2500), with Skin game at 2000 and Dark at 592. That means Butcher had a theoretical max of 2592 votes (if everyone voted had Butcher below Anderson on those ballots). That’s the closest a Puppy pick outside of Drama got to not getting No Awarded. That’s pretty close!

If we go back to my stats, does it work? 2500 No Awarders, 1800-1300 Rabid, Sad, and Puppy-leaners, plus around another 2000-1500 neutrals, some of whom sat out this round? If we subtract the 700 who didn’t have an opinion in this round from the neutrals, that leaves us with 1300-800. Did Butcher pick up close to 700 votes from that group?

It’s unusual to No Award a Novel in the Hugos. Even last year, Correia squeaked by, with a vote of 1161 to 1052, and Butcher is an order of magnitude more famous/well-liked than Correia. The vote here was a potential 2674-2592 against Butcher, meaning that the Anti-Puppy picked up (2674-1052) 1622 votes from 2014 to 2015, and the Puppy vote 1431 votes. That’s an almost 50/50 split in who was added this year. Of course, there are a lot of variables, including Butcher’s massive popularity—and we’re not talking about Butcher wining, put simply placing above No Award.

Still, a lot of numbers to chew on. Everyone take a look at the Best Novel stats, chew on them, and let’s see what else we can learn!