2015 Hugo Analysis: Best Novel

The controversial 2015 Hugos have been given, and now it’s time to sift through the stats to see what we can learn. A couple preliminaries before we dive into the Best Novel category:

Here are the stats themselves: 2015HugoStatistics. I believe they are going to release the anonymous ballots at a later data so people can run simulations on them, but I don’t know when or how they’re going to do that.

If you’re not familiar with the tabulating process, this post from Staffer’s Book Review does a good job explaining the basics.

I know emotions are highly charged around the 2015 Hugos, but Chaos Horizon is a stat-driven site, and I try to hit the middle of the road in my estimates. This angers some commentators, who want me to minimize or maximize my estimates to bolster/attack a certain side. Chaos Horizon is not the frontline of the Hugo wars; we’re the room in the back where they do the autopsies. Feel free to question/interrogate the numbers as vigorously as you want, but keep the sniping and arguing about politics to a minimum.

Lastly, the analysis can only analyze behavior, not intent. If someone votes Butcher / Anderson one-two in the Best Novel, that’s behaving like a Sad Puppy. If you follow Vox Day’s suggestions, that’s behaving like a Rabid Puppy. If you vote all the Puppy picks below “No Award,” that’s behaving like a No Awarder. I can’t tell you why an individual voter did those things. I think we all know there’s a range of reasons someone might vote that way, and it’s never fun to get lumped into a group. That’s what voting does, though; it turns the individual into a list of numbers on the page. We’ll never be able to precisely “know” the reasons behind these numbers; the best we can do is have some rough estimates. Depending on your tolerance, you may want to add a “haze” around each of my estimates of 10-25%.

So, let’s get started. Here are my rough estimates from earlier in the week:

Core Rabid Puppies: 550-525
Core Sad Puppies: 500-400
Neutrals: 1400 (voted some Puppies, but not all)
Primarily No Awarders But Considered a Puppy Pick: 1000
Absolute No Awarders: 2500

That “Neutrals” is the most unsatisfying. Let’s see if we cant refine that today.

So let’s look at the initial pass of the Best Novel Results in the Race for Position 1:
1691: Three-Body Problem
1515: Goblin Emperor
1054: Ancillary Sword
874: Skin Game
268: No Award
251: Dark Between the Stars

5653 total votes. A couple immediate facts: 5950 voted for the Hugos in total, meaning 300 people sat out the Best Novel category. 95% of voters voted here. Around 3% voted No Award over everything, which is in-line with past years. That means 8% checked out of this category totally, or around 550 voters. Those are the true neutrals, I guess.

Vox Day, leader of the Rabid Puppies, suggested Three-Body Problem for first place. Is that number 525? Let’s confirm right now. Since Three-Body Problem won, it was eliminated immediately for the race for position #2. That means it was crossed off the ballot, and all of those first place votes went to the second name on the list. Since VD suggest Butcher, we should see an increase of around 500.

What do we see? 1264 Butcher votes. Subtract the first pass Butcher total 874, and we get 390 votes. That’s a little lower than expected, but not horribly so. It may be that not all the RP followed VD’s first place suggestion, or that some didn’t follow his second place selection. Anderson picked up (314-251) 63 votes from the Cixin Liu voters, which puts us back to around 450 Puppies who voted for Liu in round #1, then moved to a Puppy pick in Round #2.

So I think we can estimate that least 450 Puppy voters voted for Liu #1 in the Best Novel category. Since that novel beat Goblin Emperor by 200 for Best Novel, we can reaffirm that it was the Puppy voters who drove Liu to the win.

Now, let’s estimate the total number of Puppy voters. 874 voted for Skin Game #1, followed by 251 for Dark Between the Stars. That totals 1125, which we can think of as a rough estimate of the “maximum Sad Puppy vote,” or the number of people who could look past the controversy to vote a Sad Puppy pick #1. That group will consist of core Sad Puppies (those who voted all the Puppy picks above No Award) and neutrals who liked Anderson/Butcher better than any of the non-Puppy picks. Remember this 1125 doesn’t include the 450 Rabid Puppies who voted for Liu #1.

So we can say that we have a maximum Puppy vote of 1575, including Rabid Puppies, Sad Puppies, and Puppy-leaning neutrals. I think we can confirm that number by looking at some other categories. Take the max number of people in the swept categories (Novella, Short Story, etc.), who had a preference above “No Award.” I find that by looking at the Race for Position #5. Those numbers are 1476 (Novella), 1885 (Short Story), 1527 (Editor Short), 1769 (Editor Long), 1624 (Best Related Work). While there is some variation in those categories, I think 1600 looks like a good central estimate, perhaps with a +/- 200 around it. Again, these are rough, but I think they better reflect the data than my initial estimate.

So what does that mean? It means I should refine my neutral number:

Core Rabid Puppies: 550-525
Core Sad Puppies: 500-400
Sad Puppy leaning Neutrals: 800-400 (capable of voting a Puppy pick #1)
True Neutrals: 1000-600 (may have voted one or two Puppies; didn’t vote in all categories; No Awarded all picks, Puppy and Non-Alike)
Primarily No Awarders But Considered a Puppy Pick above No Award: 1000
Absolute No Awarders: 2500

That looks like a slightly more reasonable and refined range.

So let’s look at what happened after the first pass. Dark Between the Stars was eliminated, yielding the following numbers:
1727 (+36): Three-Body Problem
1544 (+29): Goblin Emperor
1082 (+28): Ancillary Sword
1004 (+130): Skin Game
271 (+3): No Award

The + number in parenthesis is the number of votes added in this pass.

Roughly half of KJA’s vote went to Butcher, and the other half equally split to the three other candidates. I think that’s a great indication that half the Anderson vote was core Sad Puppy, half was neutral.

No Award was eliminated next.

1758 (+31): Three-Body Problem
1565 (+21): Goblin Emperor
1117 (+35): Ancillary Sword
1013 (+9): Skin Game

These are voters who had a ballot that had No Award and then some works listed after that. A lot of these awarders (175) went to No Preference, indicating they didn’t have anything listed after No Award.

No it gets interesting. Skin Game gets cut next, and those 1013 votes are redistributed. Let me add No Preference in as well

2162 (+404): Three-Body Problem
1814 (+249): Goblin Emperor
1240 (+123): Ancillary Sword
437 (+237): No Preference

We had a huge surge to Three-Body Problem from the Skin Game voters, with half of those (the core Sad Puppies) choosing Liu over Leckie or Addison. Don’t forget those 237 who checked out completely, moving from Preference to No-Preference. That’s a pretty compelling stat: 641 Butcher voters wanted Liu or nobody. If you’re looking for a core number on the Puppy side who were “protesting” the more mainstream Hugo picks, is this it?

Next, Leckie goes by the wayside. If my analysis is correct, more voters should go to Addison than Liu.

2649 (+487): Three-Body Problem
2449 (+635): Goblin Emperor
555 (+118): No Preference

That’s the first time Addison has picked up more votes in a round than Liu. This lets us really see how different the tastes are of the Leckie supporters than the Butcher supporters. If those 400+ Rabid Puppy votes weren’t bolstering Liu’s totals, Addison would have won.

We don’t get as much data from the other passes because an author quickly gathered more than 50% of the vote, making a second pass unnecessary. From the Round #2, I can tell you that of The Dark Between the Stars 314 votes, more than half (169) went over to Butcher when he was eliminated. When Skin Games 1448 votes are eliminated, a massive 860 votes go to Goblin Emperor and a mere 226 to Leckie. 860-226 is more than 600, and that might be the Vox Day Rabid Puppy block in action.

In the Race for Position #4, No Award grabbed 2674 votes (very close to my core No Award vote of 2500), with Skin game at 2000 and Dark at 592. That means Butcher had a theoretical max of 2592 votes (if everyone voted had Butcher below Anderson on those ballots). That’s the closest a Puppy pick outside of Drama got to not getting No Awarded. That’s pretty close!

If we go back to my stats, does it work? 2500 No Awarders, 1800-1300 Rabid, Sad, and Puppy-leaners, plus around another 2000-1500 neutrals, some of whom sat out this round? If we subtract the 700 who didn’t have an opinion in this round from the neutrals, that leaves us with 1300-800. Did Butcher pick up close to 700 votes from that group?

It’s unusual to No Award a Novel in the Hugos. Even last year, Correia squeaked by, with a vote of 1161 to 1052, and Butcher is an order of magnitude more famous/well-liked than Correia. The vote here was a potential 2674-2592 against Butcher, meaning that the Anti-Puppy picked up (2674-1052) 1622 votes from 2014 to 2015, and the Puppy vote 1431 votes. That’s an almost 50/50 split in who was added this year. Of course, there are a lot of variables, including Butcher’s massive popularity—and we’re not talking about Butcher wining, put simply placing above No Award.

Still, a lot of numbers to chew on. Everyone take a look at the Best Novel stats, chew on them, and let’s see what else we can learn!


15 responses to “2015 Hugo Analysis: Best Novel”

  1. davidelang says :

    I think it’s notable that after no award won, 2149 votes went to no preference.

    so there were 2536 people who were unwilling to vote for either Butcher or Anderson. Again very close to your 2500 core No Award vote estimate

  2. Peter O says :

    Thanks for the analysis and breakdowns.

    I was wondering something, though it may need to wait unitl some of the other categories are looked at, but did you find that the non NOAWARD and non-RP votes dropped off in the successive categories the way you talked about the nominating votes tend to do?

    • chaoshorizon says :

      I haven’t looked at it yet. That would be interesting, though, to subtract the 2500 No Award and the 500 RP and see what the participation curve was.

      • Peter O says :

        Did a quick and dirty graph last night, and preliminary looks are that the down-ballot fiction works were all relative to each other in the same manner that the nominations were. But the drop off from novel to the downballot was much less than in nominations (10% VS 30%)
        The editors were voted at an higher % than nominations. The fan-zine/artist/cast/etc were all over the compared to the nomination %.

  3. camestrosfelapton says :

    This is good stuff.

    The neutrals are tricky to identify in the numbers. I am assuming that people voting in a manner similar to how GRRM described his approach (i.e. trying to avoid No Award despite antipathy to the Puppies) will have often abstained or only entered some preferences. Additionally their choices are less forced (i.e. in any one category they will be voting along with other groups).

    • chaoshorizon says :

      I’m going to take a close look at the Novella category tomorrow or Friday, depending on when I have time. I think we should be able to see this by calculating how many people move from “Preference” to “No Preference” in each stage. That will show how man people voted for all Puppy picks, and how many people voted for just some. I think the Neutrals are those who voted for just some.

      • camestrosfelapton says :

        The drop off in total votes is 600 there also. To be expected but I assume many neutrals wold simply just not vote for that category given it’s overall weakness.

  4. jccarlton says :

    Reblogged this on The Arts Mechanical and commented:
    The problem with statistical anomalies is that they stick out like a sore thumb. where did those 2000 block votes for all “no award” come from.

  5. Avery says :

    Interesting analysis.

    Totally nominal data and not in the statistics to consider. I read a lot of blogs and comments both on hard core pro-puppy and anti-puppy sites. Almost everyone spoke favorably of the Goblin Emperor. I read almost nothing that strongly disliked Goblin Emperor. Admittedly, I did not read rabid puppy sites.

    This is nominal, but I think so analytically and I believe that a systematic content analysis of Novel reviews and postings would reveal little antipathy to Goblin Emperor. Your analysis backs my opinion – but I’m sure that many alternative explanations are available.

    • Craig says :

      A number of comments here and there (including my own) referred to GE as boring; there was a definite stylistic clash of taste where I could see what the author was going for, see she was doing it very well, and had a complete lack of interest in it or any future stories by the author because i have no interest in that style of story.

      But there were also a number of SP supporters who avowed that they put it first and loved it to death. Different folk like different things.

  6. Cara the Cat says :

    Thanks for all this analysis! I’ve been eyeballing the numbers for days, but it’s nice to see it all spelled (numeraled?) out so clearly.

  7. Thomas Monaghan says :

    There was a lady at the Sunday business meeting who claimed to help organize the No Awards. I didn’t catch her name but it should show up on the minutes when they get posted.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction


Pluralism and Individuation in a World of Becoming

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"


A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

three more from on high

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

Read & Survive

How-To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

SFF Book Reviews

random thoughts about fantasy & science fiction books

Philip K. Dick Review

A Re-read Project

Notes From the Darknet

Book reviews and literary discussion

%d bloggers like this: