Ballot Changes: Updating the Hugo Math

In the past few days, there have been four changes to the Hugo ballot: two authors (Marko Kloos and Annie Bellet) have dropped out, and two more authors/artists have been ruled ineligible (a John C. Wright novelette and Professional Artist John Eno). All the details can be found at the official Hugo Awards Website, and File 770 has a nice overview article.

Since Chaos Horizon is a Hugo and Nebula analytics site, these changes give us some new information about the voting totals, campaigns, and block sizes of the 2015 Hugos. In my two previous Hugo math posts—How Many Puppies and Margin of Victory—I tried to use the information we already have to estimate (and it’s only an estimate) the size of the effective voting blocks and the margins of victory. With this new data, I can update those estimates. Let’s go through the changes one by one to see what we can learn:

Best Novel Category: Cixin Liu’s novel The Three-Body Problem replaced Marko Kloos’s withdrawn novel Lines of Departure. Kloos’s novel appeared on both the Sad and Rabid Puppy slates. The range of votes prior to Kloos’s withdrawal was 256-387. The new range of votes is 212-387.

This doesn’t tell us that much. We know that The Three Body-Problem novel received 212 votes. We can’t assume that Kloos was the 256 vote getter (that could have been Addison), so this doesn’t help us estimate the size of the combined Rabid + Sad Puppy effective block vote in the Novel category. I’ve been using the term “effective block” because the current data doesn’t allow us to distinguish between 300 Puppy voters voting 67% of a slate or 200 voters voting 100% of the slate. Since there is such a range between the top and low end of the votes (even in categories the Puppies swept), I think you need to assume that not every Puppy voter voted straight down the line.

However, we can tentatively conclude that neither the Rabid Puppy nor the Sad Puppy alone was able to reach 212 votes. The Puppy slates diverged with their last choice of novel: the Sad Puppies had Charles Gannon, and the Rabid Puppies had Brad Torgersen. However, it may be that both of them were offered the Hugo spot and turned it down for various reasons. Until we have more info, we’ll have to chalk up a “learned little” for this category.

Best Novelette Category: Now we can learn something. John C. Wright’s story “Yes, Virginia, There is a Santa Claus” was ruled ineligible and replaced by “The Day the World Turned Upside Down” by Thomas Olde Huevelt. That adjusted the category range from 165-267 to 72-267. Wright was a Rabid Puppy pick; Huevelt was not on either Puppy slate. We know that “The Day the World turned Upside Down” received 72 votes. Since the Puppies swept this category, we know that 165 had to be a Puppy text.

This category gives us our first definitive sense of the “margin of victory,” or how convincingly the Puppies swept the Hugos. 165 for the lowest Puppy vote, and then 72 for the highest non-Puppy vote. That’s a huge 93 votes, or more than double Huevelt’s total.

In my prior post, using 2014 stats which showed Mary Robinette Kowal getting 118 votes for “Lady Astronaut of Mars,” I estimated the margin as much lower than that (47, to be exact). However, Kowal’s numbers may have been inflated (her story was ruled ineligible the year before, so she got plenty of press and support). Torgersen was second in 2014 Novelette nomination with 92 (he was a Puppy pick), and Alette de Bodard was third with 79. That 79 number is in line with this year’s total, which may indicate that the non-Puppy group of Hugo nominators did not increase this year. Catherynn Valente’s “Fade to White” was the most nominated 2013 Novelette with 89 votes. 89 to 72 is quite a decline, considering how many more voters there were this year.

So, we can say, at least in the Novelette category, the Puppy margin of victory was 93 votes, in a category where the most popular work usually has less than 100 votes.

Best Short Story Category: Annie Bellet withdrew her Rabid and Sad Puppy supported story “Goodnight Stars,” which was replaced by the Sad Puppy supported story “A Single Samurai” by Steven Diamond. The range changed from 151-230 to 132-226. We know “A Single Samurai” must have received 132 votes, and we also know that “Goodnight Stars” must have received 230 votes (since the top range also changed). This category was swept by the Puppies.

It’s interesting that the top range in this category (230) is so different than in other Puppy-swept categories: 338 for Novella, 267 for Novelette (back when it was swept). The block vote certainly fell off quickly from category to category.

This is our first real chance to see the Sad and Rabid Puppy votes separated. The Sad Puppy category slate chose stories by Bellet, Grey, English, Antonelli, and Diamond. The Rabid Puppies only chose a few of those: the Bellet, the English, and Antonelli. All three were nominated. Vox Day filled out his slate with a story by Wright (nominated) and a story by Rzasa (not nominated). So the Sad Puppy nominee of Diamond with 132 votes beat Rabid Puppy nominee Rzasa. If Rzasa would have kept all the Rabid Puppy votes that went to John C. Wright, he would have made the ballot. It’s interesting to see that even in the Rabid Puppy slate there is a discernible fall off between the most popular authors to the less popular ones.
EDIT: I was running these late last night, and missed looking at the Rzasa. My mistake. Sorry! Chaos Horizon should impose a “no math after 10:00 PM rule.”

So, let’s tackle this again, in the clear light of morning and after I’ve had my coffee: This is our first real chance to see the Sad and Rabid Puppy votes separated. The Sad Puppy category slate chose stories by Bellet, Grey, English, Antonelli, and Diamond. The Rabid Puppies only chose a few of those: the Bellet, the English, and Antonelli. It added two different stories, one by Wright and one by Rzasa. Both of those made the slate. If we break this down, we can start to separate some of the math out:

Short Story Category:
#1 in category: 230 votes: “Goodnight Stars” by Bellet (we know this because she withdrew and the ranged changed, story appeared on both the RP and SP slates).
#2 in category: 226 votes, don’t know the text (likely the Antonelli or English, because they appeared on both the the RP and SP ballots).
#3 in category: don’t know vote total, don’t know the story (but probably the English or Antonelli, for reasons stated above)
#4 in category: don’t know vote total, don’t know the story (but probably the Wright or Rzasa story, since they only appeared on the RP ballot)
#5 in category: 151 votes, don’t know the story (but probably the Wright or Rzasa story, because of reasons stated above)
#6 in category: 132 votes, “A Single Samurai” (appeared only on the SP slate)

With some fancy subtracting and hopefully some sound logic, here are the conclusions I’m reaching:
The SP effective vote in this category was 132, because that’s the number of votes their nominee “A Single Samurai” managed. They had one more suggestion, a story by Megan Grey, that must have placed below that.

The RP effective vote in this category was likely 151. I think it’s logical to assume that stories #3 and #4 in the ballot were the RP picks, based on the assumption that any pick on both the RP and SP ballots would have more votes since it could draw from both pools. That’s an assumption, not a fact, but I think a reasonable one. Disagree if you wish! What’s interesting is that 133 + 151 = 283, which is well shy of the # of votes the Bellet story received. This is another good indicator that neither the SP influenced voters of the RP influenced voters were moving exactly in lock step. If every SP voter who voted for “The Single Samurai” and ever RP voted who voted for either Wright or Rsaza voted for Bellet, her vote total would have been much larger. This is all complicated by possible interactions between the two groups (i.e. one person might have picked some works from the SP ballot and some works from the RP ballot), and we won’t know more precisely until after the data comes out.

Best Professional Artist: John Eno was ruled ineligible. He was a Sad and Rabid Puppy pick. He was replaced by Kirk DouPonce, a Rabid Puppy pick. There was no alternative Sad Puppy pick to elevate (they only chose 4 artists in the category). The range changed from 136-188 to 118-188.

So we know Kirk DouPonce received 118 votes. You might want to begin thinking about that number (118) as the low end of the Rabid Puppy effective block vote. That would be consistent with the Short Story category results: 118 wouldn’t quite have been enough to push the Rzasa story onto the ballot. Still, 118 votes is a huge number, and would have been enough to sweep most Hugo categories without any support from the Sad Puppies. There were two slates, both of which were large enough to effectively dominate most Hugo categories.

All of this will be greatly clarified when we get the final data after the Hugos are announced in August. Remember, these are estimates working with limited data, and are probably considered in terms of ranges (through a +/- 50 if you want to) rather than absolutes.

Anything else we can figure out at this point? EDIT: For easy references, here are the last 4 years of Hugo data, taken from the Hugo website:

As Kerani points out in the comments, so much is changing so fast in this year’s Hugo data—increased voters, withdrawals, disqualifications, block votes, etc.—that this data set (particularly in its incomplete form) can’t be considered to have a ton in common with previous data sets. Any comparisons we draw have to be considered speculative. But, since our field is speculative fiction, why not? Remember, Chaos Horizon data analysis is more for fun than anything else, and provides, in my opinion, a useful alternative to some of the other more-opinion driven sites out there. Take everything with a grain of salt, and use your own logic and analysis to make sense of what’s happening.

18 responses to “Ballot Changes: Updating the Hugo Math”

  1. Craig says :

    Where any of the known lows nominees in prior years? That might give you a change-value different from starting at zero.

  2. VD says :

    There is an error in your description of the Short Story category. Steve Rzasa’s “Turncoat”, a Rabid Puppy recommendation, was nominated.

    The maximum nomination vote is not hard to estimate. If I recall correctly, I have it Worldcon 256, Sad Puppy 157, Rabid Puppy 230.

    • chaoshorizon says :

      Thanks for the correction. I was running the numbers too late at night and must have looked at the wrong list. That’s what I get for stepping away from the Best Novel category! I’ve made the correction.

      • chaoshorizon says :

        I’ll have to think about the max ranges a little more. So you’re assuming the lowest placing novel was the Addison? I think that’s reasonable but not 100% certain, unless someone reported that she was a replacement after Correia was withdrawn. I could have easily missed that announcement in all the posts.

        I’m not following the 230 logic. Are you assuming everyone who voted for Bellet was a RP? While that is a theoretical max, it seems too high given the other data we have (such as 131 people voting for a SP story in the same category). As I said, I’ll give some more thought to this problem this weekend and see if I can come up with ranges.

      • VD says :

        Notice that I said MAX range. I’ll have to look at it again, but when you are looking at maxes, you have to look at the Novel category. Everything else always drops off from there.

        The reason it is so absurd to complain about the RP/SP “bloc vote” is that the statistical evidence makes it very clear that there is less of a monolithic bloc vote than there has been in the past among similar alliances such as the Scalzi-Stross and Hartwell-Nielsen-Hayden one. The RP variance alone is huge even within category, never mind across categories.

        Meanwhile, the Scalzi-Stross vote was virtually identical both within and without category. That’s not even remotely plausible.

        The big difference is that we wanted to recommend meritorious writers were never going to get recognized because they are outside the Emerald City/NESFA list. They wanted to collect Hugo nominations, as their 15 and 9 Hugo nominations tend to demonstrate.

        You yourself have observed that even repeated bestsellers like Larry Correia aren’t reviewed by the SF media. As we have repeatedly replied to those who say we are breaking the Hugos, you cannot break what is already broken. Hopefully a new, more broad-spectrum, more open, less incestuous system can evolve, and if it does, Larry Correia should be seriously considered for a Lifetime Achievement award.

      • chaoshorizon says :

        In that case, just don’t short-change the SP; by the same logic, we could estimate the SP max as 230. In fact, I don’t think that’s an unreasonable estimate: 230-118 SP impact in the fiction categories, 230-150 RP impact in the fiction categories. I wonder how high Monster Hunter Nemesis was. I suspect it probably beat that 387 number. Once the full data comes out, Chaos Horizon will post more precise ranges. We should be able to be pretty accurate because we can look at the SP and RP works that weren’t on the other slates.

  3. keranih says :

    Re: Short story – I think you’ve got the ranges incorrectly. “Single Samurai” “must” (*) have gotten 132 votes.

    I think that this analysis might benefit from a consideration of the amount of the vote captured by the top 15 nominations (including withdrawals) over the last however long we have that data. If I get time this weekend I’ll look into it.

    Additionally – I think that the withdrawals in an ordinary year have significant impact on ranges – but even more so this year. This is rapidly approaching the status of “crap data set”, imo.

    Having said all that – while I am sorry to see the works by Kloos and Bellet be withdrawn, I am (separately) pleased to have Three Body Problem added.

    • chaoshorizon says :

      Yep—apparently I need to impose a “no Chaos Horizon posts” after 10:00 PM rule. Turns out that I’m too sleepy to count at that point. There’s always an urge to rush the data out because I know people are interested, but I clearly need to double-check a little more closely.

      There haven’t been that many withdrawals in an ordinary year over the last 4-5 years. The Hugo people usually do a good job of crossing those out in the .pdf packets. I’m going to stick the last 4 years of .pdfs data packets at the end of this post so people can find those easily. The Hugo website does have some past data after that, but I believe a couple of the years are missing.

      I agree that this data set is going to be pretty useless moving forward. It may be that the entire award will change so much this year that most of the 2001-2015 data won’t be helpful in predicting 2016.

    • chaoshorizon says :

      Three Body-Problem was my favorite SFF novel of 2014, and I’m greatly looking forward to The Dark Forest in July. That’s definitely going to be a major contender for both the 2016 Hugo and Nebula. I’m disappointed that Ken Liu isn’t translating The Dark Forest, but you can’t begrudge him working on his own career. I recently bought The Wandering Earth, Cixin Liu’s collection of SF stories. I haven’t had a chance to start it yet—I’m too busy wrapping up my semester here at my university, but once I get through all my grading, I’m anticipating digging more deeply into Liu’s work. I wonder if we’ll see some more Chinese SF novels translated soon. Of course, this would be the most shocking result for the Hugos—a WorldCon award actually going to an international SF novel.

      • MadProfessah says :

        The Three-Body Problem was not my favorite Sci fi book of 2014 (that would be James SA Corey’s Cibola Burn) but I am very happy that it is now publicly acknowledged as one of the best SF novels of 2014.

        I, too, am definitely looking forward to The Dark Forest coming out this June.

        Can we talk a little about Kevin Anderson’s nomination? Is there anyone out there who doesn’t regard the latest Dune books he co-authored as execrable exploitation of Frank Herbert’s brand?

      • chaoshorizon says :

        I’ll admit I’ve never read a KJA book before. Sequels to classic works of SF or Fantasy by different authors are always bound to be very divisive: it’s not fair to expect anyone to recapture the original “magic” of the field’s most famous books. Most major authors can’t even follow up their own books—think of how much worse Rama II is than Rendezvous with Rama. But I’m someone who wants to judge an author on their best work, not their worst work. Maybe KJA had more freedom and less market pressure when writing his own stuff, and we don’t know how much Brian Herbert and the Dune estate dabbled in the Dune sequels. All those Dune sequels sold well, so someone must have liked them . . .

      • keranih says :

        MadProffessah –

        I’m confused – that work wasn’t nominated, was it? The nomination was for a different book…not sure what relevance that has to a discussion of the stats for this year or previous years…

        (Plus – no accounting for taste. I’m sure there was at least one reader out there who was deeply touched by the book, and that reader should have been able to vote like everyone else…)

    • keranih says :

      Addendum to above – because it appears there is not an “error-free posting period” for CH, no matter what our host may claim šŸ˜‰ –

      (*) – this is reminding me of those logic games from high school. Or playing advanced sudoku. Or getting five of my buddies and going to visit the elephant house at the zoo.

  4. keranih says :

    Just dropped you an email with a data set attached, looking at best novel, best short, and campbell for the last 10 years (ish.)

    This is a mess. We need a TARDIS.

    Let me know if you have issues.

  5. Mike Glyer says :

    If it helps — Torgersen he was not contacted about a nomination so you can include that as a data point.

  6. spacefaringkitten says :

    It’s not important for the numbers here, but Grey’s short story may also have been ruled ineligible. There was some discussion about that possibility in SP circles. I think that had a good chance of getting more votes than some other works, because you could read it online for free.

  7. keranih says :

    @ SFK – Grey’s story’s eligibility discussed on File 770 here:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction


Pluralism and Individuation in a World of Becoming

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"


A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

three more from on high

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

Read & Survive

How-To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

SFF Book Reviews

random thoughts about fantasy & science fiction books

Philip K. Dick Review

A Re-read Project

Notes From the Darknet

Book reviews and literary discussion

%d bloggers like this: