Margin of Victory: Breaking Down the Hugo Math

In my last post, I looked at the ranges of votes in the categories swept by the Puppy vote to estimate the effective min/max of the Puppy numbers. In this post, I’ll ask this question: How close were these categories to being sweeps? How many additional “traditional” Hugo voters (i.e. non-block voters distributed in the ways Hugo votes have been in the past) would it have taken to prevent a sweep?

I think this information is important because so many various proposals to “fix” the Hugos are currently floating around the web. Even if you accept the premise that the Hugos need to be fixed, what exactly are you fixing? One flaw in the Hugo system is that proposals to change the voting patterns—if such changes are desirable or needed—have to be proposed at the WorldCon, and that’s before the voting results are broadly known. Thus people are working in the dark: they might be trying to “fix” something without knowing exactly the scope (or even the definition) of “the problem.” That’s a recipe for hasty and ineffective change.

What we’ll do today is to compare the lowest Puppy nominee in the 6 swept categories (Novella, Novelette, Short Story, Best Related Work, Editor Short Form, and Editor Long Form) to the highest non-Puppy from 2014. I’ll subtract those two values to find a “margin of victory” for each of those 6 categories. That would give us an estimation of how many more votes it would have taken to get one (and only one) non-Puppy work onto the final ballot. To overcome more of the Puppy ballot would take more votes.

Now, this plays a little loose with the math: we don’t know for sure that the highest non-Puppy nominee from 2015 would have the same number of votes as the 2014 nominee. In fact, it’s highly unlikely they would be exactly the same. However, I think this gives us a rough estimate; it may be 5 higher, or 10 lower, but somewhere in a reasonable range. This will give us a rough eyeball estimate of how pronounced the victories were. On to the stats:

Lowest 2015 nominee: 145 votes
Highest 2014 nominee: 143 votes (Valente’s “Six-Gun Snow White”)
Margin of Sweep: 2 votes

Lowest 2015 nominee: 165 votes
Highest 2014 nominee: 118 votes (Kowal’s “Lady Astronaut of Mars”)
Margin of Sweep: 47 votes

Short Story:
Lowest 2015 nominee: 151 votes
Highest 2014 nominee: 78 votes (Samatar’s “Selkie Stories are For Losers”)
Margin of Sweep: 73 votes

Best Related Work:
Lowest 2015 nominee: 206 votes
Highest 2014 nominee: 89 votes (VanderMeer’s Wonderbook)
Margin of Sweep: 117 votes

Best Editor Short Form:

Lowest 2015 nominee: 162 votes
Highest 2014 nominee: 182 votes (John Joseph Adams; Neil Clarke was second with 115)
Margin of Sweep: -20 votes (if JJA had gotten the same support in 2015 as he did in 2014, he would have placed on the ballot by 20 votes)

Best Editor Long Form:
Lowest 2015 nominee: 166 votes
Second Highest 2014 nominee: 118 votes (Toni Weisskopf was highest with 169 votes, but she was a Sad Puppy 2 nominee (although Weisskopf also has plenty of support outside of Sad Puppies); Ginjer Buchanon was second with 118 votes)
Margin of Sweep: 48 votes

What does all that data mean? That means if that 2015 played out like 2014 for the non-Puppy candidates, the highest non-Puppy candidate missed the slate by this much (I’m repeating the data to make it easier to find). This many more votes for the highest non-Puppy would have broken up the sweep by one nominee.
Novella: 2 votes
Novelette: 47 votes
Short Story: 73 votes
Best Related Work: 117 votes
Editor Short Form: -20 votes
Editor Long Form: 48 votes

The Puppy slates didn’t dominate every category as much as the final results makes it seem. We won’t know the exact margins until August, but I imagine that a non-Puppy Novella was very close to making the final slate. Something like Ken Liu’s “The Regular” probably only needed a few more votes to make it onto the ballot. Editor Short Form was probably even closer; John Joseph Adams must have lost support; if he kept his votes from last year, he would have gotten in.

The other categories were crushed. 73 for Short Story. 117 for Best Related. Given that Samatar’s story from 2014 only managed 78 votes and you needed 151 to make it this year, that’s an almost impossible climb to get just one non-Puppy story onto the slate. I think this reveals a major problem: the Puppies dominated these categories not only because of their organization, but because of the general lack of voting in those categories. EDIT (4/8/15): Tudor usefully pointed out in the comments that this is probably better understood as a diffusion of the Short Story vote (i.e. the vote is spread out across many stories), rather than a lack of Short Story voters. Thanks for the correction, Tudor, and I always encourage people to push back against any Chaos Horizon statements they think are wrong, incorrect, or misleading. The more eyes we have on stats, the better.

Let’s run some rough math to see how many more voters you would need to get that one work onto the slate. Here’s how I’ll calculate new voters (you may disagree with this formula). I’m assuming you’re bringing new voters into the Hugo process, and that those voters vote in a similar fashion to the past. So, to generate 2 more votes for the top non-Puppy Novella, you’d need to account for the % of voters that bother to vote for the Novella category (2122 people voted in the 2015 Hugo, but only 1083 voted in the Novella category, for 51%). So, to bring 50 new people into the Novella category, you’d need to bring 100 new people into the Hugo voting process.

Next, you need to account for how many people voted for the #1 non-Puppy work. In 2014, that was 16.9% (that’s the percentage “Six-Gun Snow White” got). If we assume a similar distribution, we wind up with 2 votes needed / 51% voting for the category / 16.9% voting for the number one work. That yields 23 new votes needed.

For the Novella category, that’s definitely doable. In fact, if only some of the 700 people who voted for the Best Novel category voted but sat out the Novella category voted, that would add one non-Puppy text back to the slate.

Let’s run the math for the six categories:
Novella: 23 new voters needed
Novelette: 597 new voters needed (47/48.5%/16.2%)
Short Story: 1450 new voters needed (73/55%/9.1%)
Best Related Work: 1830 new voters needed (117/54%/11.8%)
Short Form Editor: no new voters needed; data shows they would have a nominee best on last years patterns
Long Form Editor: 765 new voters needed (48/33.5%/18.7%)

That’s a lot of new voters, and remember this is the number of voters needed to place only one non-Puppy work onto the final nominee list. The Novella and the Short Form Editor categories were close to not being sweeps, but the others were soundly overwhelmed. Think of how the Novel category worked: with much more excitement (700 more voters), the Puppies still placed 3 works onto the list. Finally, I don’t know how you’d add 1000+ new voters without also adding more Puppy voters.

Just for grins, let’s imagine what it would take to eliminate all Puppy influence in the Short Story category. To do that, we’d have to elevate John Chu’s “The Water that Falls from Nowhere” onto this year’s slate based on last year’s percentage vote. Chu—who won the Hugo—managed 43 nominations. To beat the highest 2015 Short Story puppy (230), he’d have to add 187 votes. Chu managed only 5% of the 2014 vote, and 55% of the total Hugo voters voted in the Short Story category. That gives us (187/55%/5%) = 6800 voters. So, if the Hugos added a mere 6,800 voters (and managed to keep all new Puppy votes out!), the Puppies would have been shut out of the Short Story category.

Of course, counter-slates could boost the % of votes going to authors, and there are other solutions that could tilt the field (fewer nominees per voter, more works per slate). What this post goes to show, though, is how organized and enthusiastic the 2015 Puppy-vote was: they not only swept categories, they swept categories decisively.


18 responses to “Margin of Victory: Breaking Down the Hugo Math”

  1. Tudor says :

    “I think this reveals a major problem: the Puppies dominated these categories not only because of their organization, but because of the general lack of voting in those categories.”

    I don’t think that the lack of voting was the problem, but the diffusion of the nonSP nominees vs. block voting (only Best Novel and Best Dramatic Presentation Long Form had more nomination ballots than Best Short Story in 2014). The fact we had 3 nominees for the Best Short Story in 2013, 4 in 2014 and now there are 5 nominees, clearly shows the difference between voting base on choosing from hundreds of short stories and block voting.

    • chaoshorizon says :

      Thanks, Tudor. Your characterization is more accurate than mine: lots of votes in the short story category, but the votes are spread out so broadly that each individual story is unable to gather many votes. When the field is more concentrated, as in the Novel or the Novella, the results are less diffuse. So the problem isn’t the lack of votes, but the plethora of choices.

  2. Aled Morgan says :

    What Tudor said — in 2014 lots of people nominated lots of short stories, which mostly didn’t get 5% of the total. Probably the same thing happened this time. There’s a lot of really great short fiction being published these days. 865 people nominated in short story in 2014.

    • chaoshorizon says :

      I went ahead and added the correction into my post. Thanks everyone for the double-check!

    • Susan says :

      Yes, there is. I proofread/edit for a couple of short story/novella authors and they are pretty good. Not good enough yet for award nominations, but I suspect they might in the future.

  3. Tudor says :

    Have you seen this proposal that uses the Condorcet method? It looks very interesting.

  4. J says :

    Have you seen the proposals by Bruce Shneier?

    I think his method is the one I like the most of suggested changes.

    • chaoshorizon says :

      Once all the proposals have been made (and some of the furor of the discussions die down), I’ll try to run an analysis of the most popular proposals to see what kind of effect they might have. A lot of people are putting forward proposals without running any models of how such things would change the Hugos.

      • chaoshorizon says :

        Just a quick example while I’m thinking of it. The 4-6 proposal has been getting traction; 4 nominees per voter, and then 6 texts in the final category. I presume you’d have to get rid of the 5% rule, because a 20% diminishing of the vote would likely move most stories out of the qualifying range. Think about what this would have done to the 2013 Short Story category. This would have resulted in adding 3 additional Seanan McGuire stories to the slate, since she placed 4th with 30 votes, and then tied for 6th with herself, Helena Bell, and Ken Liu with 28 votes. So we’d have wound up with a category that had 3 Seanan McGuire stories and 2 Ken Liu stories, and that’s only with a block vote of 28. That’s an extreme case, but it only took me five minutes of looking at the nominations to find an instance where the 4-6 model would have been pretty broken.

    • Susan says :

      Seems that Neilsem-Haydon are supporting his suggestions, which makes me suspicious in view of their extreme hysteria on Saturday/Sunday.

  5. spacefaringkitten says :

    Now that there were changes to the original ballot and a non-slate novelette was added there, we can see how wide the sweep margin was: 90+ votes, double what was expected in this post.

  6. nathancherolis says :

    Thanks guys. This is extremely informative. I knew we Rabbid Puppies kicked an enormous amount of butt… but its always nice to see just how much butt we kicked analyzed statistically. By the by… I sincerely hope the rules are changed to try to minimize our influence. There is nothing that could prove Larry’s point more.

    Crazy thought… maybe instead of figuring out how to change the rules… maybe you should just organize yourselves?

    Or maybe you’re afraid that even if you do organize yourselves… you’ll still lose. Maybe you’re afraid that we’re right… and you really are a small little insignificant clique… and the Big Bad Unwashed will own the award for the near future.

    Yes…. by all means… Change the rules.

    • nathancherolis says :

      also… point of clarification… Please don’t take the “you’ and “You’re” in this post to mean yourselves personally. I didn’t. I wrote it poorly. I should have specifically spoken about the worldcon cliques of old. I have no idea how any of you stand on this issue. I don’t even care. I just appreciate the work you’ve done and continue to do.

    • Tudor says :

      As Chaos showed in another post there were a maxim 360 puppies votes from a total of 2122 ballots (17% puppies vs. 83% non-puppies). Puppies dominated not because they are a “big clique” and non-puppies are a “small clique”, but because they were a small number of people that nominated from a small number of potential nominees vs. a not enough large number of people that nominated from a very large number of potential nominees.

    • Tudor says :

      Also, we will see the strength of each group only now, when both will choose from the same number of small nominees. But you’ll have to “organize yourselves” much better than 17% if you want to stand a chance.

      • nathancherolis says :

        and you’ll have to figure out some way to not split your own vote between various nominees… and the suicidal fit throwers dropping No Award bombs.

      • chaoshorizon says :

        We’re definitely in for some interesting times. Let’s leave it there.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction


Pluralism and Individuation in a World of Becoming

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"


A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

three more from on high

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

Read & Survive

How-To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

SFF Book Reviews

random thoughts about fantasy & science fiction books

Philip K. Dick Review

A Re-read Project

Notes From the Darknet

Book reviews and literary discussion

%d bloggers like this: