Margin of Victory: Breaking Down the Hugo Math
In my last post, I looked at the ranges of votes in the categories swept by the Puppy vote to estimate the effective min/max of the Puppy numbers. In this post, I’ll ask this question: How close were these categories to being sweeps? How many additional “traditional” Hugo voters (i.e. non-block voters distributed in the ways Hugo votes have been in the past) would it have taken to prevent a sweep?
I think this information is important because so many various proposals to “fix” the Hugos are currently floating around the web. Even if you accept the premise that the Hugos need to be fixed, what exactly are you fixing? One flaw in the Hugo system is that proposals to change the voting patterns—if such changes are desirable or needed—have to be proposed at the WorldCon, and that’s before the voting results are broadly known. Thus people are working in the dark: they might be trying to “fix” something without knowing exactly the scope (or even the definition) of “the problem.” That’s a recipe for hasty and ineffective change.
What we’ll do today is to compare the lowest Puppy nominee in the 6 swept categories (Novella, Novelette, Short Story, Best Related Work, Editor Short Form, and Editor Long Form) to the highest non-Puppy from 2014. I’ll subtract those two values to find a “margin of victory” for each of those 6 categories. That would give us an estimation of how many more votes it would have taken to get one (and only one) non-Puppy work onto the final ballot. To overcome more of the Puppy ballot would take more votes.
Now, this plays a little loose with the math: we don’t know for sure that the highest non-Puppy nominee from 2015 would have the same number of votes as the 2014 nominee. In fact, it’s highly unlikely they would be exactly the same. However, I think this gives us a rough estimate; it may be 5 higher, or 10 lower, but somewhere in a reasonable range. This will give us a rough eyeball estimate of how pronounced the victories were. On to the stats:
Lowest 2015 nominee: 145 votes
Highest 2014 nominee: 143 votes (Valente’s “Six-Gun Snow White”)
Margin of Sweep: 2 votes
Lowest 2015 nominee: 165 votes
Highest 2014 nominee: 118 votes (Kowal’s “Lady Astronaut of Mars”)
Margin of Sweep: 47 votes
Lowest 2015 nominee: 151 votes
Highest 2014 nominee: 78 votes (Samatar’s “Selkie Stories are For Losers”)
Margin of Sweep: 73 votes
Best Related Work:
Lowest 2015 nominee: 206 votes
Highest 2014 nominee: 89 votes (VanderMeer’s Wonderbook)
Margin of Sweep: 117 votes
Best Editor Short Form:
Lowest 2015 nominee: 162 votes
Highest 2014 nominee: 182 votes (John Joseph Adams; Neil Clarke was second with 115)
Margin of Sweep: -20 votes (if JJA had gotten the same support in 2015 as he did in 2014, he would have placed on the ballot by 20 votes)
Best Editor Long Form:
Lowest 2015 nominee: 166 votes
Second Highest 2014 nominee: 118 votes (Toni Weisskopf was highest with 169 votes, but she was a Sad Puppy 2 nominee (although Weisskopf also has plenty of support outside of Sad Puppies); Ginjer Buchanon was second with 118 votes)
Margin of Sweep: 48 votes
What does all that data mean? That means if that 2015 played out like 2014 for the non-Puppy candidates, the highest non-Puppy candidate missed the slate by this much (I’m repeating the data to make it easier to find). This many more votes for the highest non-Puppy would have broken up the sweep by one nominee.
Novella: 2 votes
Novelette: 47 votes
Short Story: 73 votes
Best Related Work: 117 votes
Editor Short Form: -20 votes
Editor Long Form: 48 votes
The Puppy slates didn’t dominate every category as much as the final results makes it seem. We won’t know the exact margins until August, but I imagine that a non-Puppy Novella was very close to making the final slate. Something like Ken Liu’s “The Regular” probably only needed a few more votes to make it onto the ballot. Editor Short Form was probably even closer; John Joseph Adams must have lost support; if he kept his votes from last year, he would have gotten in.
The other categories were crushed. 73 for Short Story. 117 for Best Related. Given that Samatar’s story from 2014 only managed 78 votes and you needed 151 to make it this year, that’s an almost impossible climb to get just one non-Puppy story onto the slate. I think this reveals a major problem:
the Puppies dominated these categories not only because of their organization, but because of the general lack of voting in those categories. EDIT (4/8/15): Tudor usefully pointed out in the comments that this is probably better understood as a diffusion of the Short Story vote (i.e. the vote is spread out across many stories), rather than a lack of Short Story voters. Thanks for the correction, Tudor, and I always encourage people to push back against any Chaos Horizon statements they think are wrong, incorrect, or misleading. The more eyes we have on stats, the better.
Let’s run some rough math to see how many more voters you would need to get that one work onto the slate. Here’s how I’ll calculate new voters (you may disagree with this formula). I’m assuming you’re bringing new voters into the Hugo process, and that those voters vote in a similar fashion to the past. So, to generate 2 more votes for the top non-Puppy Novella, you’d need to account for the % of voters that bother to vote for the Novella category (2122 people voted in the 2015 Hugo, but only 1083 voted in the Novella category, for 51%). So, to bring 50 new people into the Novella category, you’d need to bring 100 new people into the Hugo voting process.
Next, you need to account for how many people voted for the #1 non-Puppy work. In 2014, that was 16.9% (that’s the percentage “Six-Gun Snow White” got). If we assume a similar distribution, we wind up with 2 votes needed / 51% voting for the category / 16.9% voting for the number one work. That yields 23 new votes needed.
For the Novella category, that’s definitely doable. In fact, if only some of the 700 people who voted for the Best Novel category voted but sat out the Novella category voted, that would add one non-Puppy text back to the slate.
Let’s run the math for the six categories:
Novella: 23 new voters needed
Novelette: 597 new voters needed (47/48.5%/16.2%)
Short Story: 1450 new voters needed (73/55%/9.1%)
Best Related Work: 1830 new voters needed (117/54%/11.8%)
Short Form Editor: no new voters needed; data shows they would have a nominee best on last years patterns
Long Form Editor: 765 new voters needed (48/33.5%/18.7%)
That’s a lot of new voters, and remember this is the number of voters needed to place only one non-Puppy work onto the final nominee list. The Novella and the Short Form Editor categories were close to not being sweeps, but the others were soundly overwhelmed. Think of how the Novel category worked: with much more excitement (700 more voters), the Puppies still placed 3 works onto the list. Finally, I don’t know how you’d add 1000+ new voters without also adding more Puppy voters.
Just for grins, let’s imagine what it would take to eliminate all Puppy influence in the Short Story category. To do that, we’d have to elevate John Chu’s “The Water that Falls from Nowhere” onto this year’s slate based on last year’s percentage vote. Chu—who won the Hugo—managed 43 nominations. To beat the highest 2015 Short Story puppy (230), he’d have to add 187 votes. Chu managed only 5% of the 2014 vote, and 55% of the total Hugo voters voted in the Short Story category. That gives us (187/55%/5%) = 6800 voters. So, if the Hugos added a mere 6,800 voters (and managed to keep all new Puppy votes out!), the Puppies would have been shut out of the Short Story category.
Of course, counter-slates could boost the % of votes going to authors, and there are other solutions that could tilt the field (fewer nominees per voter, more works per slate). What this post goes to show, though, is how organized and enthusiastic the 2015 Puppy-vote was: they not only swept categories, they swept categories decisively.