Chaos Horizon

How Many Puppy Votes: Breaking Down the Hugo Math


The dust is just beginning to settle on the 2015 Hugo nominations. Here’s the official Hugo announcement and list of finalists. If you’re completely in the dark, we had two interacting slates—one called Sad Puppies led by Brad Torgersen, another called Rabid Puppies led by Vox Dax—that largely swept the 2015 Hugo nominations.

The internet has blown up with commentary on this issue. I’m not going to get into the politics behind the slates here; instead, I want to look at their impact. Remember, Chaos Horizon is an analytics, not an editorial website. If you’re looking for more editorial content, this mega-post on File 770 contains plenty of opinions from both sides of this issue.

What I want to do here on Chaos Horizon today is look at the nominating stats. Using those, can we estimate how many Sad Puppies? How many Rabid Puppies?

For those who want to skip the analysis: my conclusion is that the total Puppy influenced vote doubled from 2014 to 2015 (from 182 to somewhere in the 360 range), and that this resulted in a max Puppy vote of 360, and a minimum effective Puppy block of 150 votes. We don’t yet have data that makes it possible to split out the Rabid/Sad effect.

Let’s start with some basic stats: there were 2,122 nominating ballots, up from 1,923 nominating ballots last year, making for a difference of (2,122-1,923) = 199 ballots. Given that Spokane isn’t as attractive a destination as London for WorldCon goers, what is the cause of that rise? Are those the new Puppy voters, Sad and Rabid combined?

If you take last year’s Sad Puppy total, you’d wind up with 184 for the max Puppy vote (that’s the amount of voters who nominated Correia’s Warbound), the top Sad Puppy 2 vote-getter. If we add 199 to that, we’d get a temporary estimate of 383 for the max 2015 Puppy vote. We’ll find that this rough estimate is within spitting distance of my final conclusion.

Here’s a screenshot that’s been floating around on Twitter, showing the number of nominating votes per category. Normally, this wouldn’t help us much, because we couldn’t correlate min and max votes to any specific items on the ballot. However, since the Puppies swept several categories, we can use these ranges to min and max the total Puppy vote in the categories they swept. With me so far?

Click on that to make it bigger. As you can see, that’s from the Sasquan announcement.

The Puppies swept Best Novella, Best Novelette, Best Short Story, Best Related Work, Best Editor Long Form, and Best Editor Short Form. This means all the votes shown in these categories are Puppy votes. Let me add another wrinkle before we continue: at times, the Sad and Rabid voters were in competition, nominating different texts for their respective slates. I’ll get to that in a second.

So, if we were to look at the max vote in those six categories, we’d get a good idea of the “maximum Puppy impact” for 2015:
Novella: 338 high votes
Novelette: 267 high votes
Short Story: 230 high votes
Related Work: 273 high votes
Editor Short Form: 279 high votes
Editor Long Form: 368 high votes

Presumably, those 6 “high” vote-getters were works that appeared on both the Sad and Rabid slates. You see quite a bit of variation there; that’s consistent with how Sad Puppies worked last year. The most popular Puppy authors got more votes than the less popular authors. See my post here for data on that issue. Certain categories (novel, for instance), are also much more popular than the other categories.

At the top end, though, the Editor long form grabbed 368 votes, which was within shouting distance of the Novella high vote of 338, and even very close to the Novel high vote of 387. I think we can safely conclude that’s the top end of the Puppy vote: 360 votes. I’m knocking a few off because not every vote for every text had to come from a Puppy influence. I’m going to label that the max Puppy vote, which combines the maximum possible reach of the combined Rabid and Sad Puppies vote.

Why was there such a drop between the 368 votes for Editor Long Form and the mere 230 votes for Short Story when both of these were Puppy-swept categories? This means that not every Puppy voter was a straight slate voter: some used the slate as a guide, and only marked the texts they liked/found worthy/had read. Some Puppy voters appear to have skipped the Short Story category entirely. That’s exactly what we saw last year: a rapid falling off in the Puppy vote based on author and category popularity. This wasn’t as visible this year because the max vote was so much higher: even 50% of that 360 number was still enough to sweep categories.

Now, on to the Puppy “minimum.” This would represent the effective “block” nature of the Puppy vote: what were lowest values they put forward when they swept a category? Remember, we know that 5th place work had to be a Puppy nominee because the category was swept.

Novella: 145 low vote
Novelette: 165 low vote
Short Story: 151 low vote
Related Work: 206 low vote
Editor Short Form: 162 low vote
Editor Long Form: 166 low vote

Aside from Related Work, that’s enormously consistent. There’s your effective block vote. I call this “effective” because the data we have can’t tell us for sure that this is 150 people voting in lock-step, or whether it might be 200 Puppies each agreeing with 75% of the slate. Either way, it doesn’t matter: The effect of the 2015 Puppy campaign was to produce a block vote of around 150 voters.

If that’s my conclusion, why was the Best Related Work 206 minimum votes? That’s the only category where the Rabid and Sad Puppies agreed 100% on their slate. Everywhere else, they split their vote. As such, that’s the combined block voting power of Rabid and Sad Puppies, something that didn’t show up in the other 5 swept categories.

So, given the above data, here’s my conclusion: The Puppy campaigns of 2015 resulted in a maximum of 360 votes, and an effective block minimum of 150 votes. That ratio of 360/150 max/min (41%) is almost the same as last year’s (182 for Correia at the highest / 69 for Vox at the lowest, for a rate of 37.9%). That’s remarkable consistency. It doesn’t look the Puppy stuck together any more, just that there were far more of them. Of course, we won’t know the full statistics until the full voting data is released in August.

I think a lot casual observers are going to be surprised at that 360 number. That’s a big number, representing some 17% of the total Hugo voters (360/2122). Those 17% selected around 75% of the final ballot. That’s the imbalance in the process so many observers are currently discussing.

What do you think? Does that data analysis make sense? Are you seeing something I’m not seeing in the chart? Tomorrow I’ll do an analysis of how much the non-Puppy works missed the slate by.