Gearing Up for Hugo 2015 Analysis

The Hugo Awards are almost upon us! These will be given out tonight, but I’m more interested in the numbers the Hugos will release alongside those. Over the next week, Chaos Horizon will be doing what Chaos Horizon does, digging into those to find trends, data, and info.

It should be a truly interesting analysis this year. At this point, we have no real idea about the numerical strength of the Sad Puppies, the Rabid Puppies, or the No Awarders. Once we begin putting some of those together, we’ll have a much better sense of the current shape of the field.

It’s going to take a while to sift through the data. Here’s my early game-plan; I’m laying this out there for a full critique from anyone who wants to comment. We need good numbers, no matter your position on any of the 2015 Hugo controversies.

Here’s what I think we can do:

1. Estimate the number of Rabid Puppies: Since Vox Day, the leader of the Rabid Puppies, posted Hugo voting recommendations, we can use those to estimate the Rabid Puppy numbers. In particular, I’ll be looking at the “first pass” numbers for a few swept categories, namely Best Editor, Short Form and the Campbell to come up with my initial estimate.

Here’s my methodology and chain of assumptions: Vox recommended himself for Short Form Editor. I’m making the assumption that only hardcore Rabid Puppy supporters are going to follow that. Given how controversial Vox currently is and how niche Vox’s editing is, I find it hard to believe you would support Vox Day for Short Form Editor if you aren’t a Rabid Puppy supporter. We have to make some assumptions here; this seems the safest to me. Contest in the comments if you wish.

With that number in place, I’m going to compare it to the Campbell, where Vox recommended Eric S. Raymond, often known as ESR. ESR is best known as an open source software advocate, and he has a very popular blog. He is not well-known as a SF writer, having only published one story in a Castalia House publication (Vox Day’s house). Again, the connection to Vox Day means that probably—and this is an estimate, not a fact—that primarily Rabid Puppy supporters are voting ESR. If the ESR vote number is close to the Vox number from Short Editor, that’ll be some good confirmation. If it’s not, I’ll rethink my assumptions.

I’ll then compare this number to Vox’s other recommendations in the other categories, particularly those that are controversial. Some categories won’t tell us anything; Vox recommended The Three-Body Problem in Best Novel and Guardians of the Galaxy in Best Long Form Dramatic; both of those will attract lots of non-Puppy support. In other categories, such as John C. Wright’s “One Bright Star to Guide Them” in Novella or “No Award” in Graphic Novel, we have a narrower field of support. If these numbers are close to each other (let’s say within 50), I’m confident in calling that the Rabid Puppy range. I should be able to double-check to see how many votes move from Vox Day’s recommended first choice to second choice in the voting.

2. Estimate the No Awarders: By looking at how many people voted “No Award” for their first choice in the swept categories (Novella, Novelette, Short Story, Related Work, Short Form Editor, Long Form Editor), we can get an easy initial estimate of how many people voted “No Award” over every Puppy pick.

3. Compare the first past No Awarders to the final pass No Awarders: This will give us a good estimate of the number of people who gave the Puppy ballots a chance. So if 300 voted No Award over every choice, but by the time we get to the 4th pass 1000 people voted no Award, I can produce an estimate of roughly 700 for “voted at least one Puppy pick.” This will be most useful in swept categories, and will allow me to come up with the what I’m calling the “Neutrals.”

4. Estimate Sad Puppies numbers: This is actually the hardest number to estimate. In theory if I have the Rabid Puppies, the No Awarders, and the Neutrals, everyone else is the Sad Puppies? This would be the group of people who didn’t follow Vox Day’s recommendations but still voted every Sad/Rabid Puppy pick above No Award. We’ll have some contamination by people who just liked that individual story, but if we had a broad group from 5-6 categories, the estimate should be decent. If you’ve got a better way of estimating this, let me know.

5. See if the Rabid Puppies impacted the Best Novel: If we take the final margin of victory in Best Novel and compare it to the Rabid Puppy estimate, we’ll know whether or not they were the swing vote for Best Novel.

So that’s the initial outline. Everything I do here on Chaos Horizon is open, so let me know what you think of the methodology. Once we sort through the final numbers, I’ll go back and start working on nomination numbers.

TL;DR: So here’s the basic initial approach. I’m going to break down the Hugo 2015 voters into four categories:
1. Rabid Puppies: People who followed Vox Day’s Hugo voting recommendations.
2. No Awarders: People who vote No Award over every Rabid/Sad Puppy pick.
3. Neutrals: People who voted at least one Puppy pick above No Award.
4. Sad Puppies: People who voted all Rabid/Sad Puppy picks above No Award, but didn’t follow Vox Day’s recommendations.

I’ll primarily be using the swept or nearly swept categories to do this.

Not perfect, I know, but it should give us something. Comments? Suggestions? Mathematical or analytical tricks I missed?