Estimating the 2016 Hugo Nominations, Part 3

This is where things get messy—perhaps to the point of incoherence. I estimated 5,000 voters in last years Hugos who seemed to not be associated with the Sad or Rabid Puppies: some Hugo voters of past years, some who joined to vote No Award for the Puppies, some who joined just to vote, some who maybe joined to participate in the controversy, and some who joined for unknown reasons. We don’t have much past data on this group, so how can we calculate how they’re likely to vote in 2016?

To be honest, we probably can’t, not with any definite certainty. What I want to achieve is just getting in the ballpark, of producing a low and high estimate that we can compare to the Rabid Puppies estimate. That’ll at least tell us something. You know me: I always like numbers, no matter how rough they are.

What we do have is data from past Hugo years, and if we make the assumption that voting patterns won’t be wildly different from previous years, we’d at least have a place to start. So, first thing I’ll do is take a look at previous voting percentages in the Hugos. This chart shows what percentage of the Vote the #1, #2, etc. novel receives in a typical Hugo year. For these purposes, I average the voting patterns from 2010-2013 (the 4 prior years with no Puppy influence, drawn from the Hugo voting packets).

Table 1: Voting Percentages in Hugo Best Novel Category

2013 2012 2011 2010 Average
Book #1 17.34 18.27 15.01 20.3 17.73
Book #2 12.4 17.01 12.98 15 14.3475
Book #3 12.13 13.57 12.24 14.3 13.06
Book #4 11.95 8.45 9.96 11 10.34
Book #5 10.6 7.41 9.36 8.9 9.0675
Book #6 9.07 7.3 8.88 8.9 8.5375
Book #7 8.18 7.2 8.64 7.6 7.905
Book #8 8.09 6.89 8.28 7 7.565
Book #9 6.65 6.47 7.68 7 6.95
Book #10 6.2 6.37 7.2 6.4 6.5425

This means that the most popular Hugo book, in any given year, gets around 17% of the vote, with a range of 15%-20% in the years I looked at. So if 4,000 people vote in 2016, we might estimate the top book as getting between 600-800 votes. If 3,000 people vote, that drops us down to an estimate of 450-600 votes.

Does this estimate tell us anything, or is it just useless fantasizing? I can see people arguing either way. What this does is narrow the range down to something somewhat sensible. We’re not predicting Ann Leckie is going to get 2000 votes for Best Novel. We’re not predicting she’s going to get 100. I could predict 450-800 and then match that against the 220-440 Rabid Puppies prediction. That would tell me Leckie seems like a likely nominee.

We can go destroy this prediction if we make different assumptions. I could assume that the new voters to the Hugos won’t vote in anything like typical patterns, i.e. that they are complete unknowns. Maybe they’ll vote Leckie at a 75% rate. Maybe they’ll vote her 0%. Those extremes grate against my thought patterns. If you know Chaos Horizon, I tend to chose something in the middle based on last year’s data. That’s a predictive choice I make; you might want to make other ones.

I believe voting patterns will be closer to traditional patterns than totally different from them. You may believe otherwise, and then you’ll need to come up with your own estimate. If I’m off, how far off am I, and in what direction? Too low by 100-200 votes? Too high by 100-200 votes? And if I’m off by only that much, is the outcome of this prediction affected?

So, this says . . . if these 5,000 vote along similar lines to past Hugo voters, and we imagine three turnout scenarios, where do we end up?

Let’s not drop in book titles yet, let’s just multiply table #1 by three different turnout scenarios for our 5,000 2015 voters (40%, 60%, and 80%):

Table 2: Estimated Votes in 2016 Hugo Best Novel Category Based on Prior Voting Patterns

Turnout 40% 60% 80%
# of Typical Voters 2000 3000 4000
Book #1 355 532 709
Book #2 287 430 574
Book #3 261 392 522
Book #4 207 310 414
Book #5 181 272 363
Book #6 171 256 342
Book #7 158 237 316
Book #8 151 227 303
Book #9 139 209 278
Book #10 131 196 262

What I’m interested is whether or not these numbers beat the Rabid Puppy numbers from last post. Even if we assume robust Rabid Puppy turnout of creating 440 votes, we have 1 novel above that in the 60% scenario and 3 in the 80% scenario. Even if you pump the Rabid Puppy number up to 500, we still have at least 1 novel above in both the 60% and 80% scenario. If we lower the Rabid Puppy vote to a more modest 400, we wind up with 2 in the 60% and 4 in the 80%. This is the piece of data I want: in most turnout scenarios, a few “typical” books beat the Rabid Puppies, but not all. I’d estimate we’re in store for a mixed ballot. Only in very modest turnout scenarios (40%) do the Rabid Puppies sweep Best novel.

We do need to factor the Sad Puppies in (next post), but the numbers don’t suggest to me that the Rabid Puppies will manage 5 picks on the final Hugo ballot. The numbers also don’t suggest that there won’t be 0 Rabid Puppy picks.

Two questions remain for this post: what will the turnout be? I don’t know how many of those 5,000 will vote. I know passions are high so I assume the turnout will be high. Then again, this stage is difficult to vote in: you have to read a bunch of novels, remember when the ballot is due, realize you’re eligible to nominate, etc. To my eye, somewhere between 50-75% seems about right. I’m going to pick a conservative 60% just to have something to work with. Since I included all three bands, you’re free to pick anything according to your tastes and your own sense of what will happen.

Next, what is Novel #1 going to be? Novel #2? Novel #3? I’m just going to utilize my SFF Best of Critics list to drop in here. I definitely think the top of the list (Leckie, Jemisin, Novik) is what this group will be voting for. I’ll preserve ties, so if we go to the Top 12 contenders I’m tracking for this prediction, here’s where they show up:

1. Ancillary Mercy (estimate 17.73% of vote)
1. Uprooted (estimate 17.73% of vote)
3. The Fifth Season (estimate 13.06% of vote)
3. Aurora (estimate 13.06% of vote)
10. Seveneves (estimate 6.54% of vote)

Golden Son and The Aeronaut’s Windlass did show up on the list, so I’m going to give them a 1% of the vote. Something but not a lot. This is in line with Butcher’s past totals from before the Sad Puppies; he was only getting a few votes based on 2009 Nomination Data, where WorldCon showed us how many votes everyone got.

Could those be off significantly? Sure they could. That’s why it’s an estimate! Multiply those out in the scenarios, and you get this chart:

Typical Voters
Turnout Scenario 40% 60% 80%
Voters 2000 3000 4000
Ancillary Mercy 355 532 709
Uprooted 355 532 709
The Fifth Season 261 392 522
Aurora 261 392 522
Seveneves 131 196 262
Golden Son  20  30  40
Somewhither
The Aeronaut’s Windlass  20  30  40
Agent of the Imperium
Honor At Stake
A Long Time Until Now

Leckie and Novik get lots of votes, probably enough to beat the Rabid Puppies without any help. Given Leckie pulled in 15.3% of the vote in 2015 and 23.1% in 2014, wouldn’t her vote percentage be somewhere in that ballpark for 2016? The average of those two is 18.15%, and I predicted her at 17.73% in the chart above. Estimating is a different act than logical proof, and one that is ultimately settled by the event—come a few weeks, we’ll known the ballot, and I’ll either be right or wrong. Jemisin and Robinson get votes and will be competitive. Stephenson is down lower, but he also appears on the Rabid and Sad lists, so that jumps him over Jemisin and Robinson’s total and onto the ballot.

I know this will be the most disliked of the predictions. You know my theory at Chaos Horizon: any estimate gives you a place to start thinking. Even if you vehemently disagree with my logic, you now have something to contest. What makes more sense? If you apply those estimates, what ballot do you come up with? So argue away!

Tomorrow the Sad Puppies, and then we combine the three charts to get a bunch of different scenarios. If we see patterns across those scenarios, that’s the prediction.

Advertisements

13 responses to “Estimating the 2016 Hugo Nominations, Part 3”

  1. greghullender says :

    One factor worth thinking about is that fans are highly motivated to nominate five candidates in every category this year. In the past, the average has been three or less. Increasing that average would be equivalent to boosting the number of people making nominations.

    Did you look into power-law distributions at all? If the nominations follow a power-law with exponent p then doubling the total number of nominations would increase the score of the nth ranked candidate by 2^p. That means that the percentage of all votes going to the top-ranked candidate may drop off based on how many total votes there are. For best novel, I estimated p to be about 0.6, so if you doubled the number of people nominating, you’d only increase the top vote by 50%

    • chaoshorizon says :

      True–and I think this impact will be substantial on the downballot categories, particularly as those received lower participation in the past. If I was modelling something like Short Story, you’d have to model “category turnout” as well. That’s when you’d really need power-law distribution, because increasing the number of works you vote for doesn’t allow you to vote for a story twice even if you double the vote, etc. One (rather unsatisfying) thing I decided to do on Chaos Horizon was not to use math above the high school level in a vain attempt to keep the website accessible. I regret that decision sometimes . . .

      You guys at Rocket Stack Rank going to do any predictions? I’d love to see some math on the Short Story Category. I’ve got no real sense if those are going to be clean sweeps, one non-Puppy story in, etc.

      • greghullender says :

        “Business as usual” calculations show a total Puppy sweep across all categories–even best Novel–barring a fan turnout of unbelievable proportions. The way I figure it, if any of Vox Day’s slated choices fails to appear on the final ballot, then he has failed. He ought to sweep every category easily.

        You might understand why we didn’t feel like publishing that.

        There are several ways that things might not be “business as usual.” It hasn’t seemed productive to speculate on those prior to the close of nominations, though.

      • chaoshorizon says :

        It’s definitely hard to see what centralizing force could happen to in categories like Short Story or Best Related Work to stop a sweep. Not only are more votes needed, but more agreement on the major works of the year is needed. I do think the Novel centralizes itself because there are fewer truly viable choices, as does Best Dramatic Work. It’ll be interesting to see.

        I went and looked at this post, where I calculated the number of unique works each ballot voted for in the various fiction categories. It’s around 3-2.5. Novella would be the category most susceptible to change; it had the lowest # of works per ballot (2.5), and there’s fewer Novellas published every year.

      • greghullender says :

        Lists like RSR might provide enough centralization to make a difference, but I’m not optimistic.

        http://www.rocketstackrank.com/p/2016-hugo.html

        We provided advice in seven categories, of which short-form editor is the only one with few enough choices to have a good shot at shrugging off the slate impact.

        I’m cautiously optimistic about the fan and pro-artist pages. That means I think that at least one non-slate candidate has a good chance.

        For the three short-fiction categories, I so much hope that the very best ones will make the list despite the slate, and yet it’s so hard to believe it can happen unless his slate discipline really falls apart. Novella has the best shot, for sure, but even that’s not a great one.

        Wish I could be more optimistic.

      • greghullender says :

        By the way, high-school math does include exponents and logarithms. And probably differential calculus, depending on the student.

      • chaoshorizon says :

        True enough, but usually only basic statistics.

  2. NH says :

    Have you checked what “normal” turnout used to be? I have it in my head that it might have been as low as 20%, but I don’t have the numbers to hand. I certainly agree that higher than average turnout seems likely this year, but I think I’d be even a little surprised if it cracked 50% of the total eligible voting population (ie all of sasquan, midamericon and Helsinki’s members).

    • chaoshorizon says :

      As a matter of fact, I did! Here’s the “carryover” link, showing how the ratio between turnout in last year’s vote and the number of nominating ballots the next year. Traditionally it’s around 60%-80%, with some oddities due to location. Of course, we don’t know if those are the same voters.

      Note that this stat is different than the “eligible voting population participation,” which is always extremely low as you point out. I think that number would be well over 10,000. The 6,000 number I’m working with is the number of voters, not the total number of eligible voters.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction

AGENT SWARM

Pluralism and Individuation in a World of Becoming

Rare Horror

We provide reviews and recommendations for all things horror. We are particularly fond of 80s, foreign, independent, cult and B horror movies. Please use the menu on the top left of the screen to view our archives or to learn more about us.

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"

MyLifeMyBooksMyEscape

A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

a nightmare of wires

Reading SFF

Reading science fiction and fantasy novels and short fiction.

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

The Other Side of the Rain

Book reviews, speculative fiction, and wild miscellany.

Read & Survive

How- To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

From couch to moon

Sci-fi and fantasy reviews, among other things

%d bloggers like this: