Archive | August 2016

2016 Hugos: Some Initial Stat Analysis

So they put the stats up already.

It’s actually a pretty easy analysis this year to see what the Rabid Puppy numbers were in the Nomination and Final stage. We can use Vox Day himself to do that: on the Rabid Puppies, he included himself in the “Best Editor Long Form” category and then later, in his final vote post, suggested himself as #1 in that category. Given the controversy surrounding him, I think we can safely assume that almost all the votes for him were from the Rabid Puppies.

So, here’s where he landed:

Nomination stage:

801 Toni Weisskopf 45.41%
465 Anne Sowards 26.36% *
461 Jim Minz 26.13%
437 Vox Day 24.77%
395 Mike Braff 22.39% **
302 Sheila Gilbert 17.12% 287
Liz Gorinsky 16.27%

So that means 437 votes for the Rabid Puppies in the nomination stage. This is in line with other obvious Puppy picks: Somewhither by John C. Wright with 533,(novels always pick up more votes), 433 votes for Stephen King’s “Obits” (King is a major author, but no one thinks of him for a Hugo), 387 for “Space Raptor Butt Invasion,” 482-384 in the Puppy swept Best Related Work category, 398 for the Castalia House blog, and so forth. That’s a stable enough range for me to say that the Rabid Puppy strength in the nomination stage was 533-384, with around 440 being right in the middle.

So what happened? The Rabid Puppy vote collapsed from the Nomination to the Final Voting stage. This most likely happened because you can nominated in 2016 for free (provided you paid in 2015), but to vote in the final stage in 2016, you had to pay again. Here’s the stats from the first round of the Best Long Form Editor:

Vox Day 165

It’s highly unusual to get 437 votes in the nomination stage and then collapse to 165 in the more popular, more voted in final stage. That 165 represents the most “Rabid” of the Rabid Puppies; some of the other Rabid Puppy picks did considerably better in the first round of voting “Space Raptor Butt Invasion” got 392 votes in the first round of Short Story, for instance. It’s hard to know what exactly to chalk that up to at this point—people enjoying the joke, a broader pool of Rabid Puppy associated voters who didn’t want to vote Vox Day #1 in Long Form Editor.

My initial conclusion then would be around 440 Rabid Puppies in the nomination stage but less than 200 in the final voting stage. Thoughts? Seeing something I’m not?


Jemisin Wins Hugo Award for Best Novel

At WorldCon, they just announced that N.K. Jemisin won the Hugo for Best Novel for The Fifth Season this year! Congratulations to her in what was hard-fought and controversial year.

Jemisin’s win is something of a surprise. We’ll have the stats soon, but Uprooted had a lot of the traditional markers going for it: it won the Nebula, it won the Locus Fantasy, it was more popular then The Fifth Season on Goodreads and Amazon. Jemisin’s win goes to show how unpredictable the Hugos have become–the influx of new voters, along with the high sentiments regarding the Puppy controversy, have basically shot past Hugo patterns all to hell. After ten or so years when the Nebula winner pretty regularly won the Hugo, this may mark a new era in Hugo history. It certainly makes for a more dynamic award, even if it plunges Chaos Horizon and my predictions, into, well, chaos.

In the other categories, things played out largely as expected. In many cases, there was only one non-Puppy choice; that usually won. The two exceptions were in the Campbell, where Andy Weir beat Alyssa Wong, and in Novelette, where Hao Jingfang beat Brooke Bolander. Both Weir and Jingfang were very mainstream picks, however, and could have made the ballot without any Rabid support. In general, the Hugo voters didn’t No Award” Hugo categories just because of overlap with the Rabid Puppies. Neither Gaiman nor File 770 were punished for appearing on the slate. When categories were given “No Award” (Best Related Work, Fancast),  it was because none of the nominees overlapped with typical Hugo picks.

As the dust clears and the stats come out, I’ll continue to do some more analysis. Whatever we can say about 2016, we know that 2017 will be even more unpredictable. We’ve got ballot rule changes, more high passions and controversies, and a new seat of books to ponder over!

Final 2016 Hugo Best Novel Prediction

Let’s me finalize my 2016 Hugo Best Novel Prediction:

  1. Uprooted by Naomi Novik
  2. The Fifth Season by N.K. Jemisin
  3. Ancillary Mercy by Ann Leckie
  4. Seveneves by Neal Stephenson
  5. The Aeronaut’s Windlass by Jim Butcher

Remember, Chaos Horizon doesn’t predict what I want to happen, but rather what I think will happen based on my analysis and understanding of past Hugo trends.

First off, those past trends have been shot full of holes in recent years. The Puppy controversies have fundamentally transformed the voting pool of the Hugos, meaning that past trends might not apply given how widely the voters have changed. New voters have come in with the Puppies; new voters have come in to contest the Puppies; some of those voters might have stayed, some might have dropped out. Some voters are voting No Award out principle; some aren’t. How exactly you balance all of that is going to be largely speculative, maybe to the point that no predictions are meaningful. That’s why we’re called “Chaos Horizon” here!

However, I think the potential Kingmaker effect, when combined with past Hugo trends and the popularity of Uprooted, makes Novik a reasonable favorite. Novik has already won the Nebula and the Locus Fantasy (beating Jemisin twice); her book is a stand-alone, making it feel more complete than either the Jemisin, Leckie, or Butcher; and Novik, along with Stephenson, is more popular than the other nominees. In the past, these have all been characteristics of the Hugo winner.

In the past few years, I’ve developed a mathematical formula to help me predict the Hugos. The formula won’t be accurate this year because of the Rabid Puppy voters, but here’s what it came up with:

Uprooted 27.1%
Ancillary Mercy 20.5%
Seveneves 20.0%
The Fifth Season 17.1%
The Cinder Spires 15.3%

The formula obviously doesn’t take pro or anti-Puppy sentiment into account. Uprooted is a big favorite because of her Nebula win this year. The Nebula has been the best predictor of the Hugo in the last decade: in 5 of the last 10 years, the Nebula winner has gone on to win the Hugo. The stats are actually better than that. In the 2006, 2007, 2009, and 2015, the Nebula winner was not even nominated for the Hugo. So the only time a Nebula winner has lost the Hugo in the final voting round was when Redshirts beat 2312 in 2013. So the last 5/6 times when the Nebula winner had a chance to win the Hugo, it did. Those are nice odds.

My formula is not designed to accurately predict second place. With that in mind, I think Jemisin is too low. Leckie won too recently with Ancillary Justice to seem to have a chance to win again, and Seveneves is pretty divisive. One reason my formula fails is because it currently doesn’t punish books for being sequels. Leckie should be lower for that reason. It’s something I’ll factor in next year. Stephenson will be lower because some people will vote him “No Award” for appearing on the Rabid Puppy list. So this pushes Jemisin up two spots.

However, Jemisin is lower down in the formula because she doesn’t have a history of winning major awards, unlike Stephenson and Leckie. Check out Jemisin’s sfadb page. She’s been nominated for 12 major awards and hasn’t won any. Not good odds. That’s what my formula is picking up on. My formula has trouble gauging changes in sentiment. I think most readers believe The Fifth Season is better than Jemisin’s earlier works, but I have trouble quantifying that.

What my numbers give is a percentage to win based on the patterns of past Hugo votes, based on my analysis and combination of those formulas using a Linear Opinion Pool prediction model. Prediction is different than statistical analysis: different statisticians would build different models based on different assumptions. You should never treat a prediction (either on Chaos Horizon or in something like American elections) the same way you would treat a statistical analysis; they are guided by different logical methods. Someone who disagrees with one of my assumptions would come up with a different prediction. Fair enough. This is all just for fun! You can trace back through the model using some of these posts: Hugo Indicators, and my series of Nebula model posts beginning here. The Hugo model uses the same math but different data.

Let’s look at this with some other data. Here’s the head to head popularity comparison of our five Hugo finalists, based on the number of ratings at Goodreads and Amazon.

Goodreads Amazon
Uprooted 41,174 1,332
Seveneves 35,428 2,487
The Aeronaut’s Windlass 18,249 1,285
Ancillary Mercy 11,698 247
The Fifth Season 7,676 184

These aren’t perfect samples, as neither Goodreads nor Amazon is 100% reflective of the Hugo voting audience, nor has the Hugo Awards always correlated with popularity. Still, it gives us another interesting perspective.

Jemisin does not break out of the bubble in ways that Novik and Stephenson do. These aren’t small differences, either: Uprooted is 5x more popular on Goodreads and 7x on Amazon than The Fifth Season. I put stock in that—the more people read your book, the more there are to vote for it. While the Hugo voting audience is a subset of all readers, popularity matters.

Two other notes: it’s fascinating how different Amazon and Goodreads are. Novik outpaces Stephenson on Goodreads but gets crushed on Amazon. Different audiences, different reading habits. The question for Chaos Horizon is which one better correlates with the Hugo winner? Second, Butcher may be a very popular Urban Fantasy writer with Dresden, but he’s only a moderately popular Fantasy writer.

So, all told, Novik has big advantages in popularity and same year awards (having won the Nebula already). Neither Jemisin, Leckie, or Stephenson managed to do better than Novik in critical acclaim or award nominations. Stephenson and Leckie do beat Novik in past awards history. When we factor in the possible Kingmaker effect from the Rabid Puppies, Novik is a clear favorite.

It would take a lot of change in the voting pool to overcome Novik’s seeming advantages. I wouldn’t count it out completely—these past two years have been so volatile that anything can happen.

Last question—where will No Award place? Last year, voters chose to place Jim Butcher and Kevin J. Anderson below No Award, likely as punishment for appearing on the Puppy slates. Will it happen again this year? I have a hard time seeing Stephenson getting No Awarded, Puppy appearance or not. He’s been a well-liked Hugo writer for a long time, and he may well have scored a nomination without Puppy help. I think Stephenson will beat No Award.

That leaves Butcher. He was No Awarded in 2015 by 2674 to 2000 votes, so a No Award margin of 674. That’s a pretty substantial number. If we go back to 2014, when Larry Correia’s Warbound was the first Puppy pick to make it to the Best Novel category, he beat No Award by a 1161 to 1052 margin. So that means the “No Awarders” are picking up steam. At Chaos Horizon, I go with the past year’s results to predict the future unless there’s some compelling data to suggest otherwise. So I’ll predict that Butcher will lose to No Award in 2016 just as he did in 2015.

So, what do you think? Are we in for a Best Novel surprise, or will Novik walk away with the crown?

The Kingmaker Effect and the 2016 Hugos

As we accelerate towards the announcement of the 2016 Hugo winner, it’s time to think once again about the Kingmaker effect and the Hugo awards. This is going to be essential both for 2016 and moving forward into 2017, even if the Hugo voting changes pass. The “E puribus Hugo” only addresses the nominating stage, leaving plenty of room for kingmaker effects in the final voting stage.

The long and short of it is that a dedicated block of voters can change the outcome by voting for what would normally be the #2 or even #3 place finisher, pushing them into the winner’s circle by overcoming the “organic” winner. Let’s define margin of victory as how many votes there wound up being between the winner and the second place finisher. You can pull this information off of the Hugo voting packets. Basically, this number tells us how many votes you would need to change the outcome of the Hugo. If The Goblin Emperor beat The Three-Body Problem by 300 votes, you’d need a block of at least 300 voters to come in and vote for Cixin Liu to change the outcome (in my opinion, this is pretty much what happened last year):

Other initial Best Novel analysis: Goblin Emperor lost the Best Novel to Three-Body Problem by 200 votes. Since there seem to have been at least 500 Rabid Puppy voters who followed VD’s suggestion to vote Liu first, this means Liu won because of the Rabid Puppies. Take that as you will.

Here’s the data from 2010-2014. I left off last year because the Puppy campaigns changed the results so profoundly:

Margin of Victory 2010 2011 2012 2013 2014
Novel 0 26 161 213 644
Novella 11 244 35 158 83
Novelette 3 97 116 109 460
Short Story 167 194 210 232 307
Related Work 60 53 163 3 84

With a couple of exceptions—Leckie dominating the 2014 novel race with Ancillary Justice, and Mary Robinette Kowal winning the Novellete category in 2014 after she was disqualified a year earlier for the same story—a block vote of 300 would almost always be enough to sway the outcome. In some years, you’d only need a handful of votes. The 0 value in a 2010 is the tie between Bacigalupi and Mieville. It wouldn’t have taken much to push Feed over Blackout/All Clear in 2012, and only a little more to elevated 2312 over Redshirts in 2013. Even without deeply impacting the nominating stage, a block vote can fundamentally change who wins the Hugo award.

So, are we in for any kingmaker scenarios in the fiction categories this year?

Best Novel: I don’t think we’re in a kingmaker situation here, although I do think the Puppy block vote makes Uprooted an almost sure winner. A refresher of where we’re at:

Uprooted by Noami Novik, The Fifth Season by N.K. Jemisin, and Ancillary Mercy by Ann Leckie all made the ballot “organically,” i.e. without appearing on the Rabid Puppy list.

Seveneves by Neal Stephenson and The Cinder Spires by Jim Butcher also made the final ballot. Both appeared on the Rabid Puppy list. Prior to the Puppies, Jim Butcher had never been nominated for a Hugo Best Novel, and past Hugo voting packets show him  receiving very few votes. I often refer back to the 2009 nominating data, where Butcher only received 6 votes for Small Favor, one of his Dresden novels. If that number seems shockingly low for so popular writer, remember that Butcher is associated with Urban Fantasy writing, a sub-genre that has not historically been part of the Hugos.

Stephenson is the most complex situation. He has received Hugo nominations three times before without any Puppy help: for Anathem in 2009, for Cryptonomicon in 2000, and for The Diamond Age in 1996. Diamond Age went on to win the Hugo that year. So while the Puppies certainly helped, we won’t know whether or not Stephenson would have received a nomination on his own until the final data comes out. A few more bits of data: Stephenson received 93 nominating votes in 2009, second most to Little Brother by Cory Doctorow. In the final ballot, Anathem took second, losing to The Graveyard Book by 120 votes, 477 to 357. If Seveneves performs similarly, it could have come down to how many people voted Seveneves as “No Award” based solely on its appearance on the Rabid Puppies list.

However, that’s all a moot point. On the final Rabid Puppy Hugo ballot, Vox Day put Uprooted above Seveneves (Novik/Stephenson/Butcher/No Award was the exact order). That will pretty much clinch the race for Uprooted, based on this logic:

  1. Uprooted was already very likely to finish either #1 or #2 in the Hugo voting, based on Novik’s strong performance in winning the Nebula, the Locus Fantasy Award, and grabbing nominations in the World Fantasy Award and British Fantasy. She has also done very well with SF Critics and Mainstream Critics, all of which are good indicators of Hugo success. She’s sold a ton of copies (46,000+ ratings on Goodreads, for instance). The closest competitor seems to be The Fifth Season, but Jemisin has already lost the Nebula and Locus Fantasy votes to Novik. As such, I think Uprooted was likely to win the Hugo without any help from the Puppies.
  2. The Rabid Puppies were at least 200 strong in the nominating stage, possibly higher. They might be anywhere from 200-500+ in the final voting stage (the final voting always brings more people to the table). Let’s use a very conservative 300.
  3. 300 additional votes for Uprooted at #1 will be enough to cover any potential margin of victory that either Jemisin, Stephenson, or Leckie might have had without the Rabid Puppies. Let’s say Jemisin squeaked out an “organic” victory of 100 votes; once the Rabid Puppies are tallied, that swings outcome back to Novik. You’d have to predict a scenario where Jemisin (or Leckie) would beat Novik by a number greater than the total number of Rabid Puppies. That’s only happened once in the last 5 years, when Ancillary Justice was a consensus book against a weaker field. So could it be Leckie again? I don’t think so; she’s already won a Hugo for this series and I don’t think voters are ready to give her a second. Even if she squeaked out an organic win, I can’t see it being by a 300 vote margin. Butcher will attract tons of No Award votes, so he’s not even in the conversation.

So that leaves Uprooted as the only novel that seems to have a chance of winning the Hugos. What other book has a path to victory? You’d have to predict a huge “organic” win for either Jemisin or Leckie, and that just doesn’t seem likely. We’ll find out shortly!

Best Novella, Best Novelette, Best Short Story:

In each of these categories, the Rabid Puppies swept 4 out of 5 positions. This means that the non-slate story is the prohibitive favorite, based on how many people voted slated works No Award last year.  If there’s any drama, it might be in “Best Novella.” Nnedi Okorafor’s Binti, the Nebula winner, is the non-slate work. Louis McMaster Bujold’s Penric’s Demon, from the same universe as her Hugo winning Paladin of Souls, is the number #1 Rabid Puppy pick. How many people will No Award Bujold based solely on her appearing on the Rabid Puppies slate? Let’s say Binti wins by an organic margin of 200 (before factoring in the Rabid Vote) and the Rabids are 400. It would take only 200 voters “No Award”ing Penric’s Demon to keep Binti the winner. I expect that to happen, but this will be some great data to sort through once the packets are released.

I don’t see anything preventing “And You Shall Know Her by the Trail of Dead” or “Cat Pictures Please” from winning the Novelette and Short Story category. Stephen King’s huge popularity will be blunted by his not being primarily associated with Science Fiction or Fantasy. “Folding Beijing” might be competitive, but the Rabid Puppies put it lower on their list, minimizing its chances.

So, what do you think? Will there be any kingmaker effects this year? Or will the Hugo fiction categories play out pretty much as they would have without the Rabid Puppies?

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction


Pluralism and Individuation in a World of Becoming

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"


A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

three more from on high

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

Read & Survive

How-To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

SFF Book Reviews

random thoughts about fantasy & science fiction books

Philip K. Dick Review

A Re-read Project

Notes From the Darknet

Book reviews and literary discussion