The Hugo and Nebula Awards and Repeat Nominees, Conclusions and Discussion

Over the past two weeks, Chaos Horizon has been looking into the idea of Repeat Nominees and the Hugo and Nebula Awards for Best Novel, 2001-2014. Remember, Chaos Horizon is a website dedicated to providing predictions for the Hugo and Nebula Best Novel awards, and I want these predictions to be based on more than my opinions about what the “best” books of the year are. If you want those kinds of opinions, the internet is crawling with them.

Instead, Chaos Horizon takes the position that we can better understand the Hugo and Nebula awards by data-mining past awards to find patterns concerning nominations and winners. While this won’t allow us to know with 100% certainty how future awards will go—that’s not how statistics work—this will allow us to make informed guesses as to what the nominees and winners will be in the future.

The basic hypothesis I’m working with is that there are 7 or 8 determining factors which factor into these awards. Roughly speaking, these are: past awards history, critical reception, reader reception, popularity/sales, marketing and web footprint, genre, demographic concerns, and reputation. Some of these are incredibly hard to quantify (reputation, for instance); others are slippery (genre); others are changing rapidly (demographics); and others are mine-fields of conflicting opinions (critical and reader response). Nonetheless—and perhaps foolishly—I believe we can wade into these factors and make some sense of them.

So, in regards to “repeat nominations”—one aspect of awards history—what have we learned in Parts 1 to 6, and how can this information be applied? For those who didn’t read Parts 1 to 6 (and they got pretty technical!), here’s what I think we can conclude:

Conclusion #1: The Hugo and Nebula Best novel slates are substantially biased towards authors who have previously received a Best Novel nomination, to the tune of 65% for the Hugo and 50% for the Nebula.
Application #1: When I make a prediction for the Hugo Slate, my prediction should be 2/3 previously nominated authors, and 1/3 rookie authors. For the Nebula, I should go 1/2 previously nominated authors, 1/2 rookie authors.

Conclusion #2: The Hugo Award slate favors super-repeaters, authors who get nominated for the Best Novel award over and over again. In 2001-2014, the top 7 Hugo authors accounted for 45% of the total ballot. The Nebula award does no show the same bias towards super-repeaters.
Application #2: When putting together a prediction for the Hugo slate, I need to pay special attention to the authors who have previously received more than 4 nominations.

Conclusion #3: Winning the Nebula Award is biased towards past winners and repeat nominees, with 64% of the winners having previously appeared on the ballot and 43% having won before. Proportionally, the Hugo did not show the same bias towards past winners and repeat nominees.
Application #3: When predicting the Nebula winner, I need to strongly factor in past Nebula nominations and wins.

Conclusion #4: There is no strong evidence to suggest that Hugo or Nebula Best Novel nominees need to have been nominated in other Hugo and Nebula categories before snagging a Best Novel nomination or win.
Application #4: I need to be careful about predicting authors “jumping” from the Short Story, Novelette, or Novella categories to the top of the slate. While it happens, it’s not the advantage you would think. Or, in other words, I need to keep on open mind towards authors completely new to the Hugo and Nebula process.

Conclusion #5: Despite the Hugo and Nebula favoring “repeat nominees,” even the repeaters don’t get every novel nominated. Most repeaters only manage a 25%-50% nomination rate, no matter how popular.
Application #5: I can’t blindly put popular authors onto my watchlist, but I need to analyze how each specific novel was received, including factors such as genre and critical/reader response.

That’s a fairly fruitful study, yielding some specific application that can help improve my watchlists and predictions. As I continue to do these, hopefully Chaos Horizon can become more and more useful as a resource to the SFF community.

What this information doesn’t tell us is whether this “bias” is good or bad. Maybe you believe that there are only 20-25 exception writers at work today in SFF today, and that the centralization of the awards reflects the excellence at the top of the field. Or maybe you believe there are hundreds of interesting SFF writers, and that some are unfairly excluded from the awards because of this centralization towards past winners and nominees. Chaos Horizon, nor any kind of statistical analysis, can answer those questions for you.

Here’s my Excel worksheets with the data I used. Let me know if you have any questions about methodologies or how I came up with my numbers.

So, finally, what do you think of the results? Did you expect the Hugos and Nebulas to work in this way? Was some of the information surprising? Is there additional information we need about repeat nominees and these awards? How will this factor in predicting the Hugos and Nebulas? Has this hurt or helped your perception of some of the chances of the 2014 candidates?

Thanks for the reading, and stay tuned for the next Chaos Horizon report, where I’m going to tackle the question of genre and the Hugo and Nebula awards! How biased are these awards towards science fiction? How much do they hate fantasy? We’ll find out soon . . .