Hugo Award Nomination Ranges, 2006-2015, Part 2

The Hugo is a strange award. One Hugo matters a great deal—the Best Novel. It sells copies of books, and defines for the casual SFF fan the “best” of the field. The Novella, Novelette, and Short Story also carry significant weight in the SFF field at large, helping to define rising stars and major works. Some of the other categories feel more like insider awards: Editor, Semiprozine. Others feel like fun ways to nod at the SFF fandom (Fanzine). All of them work slightly differently, and there’s a huge drop off between categories. That’s our point of scrutiny today, so let’s get to some charts.

First, let’s get some baseline data out there: the total number of nominating ballots per year. I also included the final voting ballots. Data gets spotty on the Hugo website, thus the blank spots. If anyone has that data, point me in that direction!

Table 2: Total Number of Nominating and Final Ballots for the Hugo Awards, 2006-2015
Table 2 Ballots 2006-2015

I pulled that data off the site, save for the flagged 895, which I grabbed from this File 770 post.

Now, how popular is each category? How many of those total nominators nominate in each category? First up, the averages for 2006-2015:

Table 3: Average Number of Nominating Ballots in the Hugo Award per Category, 2006-2015
Table 3 Number of Nominating Ballots Each Category

I included to averages for you: the 2006-2015 average, and then the 2006-2013 average. This shows how much the mix of Loncon, the Puppy vote, and increased Hugo scrutiny have driven up these numbers.

What this table also shows is how some categories are far more popular than others. Several hundred more people vote in the Novel category than in the next most popular category of Dramatic Long, and major categories like Novella and Novelette only manage around 50% of the Novel nominating vote. That’s a surprising result, and may show that the problem with the Hugo lies not in the total number of voters, but in the difficulty those voters have in voting in all categories. I’ve heard it mentioned that a major problem for the Hugo is “discovery”: it’s difficult to have a good sense of the huge range of novellas, novelettes, short stories, etc., and many people simply don’t vote in the categories they don’t know. It’d be interesting to have a poll: how many SFF readers actually read more than 5 new novels a year? 5 new novellas? I often don’t know if what I’m reading is a novella or a novelette, and does the lack of clarity in this categories hurt turnout?

Let’s look at this visually:

Chart 3 Popularity of Categories

Poor Fan Artist category. That drop off is pretty dramatic across the award. Are there too many categories for people to vote in?

Let’s focus in on 2015, as that’s where all the controversy is this year. I’m interested in the percentage of people who voted for each category, and the number of people who sat out in each category.

Table 4: Percentage of Voters and “Missing Votes” per Hugo Category, 2015 Only

Table 4 % Voters in Each Category

The chart at the top tells us a total of 2122 nominated in the Hugos, but no category managed more than 87% of that total. The missing votes columns is 2122 minus the number of people who actually nominated. I was surprised at how many people sat out each category. Remember, each of those people who didn’t vote in Best Novel, Best Short Story, etc., could have voted up to 5 times! In the Novella category alone, 5000 nominations were left on the table. If everyone who nominated in the Hugos had nominated in every category, the Puppy sweeps most likely wouldn’t have happened.

Again, let’s take a visual look:

Chart 4 % Voters by Category

That chart re-enforces the issue in the awards: less than 50% turnout in major categories like Novella, Short Story, and Novelette.

What to conclude from all of this? Total number of ballots isn’t as important as to who actually nominates in each category. Why aren’t people nominating in things like Short Story? Do the nominations happen too early in the year? Are readers overwhelmed by the sheer variety of works published? Do readers not have strong feelings about these works? Given the furor on the internet over the past few weeks, that seems unlikely. If these percentages could be brought up (I have no real idea how you’d do that), the award would immediately look very different.

Tomorrow, we’ll drill more deeply into the Fiction categories, and look at just how small the nominating numbers have been over the past decade.


Tags: ,

15 responses to “Hugo Award Nomination Ranges, 2006-2015, Part 2”

  1. Standback says :

    “If everyone who nominated in the Hugos had nominated in every category, the Puppy sweeps most likely wouldn’t have happened.”

    I find this conclusion highly suspect.

    It’s making a very large assumption – that more nomination ballots would have resulted in higher nomination numbers for individual pieces. But I don’t think that’s the case at all.

    Quite the opposite – what I’m seeing in these numbers are that a good 40-50% of WorldCon members who care enough about the Hugo to nominate, still don’t consider themselves to have opinions in any of the short fiction categories.

    If next year these people decide, “YES, I *will* nominate in all the categories” —
    are any two of them going to nominate any of the same stories?
    If so, based on what? How would they achieve this synchronicity?
    If it’s based on stories they read and enjoyed – I imagine they would have nominated in previous years. I’d conclude that many of those who have nominated up until 2015, simply do not read enough short fiction to form an opinion on short fiction nomination.
    And if it’s based on some kind of coordination between the voters – well, a slate would do it, but that’s the last thing we want. Your alternative is a more organic clustering – via reviews, recommendations, signal-boosting, etc.. But the stronger any one signal-boosting locus is, the closer we get to a slate. So *ideally*, what you *really* want is lots and lots of relatively small clusters, but to increase the voting base tenfold – so that natural clusters can compete with a slate even without being entirely dominant. That’s… not simple.

    If the Puppies nominators have, say, 100, people, it is not enough to have a thousand people nominating other things, or two thousand, or ten thousand – UNLESS the nominating ballots have clusters of agreement. And at the moment, I’m just not seeing how you might hope to see those clusters to form.

    • chaoshorizon says :

      Good eye—I should have added the sentence “if voting patterns follow past trends.” That’s an assumption I make so often on Chaos Horizon that I often leave it out. The claim that the future will be relatively similar to the past is the inherent weakness of the system I use here. It’s an assumption, but I’d also challenge your idea that “are any two of them going to nominate any of the stories?” I would answer emphatically “yes.” There aren’t an infinite number of stories, and certain stories/novellas/works are more prominent than others: based on past Hugo familiarity, from sales, from marketing, from reader reaction, from blog buzz, etc. All of those factors would combine to create “clusters of agreement.” I’d say that if you look at the past Hugo data, you see how such cluster “naturally” (or maybe “inevitably”) emerge. Whether or not that clustering is fair is another issue.

      So take a look at previous data regarding the Short Story and Novella. Every year, 3-5 works have emerged with around 5% of the vote, with the top vote-getters grabbing well over 10% of the vote. If you apply that same range to 1,000 new voters, you’d get 50-100 more votes for some works. If you add those to the votes they’d already have (again, we have to model historically), that’d put them slightly above the low range of the 2015 Puppy vote. That seems the most “middle-of-the-road” model I can come up with, although I’m happy to hear alternative modeling.

      Lastly, let me point you to the Best Novel category. That’s a place where, with more votes, a Puppy sweep was interrupted. For me, that’s pretty good evidence that if other categories like Novella, Novelette, etc., had the same number of votes as the Novel category, at least one non-Puppy text would have made the final ballot. Of course, any model can’t predict the future with 100% accuracy: maybe the Puppy vote will double in 2015, maybe these hypothetical 1000 extra voters would have 0% agreement on their choices, etc. The statistical modeling I use can’t handle major shifts in voting patterns like that.

      • Lord Darque says :

        I would like to get your opinion on something. Isn’t the simplest solution to just encourage a number of slates/lists? And by encourage I mean publish them in the WorldCon program. The main problem seems to be a lack of knowledge of what is out there. The average voter just has no idea what might be popular. Which the huge number of works how is anyone to know what has a real chance of making the ballot. People won’t bother to nominate if they feel they are just tossing ideas out that nobody else is going to support. That 5% barrier is a real killer if you are not plugged into the mainstream of Fandom.

        Now I know there are plenty of lists and such out there. But only those in the know find them. If people really knew about them then we would see a lot more activity. They don’t produce the kind of clusters you say we need to make things work better.

        Set some moderate entry barrier such as needing 10 WorldCon members to get a slate published. It seems to me that people are so against the Hugos becoming political despite what seems to me clear evidence that they have been for a long time. Embrace slates and make them a part of WorldCon and everyone gets to have their voice heard.

      • chaoshorizon says :

        It’s an intriguing idea. There’s several ways to handle this, though, and that’s where things bog down—the details get complicated very quickly. What you might be thinking of is an officially “Hugo recommended” reading list, where any work suggested by 10 (or 20 or whatever) Hugo voters gets listed on the Hugo website. That would focus the discussion a little more, and would centralize the votes. Even minor centralization would make a slate sweep harder.

        But . . . should the Hugo website be doing anything intentional to centralize votes? That gets dangerously close into meddling with your own democratic process. The Hugos are currently super hands-off: they don’t even officially comment on eligibility of texts before the awards are announced. A “recommended reading” list would be a major cultural change for the Worldcon, and institutions are always slow to change. Right now, the Locus Recommended List plays something of that role, but, as you point out, you need to be pretty involved in Fandom to know that list.

        I think it’ll be interesting to see if the furor over this year’s Hugos leads to a more involved (or at least earlier involved) votership. My website has been trying to fill in those gaps for the Best novel category, producing lists of the Top 20-25 books most likely to get Hugo nominations. I wouldn’t consider these slates, but they are a tool to help readers focus their own reading. I’ve been trying to do that in an unbiased way (you can judge whether I reach that bar!). Maybe we need something like Chaos Horizon Jr. for the other categories. If only I had the time . . .

      • Standback says :

        Well, really my point is that I expect short fiction to model quite differently than novels.

        Buzz around short fiction works very, very differently than for novels – there are far, far fewer reviews, and less discussion. Accessibility is another issue – if you hear good things about a novel, you go buy the novel, or check it out from a library. But if you heard good things about a particular story in last month’s Asimov’s, getting the particular issue might be difficult, and for a short piece – most readers are unlikely to bother. Similarly, a reader might not grab an anthology just for one particular story. The exception here is online fiction – which is certainly the height of accessibility.

        My other issue is that I think you’re downplaying the significance of the 50% of nomination ballots that don’t nominate short fiction at all. You’re basically saying, “well, if they did nominate, they’d vote in a similar distribution to existing nominations.” Whereas I’m saying these are people who explicitly chose not to nominate – and that’s very significant indeed. It implies that nominating in these categories is more difficult; and that fewer readers have firm nominations to offer in these categories.
        If they aren’t firm, I’d guess that’s either due to ignorance of the field, or else failure to find Hugo-worthy favorites within it. In either case, I’d expect nominations (if these readers were forced to make them) would be fairly scattershot and random.
        (The Puppies, on the other hand, have no such concerns – a slate voter votes for the whole slate.)

        I agree that when you have enough nominating votes, clusters will inevitably form. My concern is that within the short-fiction field, there are a LOT of barriers to those clusters reaching large enough numbers to counter a slate.

      • chaoshorizon says :

        I’ll concede the point that the Short Story category in particular would model differently. Of all the fiction categories it is the most diffuse, and you may be right that more voters would just increase that diffusion. I don’t think that’s the case for Novella and maybe Novelette, which further shows how difficult any proposed fix would be.

      • Standback says :

        I’m curious as to why you think Novella (and Novellette) should be less diffuse than Short Story. I’d love to hear, if you’re inclined to detail. Or, I can be patient and wait for the next few posts on nomination breakdowns 🙂

      • chaoshorizon says :

        I’m planning on posting the data relating to this specifically in Part 4. As a little preview, here are the number of unique works nominated in Novella, Novelette, and Short Story this year:
        Novella: 201 entries / 1083 nominating ballots
        Novelette: 314 entries / 1031 nominating ballots
        Short Story: 728 entries / 1174 nominating ballots

        That’s quite a few more short stories! I think the other categories cluster more naturally because fewer Novellas/Novelettes are published. When we look at high/low ranges in Part 3, we’ll see that the % for the Short Story category are much lower than other categories.

    • G. Carlisle says :

      Fighting. Over deck chairs. On the Deathstar Titanic.

      Will all this puppy “movement” (I keep thinking bowel when I write that) stimulate more sf/f sales? Don’t think so.

      And, I suspect, no one in the reading community could care less about any of this.

      Which means we should probably just laugh.

      And start worrying about something that actually matters.

      Childhood hunger in America anyone?

  2. Lord Darque says :

    Any solution has to recognize the limitations of the structure of WorldCon. I would not be in favor of a single list compiled as you suggest. It would just be another level of election and I don’t think they could handle that.

    A simple system that requires 10 signatures or approval emails and then a whole list gets published on the Web site. I would say putting in the program as well but either way would be a good start.

    I have seen a lot of proposals for modification of the vote procedure but most fail because they are just too complex. As you have pointed out I believe any rules changes that work for the upper categories could be very damaging to the lesser ones. So what they really need is perhaps 3 different sets of rules on how to handle the top, middle and bottom categories and that is absolutely not possible.

    In the end I believe everyone will realize that no system that is simple enough to be done will stop the ability of groups to manipulate it. Because of this either nothing will change or they will eliminate supporting noms and votes completely. You can see the first acknowledgement in a recent post by David Gerrold. He said perhaps it is time to discuss kicking people out. This will snowball into the only possible way to return the Hugos to its former state by eliminating the supporting noms/votes completely.

    Thank you for the reply. I know you do not wish to get into the messy side of things but I just wanted to float the idea by you.

  3. jaed says :

    I’d be interested in seeing how far back that trend goes, if you have the historical data from decades ago.

    Lord Darque’s suggestion that “The main problem seems to be a lack of knowledge of what is out there.” reminds me that I have a half-baked theory that short SF used to be largely “push” in nature, because so much of it was published in a few magazines. If you subscribed to (depending on the era) three or four or so magazines, you would get much (most?) of the eligible short fiction for the year delivered through your mail slot. There was less of it back then, but more to the point, you didn’t need to seek it out: it came to you, and you probably read all of it in its eligibility year, just in the normal course of things.

    If the falloff from novel to short categories was less steep in say, the 60s and 70s, that would support this idea.

    • Lord Darque says :

      I suspect you are on the mark. The huge amount of new stuff has helped to diffuse the vote. So any campaigns on any level are bound to be effective. When it only takes a handful of votes to get something on the ballot the system is just ripe for manipulation.

      While I believe there has been a lot of this behind the scenes most of it has been fairly public. But what I don’t want to see is any one group dominating which is where my suggestion came from. Level the playing field and let everyone get their lists out. It is not right that those with the biggest fanbases have such an advantage.

      My worry is despite the words that have been said there are those who liked the influence they have had and the only solution they want is one that returns things to that status quo.

      • Craig says :

        There are, often, as many motivations as there are people involved.

        (I’ll also note that until we see who made it to the ballot in place of MHN, we can’t be entire sure that the SP/RP dominance there wasn’t stronger than the first disclosed nominee list).

  4. Lord Darque says :

    We will find out what people really think and want in two solid ways. 1) The actual vote and 2) The proposals for rules changes at the Business Meeting. Those will show us everything we need to know about the make-up and sizes of the current groups.

    It would be helpful if Sasquan would come out with more numbers. The type of analysis done here are invaluable to people looking to make changes or realize that none are going to do the job.

    I had come to the conclusion that trying to control things via rules changes would be impossible and seeing the numbers here seem to me to prove that conclusion rather well.

Trackbacks / Pingbacks

  1. The Fellowship of the Puppy 5/8 | File 770 - May 11, 2015

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Xeno Swarm

Multiple Estrangements in Philosophy and Science Fiction


Pluralism and Individuation in a World of Becoming

Space and Sorcery

Adventures in speculative fiction

The BiblioSanctum

A Book Blog for Speculative Fiction, Graphic Novels... and more!

The Skiffy and Fanty Show

Running away from the thought police on wings of gossamer and lace...

Relentless Reading

"A Veritable Paladin of Blogging!"


A little about me, a lot about books, and a dash of something else

Far Beyond Reality

Science Fiction and Fantasy Reviews

Andrew Liptak

three more from on high

Eamo The Geek

The Best In Sci-Fi And Fantasy Book Reviews by Eamon Ambrose

Read & Survive

How-To Read Books

Mountain Was Here

writing like a drunken seismograph

The Grimdark Review

The very best of fantasy.

SFF Book Reviews

random thoughts about fantasy & science fiction books

Philip K. Dick Review

A Re-read Project

Notes From the Darknet

Book reviews and literary discussion

%d bloggers like this: