It’s the end of the month, so let’s check in on Goodreads and Amazon popularity (as measured by number of rankings) for various Hugo and Nebula contenders. This is one of many different measures I look at when predicting the Hugo and Nebula nominees.
As I’ve said before, this data is interesting but not necessarily predictive for the Hugos and the Nebulas. Goodreads and Amazon # of rankings doesn’t accurately measure popularity; they measure popularity with the Goodreads and Amazon crowds, which may or may not be well-synced up with Hugo or Nebula voters. We have no real access to sales number to actually measure books sold, so this is about the best we can do. Historically, being popular hasn’t helped much for the Nebulas. For the Hugos, it matters more, but only when that popularity is combined with strong critical response and past Hugo history.
I’m slowly migrating all my data over to Google Sheets and the cloud, so that you can look at and process the data any way you want. Here’s the link.
Table #1: Popularity of Hugo/Nebula Contenders on Goodreads, December 2015
It’s interesting how static these charts are; no one really moved up or down more than 2 spots from November to January. I also track some books that aren’t contenders (Armada, for instance), just to give us some reference.
What does this mean for the Hugos? Well, Uprooted and Seveneves are hugely popular books this year, with 4 or 5 times more rankings than other award contenders like The Fifth Season or Ancillary Mercy. Even though someone like Stephenson may prove divisive (lots of people love or hate that book), the sheer number of readers may translate into more voters. Remember, you can’t vote against a book in the nomination stage. All that matters is how many people like a book, not how many hate it; the reverse can be true on the final ballot. The huge number of rankings for Novik and Stephenson is why I’ll have them very high in my initial Hugo predictions.
On the flip side, a book like Karen Memory is languishing with only 1,500 Goodreads ratings / 75 even though it came out in February. I don’t think that’s enough readers to drive Bear to a Hugo nomination in a competitive year, but only time will tell. I often use these popularity charts to distinguish between similar books. If Dickinson, Cho, Liu, Jemisin, and Novik all vaguely fall under the category of “experimental fantasy,” I’ll pick Novik/Jemisin over Liu/Cho/Dickinson based on their apparent popularity, using the theory more readers = more votes. Hopefully once I have several years of data I can find a more solid correlation, although one certainly isn’t visible yet.
Lastly, it’s fascinating at how different the Amazon rankings are than Goodreads. Why does Goodreads like Armada more than Seveneves? A book like A Long Time Until Now does terribly on Goodreads but well on Amazon (#12 on my Amazon chart, #28 on my Goodreads chart). Darker Shade of Magic is loved on Goodreads but middle-of-the-pack on Amazon. This goes to show how fundamentally different these audiences are. We shouldn’t trust either. Instead, I boost a book’s chances when it’s high across many of my different lists: if Uprooted is #2 on my Goodreads list, #3 on my Amazon list, #1 on the SFWA list, #1 on the Goodreads vote, #7 on my Mainstream Critics list, #1 on my SFF Critics list, etc., shouldn’t I predict it near the top? Throw in past Hugo/Nebula history, and that’s how the Chaos Horizon logic works; make what you will of it.
Later this month (let’s say mid-January) I’ll look to see what the ranking score is for each of these texts. Those scores don’t change much over time, so it hasn’t been worth tracking them month to month. I’ve also not found any correlation between the ranking score and award chances.
Let’s finish with a threat: I’ve gathered enough lists, 2016 is almost upon us, so I’ll make my first Nebula and Hugo predictions tomorrow!
Awards season moves ever closer! Right now, the Hugo and Nebula Awards for 2015 are still wide-open: what we have is disorganized sentiment, and that’s going to begin to get organized over the next 2-3 months. Some of the first posts about the 2015 Hugos are beginning to appear, including this excellent post from A Dribble of Ink. As more and more people begin talking about the Hugos and Nebulas, new contenders are going to emerge.
To help get those discussions perspective, I’m going to launch a new monthly feature of Chaos Horizon: checking on the popularity of the major Hugo contenders. Without further ado, here’s the chart, with all numbers taken from Goodreads as of October 31, 2014:
The chart lists the number of time each book has been ranked on Goodreads, with also the overall ranking. I think this chart is interesting, as it allows some of the differences in number of readers/reception based on these texts. While The Mirror Empire was embraced by critics on SFF websites, it’s been outread almost 75 to 1 by Words of Radiance. Sure, the Sanderson has been out for 8 months and the Hurley only 2, but that’s a staggering difference in number of readers. Also look at the score: 3.81 for Hurley, 4.76 for Sanderson. Have enough people read and liked The Mirror Empire for it to make the slate?
When you see the numbers like this, it’s clear how popular Words of Radiance and The Martian actually are, and how quickly writers like Scalzi or Mitchell are moving copies of their books. Based on number of ratings alone, Annihilation looks like a strong candidate. I don’t know yet how much raw popularity factors into the Hugo awards, but it has to be a factor, doesn’t it? I think by touching base with the popularity of these books every month, we can get a good idea of how often they’ve been read, and, in turn, how many voters can vote for them. That is, if you believe Hugo voters only vote for novels they’ve read . . .
About the Chart: One of the frustrations I have about the publishing industry is how secretive they are with numbers. The movie, music, and television industries are all relatively transparent about their numbers, and publish them regularly. We know within a few hours how well a blockbuster movie did at the box office, for instance.
For books—it’s all a deep, dark secret. The bestseller lists we have like NYTimes are calculated using obscure and byzantine formulas, and they don’t even release estimates of numbers sold. Bookscan covers a solid portion of physical book sales, but all those numbers are locked behind an extraordinarily expensive paywall. Publisher’s Weekly gives us some Bookscan numbers, but only for top-selling books—which often excludes SFF novels. So we’re left having to estimate popularity, either by word of mouth, blog traffic, Amazon sales ranks, etc.
I’ve thought long and hard about what the best measure of popularity might be, and I’ve settled on Goodreads as our current, most reliable measure of a book’s popularity. It’s not perfect by any means, and we don’t know the correlation between the # of Goodreads ratings to total sales (I’d estimate that at 5%-20%). What we do know is the following:
1. Goodreads has tons of users. For an average SFF book, there might be anywhere from between 1,000 to 50,000 ratings, and that has to represent a significant % of overall readers. Goodreads tends to have at least 10x the amount of ratings that Amazon.com does, for instance.
2. Goodreads doesn’t distinguish between electronic or print versions of the book. If you’ve read the book, you can rank it, no matter how. To my mind, that makes Goodreads even more reliable than Bookscan.
3. Even if Goodreads is somewhat skewed (the % of Goodreads readers is not 1:1 with the general reading public), it’s likely to always be skewed the same way. This makes for good “apples to apples” comparisons. Or, in other words, Goodreads is equally unfair to everyone.
A couple things to keep in mind:
1. The pool of Hugo voters is not the same as the general reading public. Hardcore SFF fans may like different things about novels than the more general pool of Goodsreads readers.
2. We don’t know how popularity correlates to the Hugo awards. I’ll need to collect data for a few years to begin to see patterns.
3. We don’t know if Goodreads is skewed towards certain authors. Most social media sites skew young, and books that appeal to readers in their teens and twenties may do better on such sites. I’ve got some ideas to figure this out, but it’s going to take time.
4. Goodreads only works for comparisons within the same year. More people join Goodreads all the time. Also, I can’t go back in time to measure how popularity worked in previous years; as soon as novels get Hugo or Nebula nominations/wins, that greatly increases their popularity, and we can’t know whether those novels sold well before or after they were nominated.
So, what do you think of this measure of popularity? Can it help us understand the Hugos or Nebulas better?