Amazon’s Reviews

Image representing Amazon as depicted in Crunc...
Image via CrunchBase. Amazon & Reviews

A while back, I posted my thoughts on how to use reviews (as a reader, not a writer ūüôā ). Much of that post was devoted to how to tell if a review was honest. That is, how do you tell the honest review from a favorable one from the author or author’s friends? The New York Times has an article¬†on their¬†Sunday front page about Amazon’s attempts to deal with this. I’m heartened that they are taking the problem seriously although their efforts seem a bit clumsy. For one thing, if I ever put a book in Amazon will I no longer be able to review books? I guess there’s always Goodreads for my reviews, not that I post tons of reviews anyway.

But it is a real problem for Amazon. A few months ago they sent me one of their regular fliers with a list of Fantasy books worth a look. I was in the mood for some good fantasy and pulled up the first book recommended. The first one was 5-star rated: cool. Checked the reviews. Only four of them (bad sign). Read the reviews. They were all variations on the same theme. Clearly, every one of them was written by the same person, most likely the author. That was the last time I’ll pay attention to book recommendations from Amazon. One can only imagine that they had some tool automatically pull some high rated books from genres I like. If a human had read the reviews, it would have been obvious. So, for me, it’s back to more proven methods of finding good books (basically from friends and blogs I like, or as in my earlier post, from Amazon reviews when there are enough posted that it’s clear that many are from un-biased readers.)

But for Amazon, they just lost a way to market books to me, thus, their crack down on bogus reviews. While I don’t think their current method is the way to go about doing it (how are they really going to identify¬†shill reviews?) I support their intent. I want to read useful reviews.

However, I think the real way to address it is, essentially, to let the community police it. Allow the community to quash bogus reviews, as many sites do now in their comment section, and just as important, allow the community to identify reviewers worth paying attention to. Here, I mean not just reviewers that aren’t shilling for an author but reviewers who have a decent critical sense. Reviews by people who give everything 5 stars aren’t much better than reviews from a friend. With this in place and the ability to filter so I only see the good reviews, or least the vetted ones come first, reviews would suddenly be worthwhile again.

I hope Amazon’s fumbling¬†effort here starts a discussion that leads to some meaningful Amazon reviews. In addition to their ability to provide the desired goods, their real claim to fame is letting prospective buyers see what others think. So fixing this is at the heart of their business.

How to Use Reviews

I’ll confess I read more non-fiction than fiction, partly because it is so hard sometimes to find something I really enjoy on the fiction shelf. But from time to time, I’ll make a foray into the fiction world, usually in search of some good fantasy. Happily there is plenty to choose from with a little searching but my guide during these excursions are reviews. Yes, reviews are full of fake posts by authors, biased posts by friends, even annoying herd-think that can skew a¬†rating up or down. So where to start on reviews?

As a first step, ignore reviews if there are only a handful. Buy it if you like the cover, the blurb, the author or what have you but with only a few reviews there is really no way to trust them. They might be the work of disinterested readers but more likely they are at best a well-meaning friend of the author. The only exception, and it is still somewhat risky, is if the book has been out for some years and the reviews are scattered over geogaphy and time, ideally from real-name reviewers if the site allows that. In such a case you might find a nice gem of a book that was simply not marketed well. However, aside from the danger that the handful of reviews are baised, there is the fact that for whatever reason, it seems to take a lot of reviews to get a balanced representation of the book. That is, the good reviews out weigh the bad which leads into my next suggestion: poor reviews are usually more useful to the critical reader.

Why are negative reviews more useful? For starters, they won’t be written by the author or a friend of the author’s. While there is the risk they are written by a competing author or someone who just likes to slam other people’s work, they are more likely to be an honest review. That doesn’t mean you need to accept the review as definitive but a poor review will generally call out some specific aspect of the book that might be useful in making a decision.¬† It’s here that you’ll learn about shallow charecterization, weak plotting, deus ex machina, etc. Next look for consistent themes. If no one else calls out something, the reviewer might be off-base. A gushing review or a slam on the ending that no one else seems to pick up should be discarded as an outlier. That person might not like the genre, might be a terrible reviewer or simply may have had other factors in their life intrude on their experience of the book. If your wife just left you and you lost your job, who knows how you might react to a book and what you might write about it.

Allow for “high score bias”, meaning most reviewers seem to score fairly generously. Part of this is that one site’s 3 out of 5 might mean “okay” and another’s might mean “liked it”. But regardless, as a whole I find reviewers tend to be pretty generous. Gives me warm fuzzies about hope for humanity but makes it a little harder to find a good book.

Even more skewing is the herd mentality. It usually skews positive but can skew negative. You typically find it when something is heavily marketed because marketing works. If people are told the book is good by a number of authors they have read, they will assume it is good. Heavily-marketed books¬†tend to be better overall (after all the publisher is spending a lot of money to market it; they want to pick a winner) but hype does seem to affect reviewers and over-state the book. I’m sure there is a study on this behavior somewhere but I’m pretty sure it harkens back to that human desire to be part of the winning team, the in-crowd. So-and-so-said this book was good and he’s a big name author? Well, I liked it too! Isn’t it¬†great my opinion is the same as this important person’s?

How about the professional reviews you can find on Amazon before the reader reviews? For me, they are surprisingly useless. They rarely tell me much about the book. They may not be a paid review but they are generally part of a cozy publishing world. At the very least, beware of a bland review that might indicate the reviewer was more interested in not offending a publisher than writing an honest review. I’d skip them entirely and focus on reader reviews.

Look at the overall review distribution. If the review count is over 40 or so, the distribution of reviews will tell you something. Does a book have 600 reviews with no 1s and 2s? That’s almost certainly a strong book. It’s possible you won’t like the genre or premise but it ought to be at least well written and engaging. Are there some 2s and 1s with a notable number of 3s? Read through the low reviews. This is almost certainly a sign that the book has some flaws or weaknesses. They may not be flaws you care about but then again, they might be. It’s here that you will learn that the author’s worldbuilding is weak or the protagonist flawless (a “Mary-Sue”).

How to assess individual reviews? An ideal review will be balanced and call out plusses and minuses without being overly negative or positive. Balance and even-handedness indicate a rationale appraisal. Gushing positives or harsh slams are more likely to indicate a partisan viewpoint or someone who just likes to put others down. If a few reviews are about to tip your decision (pro or con), check the “other reviews by this reviewer” link that most sites provide. If this is their only review and it is positive, it’s probably a friend of the author if not the author herself. Or if the reviewer gives almost everyone 2s and 1s, discount them; they just like writing nasty reviews. Similarly, if they consitently give 4s and 5s, they aren’t a critical reviewer and should also be ignored; they are not giving you any useful information.

A few specific things to look for in individual reviews:

  • “I don’t usually read this type of book but I loved it.” This is the author’s sister, friend, spouse, or what not. Ignore the review.
  • Short reviews with no specifics: there’s not enough information for critical analysis of the review. Therefore, ignore it.
  • A lot of positive reviews posted in a short time, especially if they all tend to have the same tone and use similar phrases. Don’t just ignore the review, ignore the book. This is almost always a bad book being pushed by the author. Fortunately, this seems to elicit a lot of pissed off negative reviews that can help point this out, which can be fun to read (the reviews, not the book). Usually there are no more than 3-8 of these sort of reviews but I did find one book with 26 of them. Fortunately, they were clumsily written: all about the same length, all starting with a variation of “Simply put, this is the best fanasy novel I’ve read in 10 years.”
  • Be aware of spoilers. Most reviewers will note the presence of a spoiler but if you are still seriously evaluating the book (and not just reading bad reviews for fun), don’t wreck your reading experience by reading the spoilers.
  • The counts of people who found the review useful aren’t all that valuable for a specific review (since you can see for yourself if the review is useful). But that reviewer’s overall “usefulness” rating can be quite handy. A strong or bad review from someone who writes a lot of reviews and whose opinion is generally respected is worth consideration.
  • Comments are a mixed bag. You might find someone knocking the reviewer for being mean to the author (a personal attack is one thing, a well written negative review is another thing: that’s not an attack). Sometimes the comments can add some color to the review. Often not.
  • Multi-volume novels: read the reviews for the sequels. These are very telling as only someone who liked the first book will read a second; therefore you are seeing the reviews of those who gave the first book good marks. If you see the rating fall off or lots of mediocre reviews, it means the sequel isn’t as good as the first book, or just as likely that there are flaws in the first book that were overlooked; in either case, you probably want to think twice about reading that first book.¬†A classic case of this is a one-note book, one that strikes-a-chord with some readers: “The book has dragons? Cool! I love dragons. High marks to the first book.” But by the second or third¬†book the fact that the characters are shallow, the world building is weak and the author can’t drive a plot will come out.

To put it all together, look for a large sample of reviews since it is hard to fake a large number of reviews, beware of biases, sample the negative reviews to get some idea of flaws. If a few reviews are especially telling for you, check out those reviewers to see if they are someone whose opinion you can respect.

What this means is that just because a book got 4.5 stars, it doesn’t mean it is good. It also means that a solid 4, even a 3.8 is not necessarily the mark¬†of a book to skip. You need to sort out the fluff and find the reviewers who are telling you something about the book then base your decision on those reviews, ignoring the rest.

Happy reading…