Why Amazon Reviews of Office Fitness Equipment Tend to Be Inflated
[Editors’ Update: Since we first published this article in 2016 an additional dynamic has surfaced that make Amazon user reviews an even less reliable indicator of product quality than before, and not just in the office fitness category. Many news reports have surfaced since then that describe “review factories” in China. Amazon sellers can buy artificial reviews from these dark web services to inflate their products’ ratings—and they’re getting away with it. This was discovered when consumers in the US began receiving serial shipments of products they never ordered, blind-purchased by these review factories only so that false ‘verified buyer’ reviews could be posted on Amazon. Not quite identity theft but they literally pick unwitting recipients out of the phone book.
Unfortunately, like all “fake news,” Amazon’s algorithms have a hard time spotting these deceptive tactics, and more and more users have been complaining on social media about poor experiences with products that had 4.5 or 5 star reviews on Amazon, and of receiving mystery packages of product they’ve never ordered. All this reinforces our own mission as an independent reviews site to conduct expert lab testing of products, and to provide readers with authentic user reviews that are not subject to tampering – i.e. review factories don’t bother with pure review sites, just ecommerce sites where their products can be purchased on impulse.]
The Veracity of Amazon Reviews on Office Fitness Gear
Customer reviews from verified buyers are an essential source of information for our staff when we’re getting a first look at a product. We generally disregard user reviews posted on manufacturers’ own websites, as these tend to be highly curated and don’t often include any seriously negative reviews. A better source of independent reviews are marketplace sites, such as Amazon.
Readers should note, however, that not all reviews they’ll find on Amazon are from verified buyers, and sometimes, verified buyers can be the manufacturers’ own employees or their competitors’ employees shilling out positive or negative posts to manipulate consumers. This is particularly true for new products with very few user reviews posted as yet. You might think Amazon enforces against these kinds of manipulations, but in point of fact, they have no such police force. Ideally, we’ll see hundreds of customer reviews on a product, not just a handful, if we are to ferret out consistent themes from verified buyers—whether positive or negative.
We have found that the most helpful reviews are 4-star because consumers will typically share at least one thing they didn’t like about the product that held them back from giving it a 5-star rating. The 1-star and 2-star reviews, unless there’s a good deal of them in proportion to 4-star and 5-star reviews, are typically from unreasonable customers or just unlucky ones who got a lemon or had a really bad customer service experience—so we take those with a grain of salt. In any event, the words Amazon users write are far more telling than the star ratings they give because there’s no published guideline for how many stars to give a product, and people tend to be highly subjective. Plus, a lot of people hesitate to “harm” a manufacturer on Amazon and don’t want to be hassled to remove their negative review; they’d rather assume their case of disappointment was a mishap and not the norm.
Rushing to Judgement
A phenomenon that we observe with great regularity with regard to Amazon reviews of office fitness products specifically is that many of them are posted before the merchandise even arrives, or after the first day of using the product, and are highly optimistic about how the product will transform the reviewers’ lives. In part, this is a psychological slap on their own back for doing something positive to improve their health, back pain, obesity, or whatever ails them, because they believe this product will help alleviate those problems. This is, in part, simply a lack of context. For example, if this is the very first, if not only, standing desk converter you’ve ever seen, and you really liked the TV commercial that drove you to Amazon to buy it, you might give it a 5-star review because you think it’s a revolutionary solution to your problems. On the other hand, if you’ve seen this very same product side by side with its competitors in a test lab, you might give it a 2-star or 3-star review. This happens all too often. In fact, it’s the rule rather than the exception.
These phenomenon are so common that an average 4-star Amazon review in the office fitness category often translates to a 2-star or 3-star review from our expert staff reviewers, and even a 4.5-star Amazon rating could translate to a very mediocre experts’ view of the same product. In contrast, 5-star Amazon reviews tend to be a much better indicator of a product’s quality, but even in this case, we find occasional exceptions, such as the Varidesk. While Varidesk’s massive television and print advertising spend helps float all boats in the industry, there are already more than two dozen competitors with products that offer more features and customer value but have very little consumer visibility. While infomercial buyers dominate the pool of 5-star reviews for the antiquated Varidesk design, our experts differ greatly in their views because they come at them from a more informed perspective.
Most People Were Raised to Be Nice—Perhaps Too Nice?
What we find particularly stunning is that the average rating given to a product that the user has so disliked that they returned it, is 3 stars. That’s right, a 3-star review can essentially mean “I hated it, but maybe I’m weird and you’d like it. In any event I don’t want to be mean to the manufacturer, maybe I had a lemon, it’s not their fault.” Many people also know from experience that if you give a product on Amazon a one or two star review you may get pestered incessantly by the merchant to beg you to remove it. That’s because it can kill their business, not only by depressing sales of that product, but due to the way Amazon’s machine learning algorithms operate, it can prevent all of that merchant’s products from even being shown in search results, which, frankly, can indeed be devastating to their business.
The bottom line is that we glean tremendously useful information from users’ writings in trying to broaden our observations about a product, but not much of value from their star ratings. While we’ve never seen a poorly reviewed product on Amazon turn out to be a great one in our own lab testing, we’ve seen a preponderance of over-rated products on Amazon from customers who simply have never compared the products side-by-side in a test lab, have rushed to post a review before even giving the product a good run, or are just too nice to say what they really think of it. And that’s why you’re reading our reviews—to get the straight skinny from people who actually test products for a living.
Learn more about how we review products at WorkWhileWalking & WorkWhileStanding by reading Anatomy of a Review.