How scared should you be of 1-star app reviews?
It’s well established that a high average star rating is an important factor in getting featured or ranking in the top charts in the App Store — Apple really wants to feature apps that users love. Here at Appbot, we are often focussed on helping customers who are getting a large volume of negative reviews shift their average star rating upwards. As a team we’ve all dealt with the fear of negative reviews either for our own apps, or those of the businesses we’ve worked for, but is the risk of a deluge of negative reviews as real as it seems?
We were keen to figure out whether there’s a relationship between review volume and average star rating — do users prefer to leave positive feedback or is the perceived risk of negative reviews in the app store real, with many apps that get low average stars (2 or lower) also getting a big volume? If you release a version users don’t like, are they as inclined to hate on you in reviews as they are to sing your praises when the opposite happens?
We guessed that apps at each end of the average star rating scale would get more reviews than apps with 2.5–3.5 average stars, but the results surprised us. Here’s the details of our study…
We analysed 10,000 apps from our database that had received at least 1 review in the iOS App Store in the past 30 days. Here’s how the average star ratings were distributed:
Hmm, OK. The count of apps averaging 1 star isn’t as high as we’d guessed it might be — the count of 3-star apps is almost identical. Having said that, we’ve rounded to the nearest whole number in this chart, so apps with an average of 1 had an average of between 1.0 and 1.49, whereas apps with an average of 3 had an average of 2.5–3.49. With that in mind the count of apps getting 5 is definitely higher than we expected — these apps averaged 4.5–5.0. Next, we looked at how the average star rating stacked up when we added the review volume into the mix. We plotted each app as a point on this scattergraph:
As we mentioned earlier, the sample size is 10,000 apps. The number of reviews per app over the 30 day period ranges from 1 to 12,262. The average count of reviews per app is 129. The average star rating per app is 3.19 stars.
Happily, there is a strong correlation between number of reviews and average star rating: the apps with a higher review volume also trended towards a high average star rating.
Not only that, the apps with crazy-high review volumes were significantly more likely to have high star ratings. In fact, there was only a single app with <2 star average and more than 5,000 reviews in the period.
Compare this chart to the earlier one referencing the whole sample — note that in this sample of apps that received >1000 reviews (296 apps) in the period there were ZERO that received an average 1 star rating (1.0–1.49), and only 6 with 2 stars (1.5–2.49). If we look at a scattergraph of only those same 296 apps this is what we see:
There are a few other apps on this chart that are really suffering (marked Ouch), but they’re in the overwhelming minority. Without naming names, these apps are getting so much negative feedback from users because earlier versions were loved by users but the company has removed a much-loved feature or significantly altered the UI, or because they support a very large (predominantly offline) business and fall short of that business’ customers’ expectations, or both. It seems that runaway volumes of negative reviews is really only a major risk if you have a large customer base, and you’re not paying attention to what those customers want.
The results make a lot of sense when you think them through. Here are a few of the reasons we believe that apps getting high volumes of reviews tend to get higher average star ratings:
People only really tell others about apps they LOVE
When was the last time someone said to you “Hey, you’ve gotta download this really crap app!”? That’s right, never. Large volumes of positive reviews create a snowball effect — positive word of mouth offline, positive word of mouth online. Large volumes of negative reviews are only really a risk if you already have a huge customer base, and you’re either falling very far short of their standards or making a change to your existing app that they might hate.
Conversely, positive feedback has a snowball effect. Josh Pigford (@Shpigford) from baremetrics wrote a really great piece that highlights this — highly recommended reading.
Positive feedback culture
At the risk of sounding like my Mum, social media really focusses on the mantra “If you can’t say something nice, don’t say anything at all”. There’s no “hate” or even “dislike” button on Facebook, Instagram, Pinterest, Twitter…. We’ve been conditioned to engage positively, and it takes serious aggravation to make us behave otherwise.
Smart developers only ask for a review when the outcome’s likely to be positive
Smart review workflows are becoming more commonplace, and if you’re not already using them you might be interested to check out our Chief Bot, Stuart’s, post on Medium recently about his experiences with 7 Minute Workout app — they can have a massive impact on your average star rating. Smart review workflows focus on only asking for a review when you’ve qualified that the potential reviewer is feeling positive about your app.
Once bitten, twice shy
If you’ve been unlucky enough to be product manager for one of those apps that’s had a ton of bad reviews, chances are you’re not pushing as hard to attract more reviews as you would be if the majority of feedback was good.
In summary…
Smart people are always cautious, so it wouldn’t be reasonable to expect all of us to cast our 1-star review phobias aside as a result of this little study, but the evidence seems to suggest the risk is lower than you might think. Exercise extra caution if you have a big user base already (not just in the app itself — if you have an existing business with a large customer base that you’ll be promoting your app to then you should also be very careful, and always test the response with a subset of customers first). The other encouraging thing to come out of this little study is that if you’re getting positive reviews, you can probably push harder for a greater volume of reviews without too much fear that your average star rating will fall, so if you’re already getting a lot of star love get cracking on those social shares and review workflows and watch the downloads flow. :)
If you enjoyed this, please hit share on your social networks of choice. We’d love to hear your feedback on this data so please tweet your comments to @AppbotX.
