I already talked about the problems with the “recent review score” on Steam here (http://www.over00.com/index.php/archives/3141) and I was actually preparing another post about how the review system on Steam is problematic and could be improved (like how a thumb up/down system doesn’t encourage nuanced reviews as in the end you just see the score based on thumbs and not pros/cons lists).

But before I could post about it again here’s Valve announcing some new changes to the review system (http://store.steampowered.com/news/24155/). “Great!” I told myself, they read my mind. Well, not so much…

Recent reviews score still there and more prominent

Not only you might have to deal with a recent score completely different from your overall score even if your game was not updated since release (March of the Living had a recent score of positive, mixed, mostly negative and very negative at different times even if the global score always remained “very positive”) but now the “most helpful recent reviews” appear right in the big section of reviews instead of just the side.

So basically, if you had a positive and very helpful review appearing first before, now it might be replaced by whatever review is marked as most helpful and posted in the last 30 days even if you didn’t change anything to the game. If you’re lucky this review will be positive and if not then you’ll have a nice big negative review showing first just because it was posted more recently.

This doesn’t reflect the state of the game at all. Putting focus on reviews posted in the last 30 days when the game didn’t change isn’t helpful at all. Most likely you’ll see reviews appearing there from people who got your game during a seasonal sale for cheap and that don’t really care about your game to start with. Worst, this might be a review from someone who already refunded the game… Don’t put discounts on your games you say? Well, this doesn’t work like that. For smaller games still able to make money from time to time discounts are essentials. That’s the reality.

Using a time metric for games that are exactly the same as before makes no sense at all to me.

Filtering out key activation keys from the score

Only keys bought directly on Steam will count toward the score now. I can see this was done to stop some people from giving away a ton of keys for free only to get good reviews improving their score but here’s the thing: As a dev I don’t care about other games’ scores, I only care about mine. There are enough people using Steam (millions) that if some other game has a better score than mine it won’t matter that much. If my score is bad though, it will matter a lot.

Because of this, MotL lost 26 positive reviews that were helping the global “very positive” score. 26 isn’t much you might think but out of 389 reviews it matters. These were not keys I sent telling people “I’m giving you the game for free but you need to write me a positive review”. These were keys sent to press, YouTubers and a few selected Steam users doing a good job writing detailed reviews (meaning they actually do some research and work in their review and yes some of them were negative even if they received the game for free).

This is also a problem for copies you sold directly on your own website (or other stores) or Kickstarter which are keys that were acquired without any requirement of posting a positive review. Sure, it might prevent a flow of negative reviews after putting your game in a bundle (but then you should know what you are getting into when participating to bundles http://www.over00.com/index.php/archives/3061) but when your game didn’t appear in any bundle it really doesn’t help you at all.

Changes particularly affecting solo or very small indie teams

I said on this blog before that you don’t need to sell 100,000 copies of your game to do okay when you’re a solo dev or a team of two. A modest success can be just enough to do well but these small changes to the review system only make it harder for modest success to remain successes.

MotL is such example of modest success with “only” 389 reviews. My publisher made his investment back in only 3 days after release and it made me a salary for 2016. Here’s the thing though. Even if it’s a success, the smaller variation to reviews have a big impact when your score is based on 389 reviews. A negative recent score has a big impact on sales even if your global score is still very positive as I could see during the last Summer sale on Steam.

If your game has 10,000 reviews then chances are that you will barely notice the changes and your game already being a success it means that it’s regularly promoted to key spots on Steam so it helps to negate possibly bad reviews (MotL appeared on the front page during the Summer Sale this year and we saw a boost of sales despite our bad recent score). For example, No Man’s Sky might have a mostly negative score on Steam but I can assure you it will receive more spotlight than my own “very positive scored” game in the future because it sold a ton of copy.

So when big games badly scored still receive more spotlight than smaller games well scored, these changes to the review system only screw up these small games as your score might suddenly shift. Makes you wonder what is the point of the review system if Steam still promote badly scored games anyway. On one hand they want gamers to be “informed” but on the other they are telling you “hey look, it might be bad but it’s in a big capsule”.

So yeah, my game is better scored than No Man’s Sky but so what…

Reviews for refunded copies are still mixed in the lot

They take out reviews from key activation that you might have sold in a legit way on Humble Store or your website but not reviews from people who refunded your game. Why not apply the same logic here?! Leave the reviews there like key activation but don’t count them in the score.

When you can instantly refund a game after 2 hours just because “it wasn’t fun” based on your own taste, then games essentially become demos. Even if there were no reviews at all on Steam, players would be safe from any harm as they can get their money back almost right away for any reason that makes sense to them (literally any reason).

Game doesn’t launch? No harm done, you get your money back. Game crash? No harm done, you get your money back. You got bored after 15 minutes? No harm done, you get your money back. You suddenly changed your mind? No harm done, just select “bought by mistake” (?!?!?!?), you get your money back. I mean, in any scenario the player is covered, get his money back but devs are always screwed by not making any money and being stuck with a negative review counting toward the score anyway…

Maybe leave the refund reviews there but at least don’t have them count more than legit copies you are selling on your on website as the score matters a lot for smaller games. At least now we know to not bother with direct sales as the score on Steam matters more than making a few more dollars by direct sales (as your score might help exposure on Steam which will eventually make you more money probably than direct sales).

Tiny suggestion to help a bit

It’s not perfect but instead of simply showing first a negative review that is marked as the most helpful review when the score of the game is still “very positive”, why not show side by side the most helpful positive review as well.

How is it making sense that a well scored game have a negative review showing first with nothing to counter it. In a binary system where you can only say “good or bad” (the thumb system), the least you can do is to show the two sides of the medal.

steamReviewThumb

Share