So, I was nosing around the Internet, as I often do, and I came across an interview with Larian Studios that really struck a chord with me.
I genuinely feel that it's worth a read.
What's happening here is that advertising companies are dominating reviews, and scores can be affected by that. Moreover, sometimes, if you don't provide a magazine with enough 'favours,' they may call in an intern or just hand a game review over to someone who'll 'work cheap.'
The problem is is that they know that they can get away with it, due to how much Metacritic and the big review agencies dominate opinions. As the interview points out, there are PR companies that believe that sales live or die based upon Metacritic scores.
I was immediately reminded of a recent news article here on VG24/7 about how New Vegas didn't hit its bonus because it only reached 84 on Metacritic. And it needed to hit 85. And that was a crying shame, frankly, because New Vegas is one of the best RPGs I've ever played.
What's interesting is that a lot of the negative reviews I checked of New Vegas were BS. They highlighted 'bugginess' without ever saying what those bugs were. And some reviews, like Quintin Smith's over on Rock, Paper, Shotgun, even falsified screenshots just to be exceedingly negative.
This was picked up on by the comments thread, a comments thread that went to roughly around 10,000~ comments before it was cleaned up. There was a lot of anger about this, and many people saying 'did he even play the same game that I did?'
The problem here is that you have people harping on about games based upon reviews, when reviews aren't very factual or often even very trustworthy. You have to be careful of whom you trust, and often I think it's better to buy from a developer you know you like, or buy based upon demos.
Reviews seem to be becoming increasingly ever more inaccurate as time goes on.
The solution posed by the man at Larian being interviewed was an absolutely fascinating one: A review site for reviewers, where you can grade a review based on its accuracy. Sort of a Metacritic for reviewers. So that the people who're often rated as being the most honest and objective float to the top, and you have a better idea of whom to trust.
It's like crowd-sourced truthfulness checking, and I think that's a great idea. I think that's an idea that should be turned into a reality.
Perhaps publications may be less greedy if their publication suffers for it. I mean, if a review causes the overall per centage of their reviewer and linked publication to drop, then they're going to want to sort that out. I found it thought-provoking, anyway. I thought I'd share.
The reason we need something like this though is because there are far too many people who'll harp on about what they read in a review without even having played a game themselves, or without having done any fact checking. And that can be infuriating for the people who have played it - and genuinely depressing for those who developed it.
---
Oh, I forgot to note...
One of the reasons I brought this up is because of how worrying it is.
Consider that when a game gets a bad Metacritic score due to a number of greedy publications, it can kill a small studio. If you're an independent, small developer, it can just finish you off. And personally I don't think it's right that just because you can't fly a group of reviewers out to somewhere fancy to show off your game, you'll get less of a score for that.
You know this has been rattling around in our heads for a while now, and here it is right from the mouth of a developer. I simply had to bring attention to this because it is a worrying trend to set. It means that more and more we're letting big business shut down the competition.
Who needs SOPA?
And yeah, I think that's terrifying. I just think that we really need something that's going to create an attitude adjustment in buyers, so that people won't blindly obey Metacritic. And this is what brought up the idea of a site which reviews the reviews. We really need to shed some light on all the bullshit, here.
And, frankly, with games journalism the way it is now, there is a lot of bullshit.