Sections

The Great Review Debate: Can we find our way back?

Tuesday, 15th November 2011 13:22 GMT By Patrick Garratt

This holiday review period has brought us “h8/10,” journalists griping over review code and the concept of pre-ordering pushed to the limit. How did we ever get to this point? The publishers have the answers, says Patrick Garratt.

Put your feet in the shoes of the PR and marketing departments at a large publisher. You’ve got your game, and you’ve worked on it for about a year. You’re just as aware of the weak areas as you are of the strengths. You know there are elements sure to be picked on by journalists and you’re expecting 8s, the odd 7 and a few 9s. The developers and top brass are expecting 12s.

The holiday season brings with it the year’s biggest game releases, and some kind of review controversy is always in tow. This year, however, the entire process of reviewing games has reached an impasse as the industry polarises between triple-A and “everything else,” with sky-high expectations on certain products making for farcical situations surrounding scores.

In addition, journalists have been constantly griping, both behind the scenes and on social networks, that exclusive reviews and deliberate stalling of code delivery is creating a difficult situation both for those buying games and those seeking to have a review live for launch day.

To understand why game reviews are handled in the way they are, and why publishers seek to exert so much control over when reviews go live, it’s necessary to face a few facts.

Firstly, a games publisher’s entire reason for being is to promote and sell video games. Once you accept this, all of the behaviour you see surrounding reviews from journalist, developer and publisher perspectives becomes crystal clear.

Secondly, publishers will do anything they possibly can to counter the cancellation of pre-orders. Pre-orders are vital for the success of any bigger game these days, before word of mouth and any negative reviews have the potential chance to stop people buying it in the weeks following release.

And thirdly, we have the continuing presence of Metacritic as a metric for creative success.

There’s no fourthly. Let’s take a look at some of the aspects of game reviewing journalists and gamers constantly moan about and explain why they happen, taking the aforementioned truths into account.

Exclusive!

As it’s the first one you’re likely to see, let’s think about the exclusive review. This is arranged between an outlet and a publisher. The publisher will never agree to this if there’s a risk of a low score, for obvious reasons. That’s not to say the publisher will demand a certain rating; the publisher in this instance, and the outlet involved, will be well aware of what it means to have this happen. You’re looking at a 9 or higher here.

An exclusive review for a multi-platform game is normally reserved for mid-spend games from third-parties. Dark Souls is a topical example. IGN went live with an exclusive review days before the general embargo. It got a 9. The rest of the world gave it a 9, too, so there was no danger there, and IGN did a 24-hour play of the RPG leading into the review, which was probably part of an overall deal.

Why would IGN want to do this? Traffic, obviously. Why would the publisher want to do this? Positive promotion of the game directed straight at the people most likely to buy it. Everyone searching Google for “Dark Souls review” in the days before the general embargo lift would land at IGN, and would see a high score. To enter into this agreement both parties would have to be confident that the game was of a very high quality. Keza MacDonald wrote the review; she was a big supporter of Demon’s Souls and, as I’m aware of first-hand, wanted to play Dark Souls more than being able to ingest food for most of this year. The deal worked for all parties.

Exclusivity in this way doesn’t have to be limited to a single site. Publishers may offer regional exclusives, print exclusives, online exclusives or platform exclusives. They will try to work with people they trust and journalists they know appreciate the product. The reason for doing this is always the same: it’s so the first critical contact a consumer has with the game is a positive one.

The Dark Souls exclusive is an instance of a deal working for everyone, very much including the consumer. It doesn’t always. It’s likely that larger sites wouldn’t enter into something like this if they knew the game wasn’t up to scratch, but a publisher is still likely to try. And if the larger sites don’t want anything to do with it because it may damage their integrity, there are plenty of other outlets that will get on board because they need the traffic or magazine sales. The danger is obvious: you get a skewed score in return for exclusivity. This is why people pull a face when an official mag comes out first with a review and gives a very high score. Magazine covers are even more vital to the survival of the publication than exclusive reviews are to websites. If publisher PR can get a decent cover with a good score and agree on an exclusivity window, all parties are likely to agree. There’s a grey element here, though, and people aren’t that stupid.

The comments are always the same: wait for the general reviews.

Where’s my code?

Which leads us on to the general reviews themselves. Again, to understand why this process works in the way it does, it’s important to think like a publisher. The review element of a game’s life-cycle is only a part of its customer-facing promotion, and is the most difficult for the publisher. Note I said “promotion”: if you’re a publisher, the review is not a completely unbiased appraisal of the game; it’s a chance for you to sell more copies.

Put your feet in the shoes of the PR and marketing departments at a large publisher. You’ve got your game, and you’ve worked on it for about a year. You’re just as aware of the weak areas as you are of the strengths. You know there are elements sure to be picked on by journalists and you’re expecting 8s, the odd 7 and a few 9s. The developers and top brass are expecting 12s.

First of all, you create a pre-order campaign. You’ll have to judge the level of fandom here. If the IP’s got a rabid following you can go large, as Activision did with Modern Warfare 2 or Namco did with The Witcher 2. Big package, high price, silly excitement, large mark-up.

If you’ve got something a bit newer, you might offer a lesser, cheaper premium package which is designed to make people think, “Why not? It’s only an extra tenner and I get the blue multiplayer jumpsuit of Quarg on day one!”

When you pre-order a game you are buying it based on an image that is completely controlled by the publisher. All you’ve seen of the game at this point is some screens, a few edited trailers and maybe the odd interview. This is why publishers do it. It’s cash in the bank based on the best possible perception of the product.

Your pre-orders have gone well. Execs are saying things like, “Gungo-Fuckers 3 is tracking well ahead of expectation in qwarder four” in fiscal calls; their optimism is based on the level of pre-ordering generated by PR and marketing. Loads of pre-orders, guaranteed great initial sales, happy investors.

Then, though, you have a problem: you have to stop people cancelling their pre-orders for any reason. This is vital for the publisher, and it will usually do everything possible to protect the game from being shown in an uncontrolled way before release. Pre-orders really can make or break a game in today’s market. Publishers are in the business of “make”.

As an example, try going to somewhere like the Eurogamer Expo or PAX and filming a game off-screen. You’ll be immediately told to stop. Why? Because the publisher doesn’t want the footage appearing online at a crap resolution, and risk damaging the perception they’re trying to foster that Gungo-Fuckers 3 is the greatest piece of software ever committed to disc. Some publishers are fiercely protective of footage appearing on YouTube, for example, and will have any videos removed instantly.

Gears of War 3. Did Eurogamer really “h8″ it?

See how important pre-orders are if you’re a publisher? And if unauthorised online footage presents an issue for pre-order retention, reviews are a absolute nightmare which must be controlled. Honesty, when you’re trying to sell something, may not be the best policy.

Which brings us onto the second favourite gripe we see with reviewing: the timing of review publication and the availability of pre-release code. There’s a perception that anything below a 9 for a core title isn’t going to cut it, especially in the triple-A space. If you suspect scores may fall below that, you have to manage publication timing. If a load of 7s go live a week before the game’s released, there’s a good chance a ton of your pre-orders will cancel before they ship. And that, as any publishing executive will tell you, can’t happen.

You manage this in a few different ways. Firstly, you refuse to allow outlets review code unless they sign an NDA saying they won’t say anything about the game, at all, anywhere, until launch day. These NDAs now include clauses about Twitter and Facebook: if you sign, you really can’t say anything in public about the product itself.

With the proliferation of social networks, we’ve seen an explosion of people tweeting images of the review code disc itself, or the press kit, or whatever else, with a note saying they’ve signed an NDA and can’t talk about the game. As a reaction to this, the most recent addition to the NDA we’ve seen is the “super-NDA”. This NDA stops any mention of the fact an NDA has even been signed. Once you’ve got a journalist with their name on this sucker, he’s yours until embargo.

Secondly, and even if you’ve put NDAs in place, you send the code at the absolute last minute. This cuts down the risk of any media or impressions leaking before the cut-off for pre-order cancellation. VG247 staff regularly have to sign final code NDAs, despite the fact we don’t publish our own “reviews”. We do, however, capture video footage, do voiceovers and write opinion pieces on code, all of which could have exactly the same effect as a review score.

This is why you get reviews on launch day, and not weeks before so you can “make up your mind” as to whether you might want to buy the game. The only decision the people involved in creating, marketing, PRing and eventually selling this game want you to make is that you should buy it. Think like the publisher and it all becomes clear.

I still know journalists who are baffled as to why they haven’t received games in good time to have a review published for launch day itself, or even to be live well before launch, so people can make an informed purchase choice. Ask yourselves these questions: do you work for a popular publication? Are you known for “honest” reviews? Could you affect week-one sales and even pre-orders with stand-out negativity?

That’s why you don’t have the game.

And before you all start having a little fanny about any of this, get some perspective. As a journalist, you are not “entitled” to be given a company’s property before it goes on sale and potentially wreck its chances at market. Publishers do want reviews up for launch because it’s a norm in the games world and looks suspect otherwise, and they want scores live for consumers looking to make their minds up on launch day, but they need high scores. Some games may have a huge amount of reviews live on embargo, some less so. This is the publisher manipulating the review spread, working with publications it believes will deliver good scores for launch. Why are you remotely surprised by this?

As a gamer, no one is forcing you to pre-order any video game, or even buy it at launch. You are being marketed at. It’s your money and you can do what you like with it. Publishers buy ads on sites like VG247, IGN, Gamespot, CVG, Eurogamer, Destructoid, Kotaku and all the rest because they command large audiences of video game enthusiasts. That’s you. You probably already want to buy the game. You want to be excited about it. You want to talk about it. You want information on it. You want to know the release date and what you can get for pre-ordering. You’re a perfect target for “enforcement” marketing, and, in all truth, you’ll probably enjoy the game in question regardless of whether or not it gets an MC average of 72%, 84% or 92%. You’re a hardcore, knowledgeable hobbyist and you’ve got a good idea of what you’re getting yourself into when you hand over your cash.

You have a choice. If you’ve been bitten before or you’re just more careful with your money, wait until the game’s launched, do your research, find some reviews published after the embargo, let the game get patched a few times and then buy it (or not, as the case may be). You’re allowed to do this. You are an individual. This is why you have a brain.

h8/10

This situation is not exclusive to games releases, but exists with product reviewing in any sector. In the eyes of the publisher, we’re part of the process of marketing video games, and this is why the entire foot-stamping spectacle from journalists and gamers is so ridiculously naïve. Did you see a review of iPhone 4S before it released? Of course not. Be Apple for a second: why on earth would you take the risk? Why do you think Apple went so beserk over Gizmodo’s iPhone 4 exclusive in 2010? It’s because early, uncontrolled access to products can impact sales. Nothing more or less. Exactly the same thing is true of any marketed product, and that includes games.

What is new this year in the games industry, though, is the level of expectation on certain products producing worrying attitudes around scores. Both of the most acute examples of this have been brought on by Eurogamer reviews this autumn, the first for Gears of War 3 and the second for Uncharted 3.

If you’re a review journalist you can pre-order the game yourself, get it on launch day, play it, write a review and put it on the internet. Radical, I know, but it can be done.

Eurogamer’s Johnny Minkley gave Epic’s latest 8/10. I interviewed Cliff Bleszinski the day after the review embargo lifted, and he jokingly said Eurogamer was staffed by “haters” as a result of the score. He went on to say he believed the third game was the best in the series and was “upset” that EG had rated it lower than Gears of War 2′s 9/10.

I can’t speak for Cliff, but I believe he only used the word “haters” because it’s internet-speak. Regardless, though, he was clearly annoyed by it. In his eyes, an 8/10 simply wasn’t good enough.

Following from this was Simon Parkin’s 8/10 for Uncharted 3, again on Eurogamer. This really did kick the hornets’ nest. Jim Sterling’s latest video for The Escapist has a look at what happened on sites like NeoGAF in the wake of the review being posted, and even David Jaffe threw his weight around in support of Parkin, such was the severity of the pushback.

Sterling himself has been at the centre of this type of hysteria this year, after giving Batman: Arkham City 8.5/10 on Destructoid. Watch the video to get a steer on the type of reaction this drew from gamers.

The reason gamers get so angry about a game they’ve been waiting on for the past year is obvious: they’ve been invested for months, and they want to see a 10. Why? Because they hope the game they’re about to play is “perfect”. Anything below a 9, especially for one of the big platform-specific games, is now viewed as a low score. Oh, the irony: back in the day, we used to combat the notion that 7/10 was an average rating, when it was, in fact, 5/10. Giving a game a half-mark now is akin to saying it’s only worth buying if you need something to use as a firelighter in the cold winter months.

The situation has clearly reached an absurd level if scoring a game at 8/10 is deemed unworthy. How did we get here? Again, think like a publisher for the answer.

Metacritic is pretty much the only hard measure of critical success for videogames. It is used by all levels of the games business – including the media – as a metric for whether or not a game is “good”. Publishers like EA base entire strategies around lifting Metacritic scores across their portfolios, and investment execs like Michael Pachter routinely quote MC averages as a rule of accomplishment.

Pachter tells people where to put huge amounts of money.

This need for a high MC score by senior executives at publishers pushes down on entire corporations, right down to PR level, where they interface with the media. And it’s this desire, this need for very high Metacritic marks that has led to a culture where games that carry sub-9 scores are no longer seen as true hits. And everyone is now aiming for true hits in an environment where disc-based, double-A games are being mercilessly slaughtered. In turn, the gamer community now expects the likes of Battlefield 3, Modern Warfare 3, Uncharted 3 and all the other threes to be 90-plus by default. When they’re not, we see a great deal of friction.

This is, of course, a completely manufactured situation based on corporate politicking. The result is that we may have allowed ourselves, as an industry, to arrive at a point where reviews scores are essentially meaningless. If an 8/10 doesn’t say a game’s good, what does that say about the field of game reviews as a whole?

Fortunately, the situation’s not hopeless. The media doesn’t need to be part of the publisher’s review process. If you’re a review journalist you can pre-order the game yourself, get it on launch day, play it, write a review and put it on the internet. Radical, I know, but it can be done. Maybe your readers will trust you more if you focused on reviewing the game after the fact, in a control-free environment with all online modes working in the wild, instead of working by the publisher’s rules, attending “review events” and insisting on having a launch-day score.

If you’re a gamer, and you’re seeing an 8/10 as a h8/10, maybe you should just get some perspective. Why don’t you play the game and see what you think? You never know: you may just enjoy it despite the number and learn to find reviewers that work for you instead of screaming into the void over arbitrary metrics.

And if you’re a publisher? If you’re a publisher, perhaps you should stop placing such a ludicrous emphasis on review scores and concentrate on making the best games and marketing them in the best way. Focus on excelling instead of controlling. There’s been a lot of talk about gamers calming down over scores, but you’d be amazed at the sort of reaction a “bad” score can cause at a publisher. Maybe, Mr Publisher, if you weren’t trying to constantly balance the need for early reviews and high day one results with a façade of press independence then none of this would matter so much. Maybe having a day one review isn’t so important after all.

Maybe. See you this time next year.

Breaking news

78 Comments

Sign in to post a comment.

  1. Zidane

    Excellent read! I’m a reviewer myself and I’m very much aware of how publishers want their games to be given the best scores possible. I’ve even had arguments with two publishers in the past about two of my reviews. I regret nothing and I will never be influenced by any company to give a game a “good” score.

    #1 2 years ago
  2. BULArmy

    Absolutely fantastic article! I hate those scores and mostly the scores with decimals added and those with %. What is the difference between 8.6 and 8.7 or between 83 and 84. On what bases you take and give such meaningless numbers. Nobody tells us and when you watch a Gametrailers review or read other review with such scores, for a big game and after that a histerical cry why certain game got 9.3 and other got 9.4. Absurd, we need to loose those meaningless scores or just confine tham to grading used in school, either the US letter based or the EU 1-5/2-6/1-10. That is why I mostly read reviews from sites without scores like those on Kotaku or Rock, Paper, Shotgun and those with solid numbers, but sometimes I don’t even look at the score just read the rewiev, there is the truth and those words will help me decide to buy or not a certain game not that stupid score.

    #2 2 years ago
  3. Joe Musashi

    I find it disingenuous to put the burden of responsibility on such reviews onto gamers and games publishers. I’m not saying they don’t play some part though.

    I think very few regular readers of sites like Eurogamer or Destructoid would have their opinion of a forthcoming game completely changed by a review. A point or two above or below their expectations probably won’t make that much difference. A game turning out to be a complete turkey, rated as such from a number of disparate sources – yes, that’ll make a difference.

    What I find hard to fathom is just who these writers/journalists are supposed to be writing FOR. Who is their intended audience? Because, in this harsh world, a website is a revenue earning product. An article is part of that product. Are you servicing your audience properly or are you pandering to a different party altogether? If you are not producing with your audience in mind then do not be surprised when your audience lets you know their dissatisfaction.

    There is an argument concerning consistency too. If you are criticising a product for being non-linear and following a formula franchise then those criticisms (should) stand for other products. If you are marking one product down for those reasons, why are you not doing so for other products? Why are you reserving this criteria for higher profile products instead of all that come under scrutiny? Where is the consistency? What is the barometer of professionalism that your readers (who may or may not be your ‘audience’) are expected to use?

    These questions typically go unanswered and unacknowledged. Instead of reassurance of integrity responses that imply it is wrong to question one product (an article) but it is okay for an author to question another product (a game). Again – who is being serviced? Why this typical wall of silence?

    There are far far better ways for such disputes and ambiguities to be resolved than with dismissal to the tune of “trust me, I’m a game reviewer”. If games journalism and games reviewing finds itself with a questionable reputation then being aggressive or dismissive about it will not improve the situation. Far more transparency is required for questions of professional integrity to be properly addressed.

    Using the full range of numbers from 1-10 would be a start.

    JM

    #3 2 years ago
  4. Hcw87

    The game in the headline (UC3) did not deserve all the praise it got from IGN atleast (10/10 on everything, come on). It was good, but certainly not perfect. Actually traded it in after completing the very short campaign (and it got a 10 for longevity for god knows what reason).

    #4 2 years ago
  5. BULArmy

    As JM said we have huge problem with consistency in reviews and this is the worst problem of all. How can the not so original 6h SP of a certain game get a 9 and the almost same SP in other game get 8. Also abysmaly high scores for story for games like Halo and than much lower score with excelent story – Metro 2033.
    Also big problem with huge releases and small releases. We can rarely see a 90+ indie game with a lot of innovation and clearly better experience, but certain cliche “aaa” get those 90+.
    And lastly platform bias in the reviews. If you look at certain multi-platform you will see 360 to have tha biggest score and in not all the cases it is the best version. You look closely who rewiev tham and we often see 360 only magazines to have the highest score for the games and is a problem even with mediocre games for which mags like OXM give a 9 or a 10, but most of the other reviews are 8s.

    #5 2 years ago
  6. DSB

    Nice one, Pat.

    Perspective equals: Publishers need money, reviewers need games, and gamers are hysterical little girls that want to decide on games before they’re out, to justify the ritual of mindlessly throwing money at them.

    I couldn’t disagree more with the Dark Souls example though. Reviewers being handed titles that they’re already in love with poses a problem more than an advantage. The review might please the fanboys and tell them what they want to hear, but it probably won’t be an outstanding example of media criticism, which is what is so badly needed these days.

    Games writers are so busy with themselves, you’d think this would get more attention. If you want “the medium to be taken seriously” as some writers so badly need it to, then I think the entire infrastructure should reflect that.

    That requires not only a more civil relationship between publishers (who in some way since the 80′s have been trying to dictate to their developers as much as the games media) but certainly also the kind of ambition and dedication that I was pretty happy to find towards the end of the article.

    Because it’s true. The reviewer, the editor and the outlet can act as independently as they choose. The radical thing is letting yourself be spoonfed by publishers.

    #6 2 years ago
  7. somberlain

    Yup excellent read. It’s crazy how video games became “all good” or “garbage”, with no in-between.
    What’s crazy is that, now, all “AAA” games are very highly rated.
    I’m playing Gears of War 3, and for me the game is a clear 7ish, great graphics, solid gameplay but nothing “over the top” (walk 10 meters, arena with 20 enemies spawning, kill, walk 10 more meters, other arena with 20 enemies,etc…).
    Uncharted 3 would be for me a solid 7 – great visuals, but horrible gameplay (enemies IA is horrendous, idiotic spawning of enemies behing crates coming in waves, and the story is really weak compared to the second episode).

    It really seems that today, everything is about the hype, and how “big” it is, rather than the actual game itself and the pleasure you have playing it. Sad times…

    #7 2 years ago
  8. jedieagle

    Funny thing is, the publishers are trying even to influence a magazine in Slovenia, which is a small country. They offer them a game to review in advance if it will get a great grade, but they said no. And the review was in a later issue. The really funny thing, the game got 89 :D but they dont do such deals :D so respect to them.

    @4 longevity probably included multiplayer aspect of the game :)

    #8 2 years ago
  9. YoungZer0

    @8: Same thing was reported by COMPUTER BILD SPIELE. A German game magazine. Ubisoft wanted good scores for AC2, for that they’d get the game earlier.

    They refused, instead decided to write an article about it. When the game came out they gave it a high score, because it deserved it.

    #9 2 years ago
  10. TimClark

    Strong read, and a timely one – I can well understand why some reviews look baffling, they certainly look baffling to me, and there’s definitely a wider, longer debate to have about consistency on individual publications (the editor deciding who reviews what has a major part to play, as your example of Keza, who was predisposed to love Dark Souls, suggests.) Likewise, the ‘how long is long enough’ to review something is the submerged part of the review iceberg, and only exacerbated by late code delivery combined with the imperative to have your publication in the mix when the embargo lifts.

    Certainly, the drift from 7 being a bad score to 8 being one has been as depressing as it has been seemingly inevitable. I actually recall, back in our day, giving an exclusive cover review a 7/10 on OPM and the shitstorm that resulted. The only point I’d take issue with is that print mags, official or otherwise, rarely do exclusive reviews these days. That’s partly due to the sheer ball-achery involved, because publishers are as you rightly explain, so skittish. (In fact, one top three publisher told us they wouldn’t be doing exclusive reviews anymore, because it was creating too much stress and bad blood for all concerned. I felt nothing but relief and agreed it was a good decision.)

    The biggest reason we don’t do many in print, though, is predictably a commercial one. They simply don’t sell. (And sure, you might argue that’s because readers are wise to them, but I don’t think it’s that.) I first really noticed the appetite was gone when we had exclusive reviews – genuine worldwide ones – of GTAIV and MGS4. Both sold *much* less than covers we’d done at first look and hands-on stages. The conclusion, again in keeping with what you say, is that hardcore gamers are making buying (specifically pre-ordering) decisions far earlier in the life cycle of the game. For us, forward looking covers, essentially stuff to be excited about, work much better.

    I could write a million words about this stuff. For various reasons I shouldn’t. But in the ten or so years I’ve worked here, I’ve listened to what must add up to actual months of people passionately arguing about how to improve reviews. I can only speak based on my experiences, but the reviewers I know (here, and at other pubs) by and large care deeply about trying to call things right and not let down their readers, while operating at times under incredible time and workload pressures. It’s tricky to see the major softies changing the way they act (and react) anytime soon, given the pressures they face, but a less hysterical approach from all concerned would surely be of benefit.

    I just realised I typed that on the internet. Jesus. 7/10.

    #10 2 years ago
  11. OrbitMonkey

    Good read that 8.5/10

    :)

    #11 2 years ago
  12. somberlain

    While browsing reviews and comments, I can’t stop thinking something: are the journalists to blame, the editors, or the dumba** play…”consumers” who write stupid things like “not 10/10??! pre-order canceled” or moan about “how the journalist is a sell-out and a piece of sh*t because he didn’t gave a 10/10 on the game”….

    #12 2 years ago
  13. monkeygourmet

    Percentage review scores used to be important in disscussions at school regarding Mario’s superiority over Sonic the Hedgehog!

    I still remember being giddy with excitement when Julian Rignall reviewed the import version of Super Mario World in Mean Machines and awarded it 98% over Sonic the Hedgehogs 90% score! :)

    Now, however, marks are not really relevant. I think scoring should be axed so the written review remains the most important factor.

    #13 2 years ago
  14. Blerk

    I’m struggling to remember the last time a review score actually influenced my buying decision. I sometimes read a review for entertainment, but they can never hope to match the level or depth of discussion that forums and social networks can offer, and the ready availability of video footage just sticks the knife in further.

    The opinions of your friends carry much more weight than the opinion of a single stranger or publication in this day and age, and that’s my first port of call if I’m unsure about a game these days. I sometimes wonder whether reviews matter at all any more.

    #14 2 years ago
  15. Harry

    Good stuff Pat. Makes me glad I’m out of the system. It got rather stressful being a site editor trying to get code to writers. A game arriving the day before release either involved (a) me pulling an all-nighter to get a review ready, or (b) sending the game to a freelancer and putting up with a late review. The former was fine when I was single, but once a dad of twins was just impossible. I know we’d no right to those games, I never pestered for code, but it was galling to receive a game 12 hours before release then be pestered by PR asking where our review was the next day.

    Then of course there are the readers. I have always believed in using the full range of scores. Actually I’m against scores – would be happy to do without them – but when I had to use them I made sure the range was anything over five was good. And I believed in my writers using scores that reflected the text. But when I gave Gran Turismo 5 less than OMFG TEEEEEEEN I received an astonishing online bullying assault directed and organised by N4Gamers users. I even received death threats against my children. Death threats, Jesus,

    I don’t work writing about games anymore. Can’t say I miss it one bit. It’s lovely to to see the massive pile of games hitting stores right now and know I don’t have to play them in a hurry or find a writer to review each one. I’m enjoying my games again after a while of it being a love/hate relationship. And I don’t miss the kind of tossers who make violent threats over review scores of games they haven’t even played.

    #15 2 years ago
  16. Joe Anderson

    Spot on, although it is nice having late review code to moan about. :)

    #16 2 years ago
  17. freedoms_stain

    Scores and ratings just shouldn’t be.

    The things I look for in a review on the seldom occassions I bother reading them are the things that really annoyed the reviewer or that they particularly disliked, whether they sound like they’d annoy me and if those annoyances or negatives would put me off playing the game.

    Next I want to know what the game does well, personally I’m willing to put up with some annoyances if the game shines in other areas enough to overlook them.

    I also think that games should be stacked up against their peers. How does the reviewed game compare to other games in the genre that readers might already have? There’s an element of cross reference there because if the reviewer is complimentry about a feature in a title I already own that I didn’t like (or negative about things I did like) then I know my opinions and this reviewers are at odds and I should probably find another one. On the other hand if the reviewer says “sound effects in MW3 are balls compared to BF3 just like all the other CoD games” for example then I’m happy this dude is speaking my language.

    7,8,9/10 really means nothing on its own, and lets face it review text and score often feel unrelated.

    #17 2 years ago
  18. Mike

    Here’s my kind-of retort/essay o the same subject:

    http://tmt2m.blogspot.com/2011/11/publisher-power.html

    #18 2 years ago
  19. GamezIdiot

    Funny how Jim Sterling’s name always appear in ddiscussions like these? He’s as bad as the publishers.

    #19 2 years ago
  20. Mike

    ” do your research, find some reviews published after the embargo, let the game get patched a few times and then buy it (or not, as the case may be). You’re allowed to do this. You are an individual. This is why you have a brain.”

    So advertising and marketing doesn’t work? Why then do they spend all that money on it?

    #20 2 years ago
  21. daytripper

    @17 Disagree, there is nothing wrong with the review system as it stands, the problem is the way people i.e publishers and gamers react to them, not being happy with a 8/10 is a disgrace.

    #21 2 years ago
  22. mojo

    once again a writeup that tells me that our current world isnt made for me..

    i cant find myself anywhere in the text.
    I dont buy games based on reviews
    i rarely read any (i just like fanboywars :D so i read the round ups),
    ive never, honestly, never bought a game because of a flashy add on some website. to be fucking honest, i cant even tell which add is currently displayed on vg247. and i dont block them. my vision just blurs them out automaticaly.
    i never(!) preorder a game (allthough this might change, seeing pubs get the creeps when preorders get canceled. i might preorder random games just to cancel them daxys before release). Someone still has to explain to me why, why, why, just simply WHY *stamps wildly with feet* u want to preorder a game?
    because u get it early? lol tell that amazon. if im that moist for a specific game that i want it day one, im going to the street dealer that sells it days before official release. Theres always one that does and normaly others follow swiftly.
    preorder bonus? DONT MAKE ME LAUGH?!
    HAHAHAHA

    what a fucking bullshit this industrie became is without words.
    FUCK U ALL!

    #22 2 years ago
  23. Freek

    There is also the added problem that there isn’t a universal scale, different media outlets have different review and scoring policies so avaraging them all out is a road to failure.
    An 8 from Edge, or EG or IGN all mean verry different things and carry different weights. They are not the same thing at all.
    If EG scores something an 8 is see that as a ringing endorsement and something I should pay attention to. If IGN does it, it’s just tuesday.
    And if Giantbomb say it’s awsome on the podcast or quicklook, I don’t even need a review, that’s enough reason to buy it.

    #23 2 years ago
  24. DSB

    @14 I feel much the same way, but I don’t think reviews have to be irrelevant. They’ve made themselves irrelevant thanks to writers who write mostly the same things and score those games mostly the same, probably based on the notion that they have a “responsibility to the reader”. I don’t see how that responsibility could ever go beyond just being honest.

    You don’t have to assume anyones likes, dislikes or general opinion to be a good critic, you just have to stand your ground and call it as you see it. You’re just fucking people over by telling them what you think it is they want to hear.

    And isn’t that the crucial difference between a reviewer and our friends? Our friends are honest, and they give their genuine opinions based on who they are. That’s why we respect their take on it.

    Reading reviews these days is like reading advertising. You’ll mostly only hear the same pitch from reviewers, and anyone who may or may not disagree with that consensus will be likely be so far from the mainstream that it doesn’t matter anyway.

    I don’t think it has to be that way.

    @19 Actually the point Pat was making was that gamers were little bitches who can’t stand to have their babies critisized. Good job for helping him out on that one, though.

    #24 2 years ago
  25. Mike

    “If you’re a gamer, and you’re seeing an 8/10 as a h8/10, maybe you should just get some perspective. Why don’t you play the game and see what you think? You never know: you may just enjoy it despite the number and learn to find reviewers that work for you instead of screaming into the void over arbitrary metrics.”

    Because they cost €50? And that’s loads, and loads of money for something unessential. Not every country has a highly competitive online retail sector driving down prices.

    #25 2 years ago
  26. mavis_beacon

    This article really sums up the games industry, lead by money and paid for by fools.

    #26 2 years ago
  27. The_Red

    Bravo. Many many many thanks for this article. Hopefully a lot of people will read and learn from it. LOOOOVE seeing all these original features VG247.

    #27 2 years ago
  28. IL DUCE

    Wow great piece Pat. Definitely agree with everything said and I’m trying stop pre-ordering games. I’ve just had so little time to complete games and I keep buying them so I need to catch up. But, I can see this situation w/ Saints Row: The Third occuring right now, it has twice as much x box reviews to ps3 reviews and the x box reviews are about 6 pts higher on metacritic with OXM releasing at least a week or more before everyone else giving it a 95 while the consensus looks to be more around 80-85…

    And also, publishers and developers are so worried about review scores but if they make good solid games they have nothing to worry about…that’s the problem today is that the media and internet and everyone being so connected allows companies to control review timings and scores in order to get the best sales…not to mention the whole pre-ordering thing roping people in w/ exclusive content and whatnot…but reviewers also are part of the problem as well, they need to be honest and not take these deals b/c if their site has good content then the traffic will come on its own…hell this site doesn’t even do reviews and I’m here everyday just to see what’s new in the gaming world b/c I’ve found it to be the most complete source of news and content

    #28 2 years ago
  29. rauper

    #23 is a very good point and probably deserved a mention in the article. EG is (last time I checked), on average, 0.9 (i.e. almost a full point) below the metacritic average, on all games. So a 90 Metacritic (or IGN, Gamespot etc) does not equal an EG 9.

    And then there’s “out of 5″ scales and so on, which get rounded by Metacritic…

    #29 2 years ago
  30. BillyBatts

    Great read, just one element missing. Just why exactly are pre-orders so vital to the big bad publishers? Retail.

    There’s a recession on, shops are closing by the minute. Shelf space is now more precious than ever as every inch has to be filled with ‘red hot’ product in the retailer’s eyes to keep the bovine customers coming through the doors to pick up their boxes. Back in the day, 7/10s could expect decent orders from HMV, Argos, GAME and the like. These days, not so much.

    “You had some middling previews eh? We’ll pass this time.”

    “It’s a colourful Japanese platformer? Come back when you have the next COD.”

    I’ve worked for publishers who insisted we delayed preview tours until all initial retail meetings were concluded, as it ‘just wasn’t worth the risk’. Think like a publisher again; the only way to prevent this kind of disaster is to protect the game’s image until the very last minute.

    #30 2 years ago
  31. rauper

    Oh and of course there’s the preceding 1000 words or so (aka the review) which are a much better indicator of whether you will like a game than just a single digit number!

    #31 2 years ago
  32. Mike

    So scrap it.

    #32 2 years ago
  33. IL DUCE

    And too add on to that us as the consumer definitely need to be more open to games that aren’t 90 or higher on metacritic…I know one of the issues is the restricitve prices of games for some @ 60 bucks a pop but I just recently cancelled GameFly and if you play games a lot it was a good rental service to have in order to prevent having to buy every game that interests you…

    That said…I’ve had plenty of fun playing games like Dead Island, Metro 2033, Operation Flashpoint, Medal of Honor, SOCOM 4, etc. That haven’t always received the greatest metacritic scores

    #33 2 years ago
  34. StolenGlory

    “Oh and of course there’s the preceding 1000 words or so (aka the review) which are a much better indicator of whether you will like a game than just a single digit number!”

    Exactly.

    Scores these days seem to represent little other than a jump off point for the disgruntled to light their fires in the relevant comment threads.

    #34 2 years ago
  35. The_Red

    @23 Spot-on (Regarding the IGN and GiantBomb). Eurogamer is a little more in the middle IMO. They are usually more critical and honset than the rest but a lot of times suddenly become “10/10 lovers” for games that don’t need / deserve such scores.

    Also, another big problem is that the 10/10s only go to super hyped and madly advertised big budged games. Likes of Skyrim always get many many 10s even on PC, where it’s simply a horrible port… yet, a smaller title like Dark Souls almost never gets a 10 because it’s not “Skyrim, Zelda or Uncharted”. If a few great games that not too many people know about, start getting 10s and 9s while super popular ones get lower scores than undeserved 10s, maybe some people will start accepting 8s again and not calling a great 8/10 review a h8 review just because people at Eurogamer actually try to write honest reviews.

    #35 2 years ago
  36. Mike

    The reason review scores are kept is just publisher pandering. Any site telling you to READ THE WORDS but still sticking a score at the end is just trying to get the best of both worlds.

    Just universally drop the score. That’ll fuck the publishers off, and maybe we’ll see some kind of paradigm shift.

    #36 2 years ago
  37. DSB

    I think it’s a bit silly to rank the sites after some percieved average. The score on Eurogamer seems to depend as much on the reviewer as it does everywhere else. It contains a couple of the worst reviews I’ve ever read. The one for Braid is pretty much burned into my brain.

    Maybe I didn’t get the memo, but since when do these 9/10 reviews contain actual criticism within the text itself? People aren’t just upset or apathetic about a number, the text itself is often just as completely useless, for saying the same thing it says on any other site, from any other reviewer.

    They might throw in a sentence like “It does have this and this wrong with it, but we won’t hold it responsible for that”, but that sort of disclaimer is usually the closest you get to actual criticism.

    #37 2 years ago
  38. OlderGamer

    As an old fart, a gamer, and a consumer, I dispise preorders (won’t do them in most cases), laugh at MC (of course its bought off and influenced), and in general, dislike the industry as a whole in the current form it is today.

    Its just big bizz. To them we are all sheep. Plain and simple.

    As always, another great read on VG247.

    #38 2 years ago
  39. Mike

    Basically it comes down tot his: who pays journalists’ wages? Publishers.

    Until that’s rectified, the games journo industry will (rightly) never be taken seriously)

    #39 2 years ago
  40. Mike

    In fact, publishers own the whole industry from site-owners, to site designers, to the journos to the tea lady.

    So asking the industry for anything like a non-partisan, truly, wholly independent stance is an oxymoron.

    #40 2 years ago
  41. mojo

    38: completly second that. every bit.

    #41 2 years ago
  42. GrimRita

    Interesting read. How about doing a part two, looking at how advertising revenues and high review scores go hand in hand? Kane & Lynch/Eidos anyone?

    Personally, you cant beat a good demo to get you hooked on a game. If it wasnt for the FM2012 demo, I wouldnt have purchased this years version (after owning FM2011/10/09 etc) and if I didnt play the Anno 2070 demo, I wouldnt have cancelled my pre-order.

    But seriously, journos are simply too worried to point out the flaws in any triple A publishers game and if they do, they always under play it – but then you look at smaller publishers who get slated if something is out of place.

    #42 2 years ago
  43. hobbesthetiger

    Excellent piece.

    As a games writer who is a little further down the food chain, I find the whole reviews process more than a little frustrating. I write for a site that was brave enough not to add scores to reviews at the start of it’s life. Unfortunately, as the site grew, the feedback from users was that they liked our style but wanted a score to be provided. So the editor imposed the need for review scores.

    It’s easy to say ‘oh well, it’s the gamers fault – they always have to have a review score’ but the entire industry is based around that little number. PR look at the scores you’ve given in the past before deciding whether to provide review copies. Consumers get all steamed up over scores. Metacritic has become the lazy analysts dream. And even websites like mine who start off with the best intentions end up doing the same as everyone else and stick that little digit on the end of review, because it’s expected.

    #43 2 years ago
  44. SplatteredHouse

    A very good piece.
    OlderGamer, I think you’re giving MC too much credit. They have a list, and they publish summaried links to sites that they’ve vetted. A passive aggregator and nothing else. I don’t remember the last time I saw them pushing their own numbers. Other people have distorted and twisted the intention of that page for their own purpose and benefit. I disagree with the makers, and maintainers of that resource being pulled in to this in that way.

    “The best critics’ reviews combined into a simple Metascore” and, the reader can decide if they agree with “the best” or not, as they can any review or piece of information. They provide a service covering most types of media now, that’s their business.

    Pat asked who the reviewer’s serve, in the article. I actually agree with Mike, and would find the more beneficial question to ask to be that of whom the SCORES serve? I wouldn’t oppose a riddance of them, and would be delighted if some amongst the most popular sites did start the ball rolling on it.
    If the scores are intended to communicate to the customer, to aptly summarise the preceding text, and the message is no longer getting through, then why give the message to begin with – is it essential to the reviewer to use such a facility to convey their view? Why?

    I always regarded that the place to find the review’s meat was within its textbody. If I don’t see the reviewer’s experience with the game coming across there then it’s a sure sign to consider it of lower value, and maybe look elsewhere. If you asked a friend, who claimed to have played a game to give an impression of their time with it, (you were planning to put money down to get it, but maybe weren’t 100%/there were several you had an eye on, etc) and it became clear they were instead basically reciting from packaging and failed to elaborate beyond, that probably wouldn’t illicit a thankful reaction.

    I did find it interesting to see #10′s comments about the preview coverage issues shown to sell better than those carrying the world exclusive review. I would guess that from that you get the initial reaction, when you have the freshest perspective. Think of it as: If I hadn’t played a game, and I saw it with no consideration of monetary value, or release date HOW GOOD OF A GAME IS IT becomes the over-riding question. How much is this writer, basically in the position I may be six months down the line, excited about this game, and why.
    How good of a game is it…But, wait, isn’t the review supposed to tell us that?

    #44 2 years ago
  45. Erthazus

    FaPtastic read. Incredible work Pat. Although,

    “There’s no fourthly. Let’s take a look at some of the aspects of game reviewing journalists and gamers constantly moan about and explain why they happen, taking the aforementioned truths into account.”

    I think there is one more. “SHOP”. Someone said to me that some of the review sites have in-site shop with games and you just can’t avoid big blockbuster games. There is always, so much anticipation for them and 9-10 is always a minimum. (I’m not talking about cheesy sites like ShopTo with their reviews. Still, they are affecting metacritic too btw)

    Thanks Pat for explaining more about “Exclusive!” and how it works. It’s a very interesting read material. VERY interesting and somehow depressing to me.

    but now i know much more. Thank you.

    9/10
    Excellent!

    @38, OlderGamer, you are correct. Agree with everything you said and thats just depressing.

    #45 2 years ago
  46. Quiiick

    A simple 5 stars rating with no in-betweens would solve some of the problems.

    ***** = Very Good
    **** = Good
    *** = Average
    ** = Poor
    * = Very Poor

    Most games would be within the top 3 ratings anyway.
    This would be equivalent to 8/9/10 which is used today but without the feeling that games are overrated. ;-)

    @#38 (OlderGamer)
    I’m an old guy as well (even older than OlderGamer) and I totally agree with you. It’s big business, that’s all.

    #46 2 years ago
  47. Erthazus

    @46, it won’t solve a thing.

    1-99% is the best thing you can do here. It’s just our industry is a big “bizz” and thats it.
    1-99 is used in movies, music in metacritic and it works.

    You can see it for yourself. Check out metacritic and see games scores and movie scores for example. Thats depressing.

    1-5 star system won’t solve mediocracy of a product or “overrated” thing.

    Why? check out http://www.g4tv.com and see scores for yourself.

    From 4 starts to 5 at minimum.

    #47 2 years ago
  48. Keza

    The hilarious thing was that even after all the love that I’ve shown Dark Souls over the course of years, on IGN and elsewhere, the comments on the review were still full of outraged people calling me an idiot/noob/hater/dyke/whatever else the Internet hate machine regularly throws in our faces because I gave it a 9.0 instead of a 9.5. It’s absolutely baffling, and I think that gaming communities are more at fault here than publishers.

    Metacritic, for me, is the worst thing to happen to games journalism since I started in this industry, because it immediately devalues your opinion if it doesn’t sit with everyone else’s. Since when were the best critics the ones who agreed with everyone else? Since when, for that matter, have all of my favourite writers and critics been the people that I always agree with? Most of the people whose writing I enjoy reading are people that I barely *ever* agree with.

    The reason that IGN scores on the high end of the spectrum compared to, say, Eurogamer is twofold: firstly, it’s mostly written by Americans, and they just do things differently (you should try working with US PR if you think working with European publishers is difficult – to them a 7 is an actual slap in the face). But secondly – and this is important – IGN presents reviews as the opinion of one person, rather than some kind of objective ideal. Greg Miller loves Uncharted, so his 10 for Uncharted 3 is natural for him. The idea is that you’ll think “I know this guy, and his tastes are similar to mine, so I guess I’ll like this too” – or that you’ll see a review is by a critic that you don’t usually agree with, and respectfully disagree. I do this all the time on Eurogamer, too. The score? Not really so important as who wrote the thing and what it says.

    The fact that scores matter SO MUCH to online communities is what’s fuelling this nonsense just as much as publishers, who often rely on Metacritic averages. I think the critics themselves – the professional ones, anyway – are third on our list of problems.

    #48 2 years ago
  49. Quiiick

    @#47

    RE: “1-99 is used in movies, music in metacritic and it works.”

    No it doesn’t work. It’s illusional.
    There’s no way to tell the difference between say 82%, 83% or 84%.
    As said before.

    There is absolutely no need to distinguish in-between “very good”, “good” and “average”. It’s all you need to know.

    If you happen to want more info than that:
    READ the fucking review and forget about numbers and stars.

    Ratings should NOT be used to judge a game. No number will ever tell you anything meaningful about a game. Therefore the rating system should be as minimalistic as it can be.

    #49 2 years ago
  50. onefivefive

    This is the best games article I’ve read in a long long time. Great job.

    #50 2 years ago
  51. Erthazus

    @49, it’s easy. 84% is 8.4 aka Good Game.

    It’s a really easy European method that is used in school/university and it works in movies and other media. It works.

    It’s really is the best system.
    1-5 is not working because there are not enough details in these scores.

    you can see it for yourself in http://www.g4tv.com .

    @Keza, ” It’s absolutely baffling, and I think that gaming communities are more at fault here than publishers.”

    Absolutely not. It’s only your fault because you are the reviewer and if you are not honest, you are getting the community you deserve. It’s only your problem.
    Want to change something? Start with yourself first of all. Until then, you will enjoy the community IGN created long time ago.

    IGN these days as you said can’t live without scores like 8,9,10 and because of that you get the community You deserve.

    and you really. I mean really felt that Dark Souls is that amazing that you can give it such a high score?

    I played most of the AAA games this year and still playing. But i can give only one game 9/10 and thats Portal 2 and maybe, just maybe Witcher 2. They still have a lot of flaws even for the 9/10.

    9/10 means that the experience is not only great, but it’s innovative and creative in every aspect there is.
    8/10 is enough for a game that is a Great Experience without innovation itn it’s mechanics or technology.
    IMO.

    #51 2 years ago
  52. GamezIdiot

    Sigh. If you ever find yourself arguing on the internet whether 85% means an 8 or a 9, YOU’RE PART OF THE PROBLEM.

    #52 2 years ago
  53. Quiiick

    @# 51 (Erthazus)
    RE: “9/10 means that the experience is not only great, but it’s innovative and creative in every aspect there is.”

    That’s exactly the problem!
    You put complex meanings (like innovation or creativity) into numbers. That’s not possible.

    A game can very well be good (aka 9 or 4 stars) without any innovation. That’s absolutely possible because it it maybe truly shines in another (unnamed) category.

    Again:
    If you want meaningful info about a game:
    READ the fucking review and forget about numbers and stars.

    Ratings should NOT be used to judge a game. No number will ever tell you anything meaningful about a game. Therefore the rating system should be as minimalistic as it can be (5 stars).

    #53 2 years ago
  54. Keza

    @51: I really don’t think that giving a game a high score is “not being honest”. Dark Souls is among my very favourite games I have played ever in my life. Obviously I thought it was amazing, otherwise I wouldn’t have given it the score that I did. Artificially lowering scores for the sake of perception is no better than inflating them.

    I don’t think there are any sacred rules about what scores mean. COD can be a 9 for a lot of people even if it isn’t innovative and creative. For a lot of people that’s not the point. The point that’s being proven here is that, once again, the discussion focusses on the score and what it means, and not the criticism.

    IGN’s community isn’t special – take a look at the comments on Eurogamer’s Uncharted 3 review if you don’t believe that. Most gaming communities put far more emphasis on the score that we ourselves as critics do.

    #54 2 years ago
  55. fearmonkey

    Excellent article and very well written.

    Maybe the answer is our ratings system, as Nigel Tufnel said “Turn it up to 11″. So if 8/10 is the new 7/10, then we need to make it 11/11 lol.

    Seriously though, as the article implies, gamers aren’t stupid. If the gamer is serious enough to go to metacritic or look at reviews, he/she will realize that you cannot take the word of one site.
    I tend to read the high scores on metacritic and the lowest scores, and maybe a few in between, on games that I’m unsure about. Gamers know what to expect in a game like Skyrim, Gears of War 3, Uncharted 3, etc. Games like Deus Ex:Human Revolution are much more impacted, as gamers take a wait and see on titles like that.

    Maybe scores aren’t the answer, maybe the star system 1/5, or a term system (must have, recommended, pass) is better.
    I remember reading an article in Computer gaming back in the day talking about the same kind of stuff, and this was back in the 90′s.
    They bemoaned that publishers took offense at low scores, took it personally at times, and thought a 7/10 was offensive if the game had a high budget especially.

    I feel for game developers, no one wants to spend to spend two years more or less on a game that gets average to low scores. At the same time, no gamer wants to spend that $59.99 on a new game that may be a lemon. Maybe this wouldn’t be such a big deal if games were a bit lower in cost? Games like Skyrim can sell at a premium, but other games with no following mostly cannot get away with the same cost, yet publishers all try to sell new at the same price as the AAA titles. Then they bemoan the used sales.

    #55 2 years ago
  56. GeekElite

    Let me preface what I’m about to say by stating that I don’t, in any way, believe an 8.0 to be a bad review score. In fact, quite the opposite – I see an 8.0 or above as the cream of the crop.

    And that’s why an 8 or above is what determined my buying habits, to some degree. A TON of video games come out every year, way more than I have time to play. I use review scores as the first of many controls to narrow down my choices, and to help me make decisions about games where I’m on the fence. If I’m pretty sure I’m going to buy a game already and it gets an 8 or higher, I’ll buy it, no question.

    If I’m pretty sure, but that game gets lower than an 8, It will make me question whether I should buy it. Not because I think that a 7.5 is a bad score, but because I need to devote my limited gaming time to games that are really going to blow me away. The 8.0 is just my arbitrary cutoff, because limiting myself to 8.0 or better just happens to mathematically work well with the amount of gaming time I have.

    #56 2 years ago
  57. SplatteredHouse

    “Games like Deus Ex:Human Revolution are much more impacted, as gamers take a wait and see on titles like that.”

    Then, why is that happening. Could it be a dearth of information reaching them (before the release date score) to enable them to make that decision of which one’s preferred out of a selection? In what ways does the game stand out, well, in the case that this question cannot be satisfactorily answered, anything but a wait and see approach could be considered foolhardy.

    Because, as fearmonkey also pointed out people don’t want to be landed with a duffer! As you wish the price to fall, notice the publisher would see it raised. They raise the sell-in to retailers already. This whole business is the kind of thing that does cause me to agree with OlderGamer, there’s a great mess and I believe the pubishers must shoulder a large degree of the responsibility for establishing and then, perpetuating of such a state of affairs.

    @51: “you will enjoy the community IGN created long time ago.”

    This part I do agree with of what Erth had to say. I also think that whoever started the smoke billowing over from out of IGN (At a guess, I’d lay the blame at the dividing line between editorial, and advertising) made a really bad call – come on, I even saw an editor the other week on one of their community shows remark in jest at having been paid off: http://uk.ign.com/videos/2011/10/20/igns-top-dog-decides-modern-warfare-vs-battlefield – I note the context absolutely, but still, you mean to insist they’re not part of the problem?

    The thing I find worst about that kind of thing, is that there’s any such suspicion or situation like that, that occured to be able to make light of.

    #57 2 years ago
  58. DSB

    The idea that it’s all about the scores and not about the substance is really just a cheap and quick defence. If a review scores a 9 or a 10, then obviously the ammount of criticism levelled in it is going to be minimal, and in some cases, namely the readers who aren’t going to agree, that makes all the difference in the world. Especially when all they’re left with is a bunch of fan-service reviews.

    I thought Mass Effect 2 was a terrible game, and I didn’t have a hard time finding mates who thought similarly. I was pretty much the idiot who kept playing till the end, just so he’d get his money’s worth in play time.

    The fact that the reviewer corps at large pretty much called it the greatest game ever, with next to nothing in the way of contention or differing opinions, makes me think that there’s something seriously wrong.

    Sticking games to the people who already love a series is, in a lot of cases, going to ensure a very positive starting point. I don’t see how that’s supposed to benefit the public at all. Should we start hiring referees for soccer matches that are fans of either team as well?

    #58 2 years ago
  59. Quiiick

    @56 (GeekElite)
    Your reasoning is exactly what most people use to determine their buying habits. There’s nothing wrong with that but it is the very reason the rating-system is broken.

    Publishers know exactly that this scheme is used to make a purchase decisions and that’s why they are so keen to get at least an 8 and prefer to have their game rated 9 or 10.

    The conclusion of all this is that you no longer can be sure to get a decent game by going after an 8 or 9 score.
    Even some truly average games are rated 8 or above nowadays.

    #59 2 years ago
  60. Quiiick

    #58 (DSB)

    RE: “I thought Mass Effect 2 was a terrible game …”

    Hurray! I’m finally not alone anymore. :-D

    (In my personal rating I gave it a 3 out of 5 stars, which is usually 7 in a 1-10 rating)
    http://www.gamespot.com/users/Quiiick/games_table?mode=own&sort=release&d=1

    #60 2 years ago
  61. SplatteredHouse

    @58: Hand-picking staff, beyond a sufficient demonstrable knowledge of the genre perhaps, is another of those questions that leads you to ask who’s it for. I suppose though, that it plays in to the hands of publishers that target games to individual categories of player. But, at the same time, you can’t overlook the chance that people may be out of the office, on assignment, etc, etc.

    #61 2 years ago
  62. Keza

    @57: IGN isn’t part of the problem. It’s the big dog, and it gets a lot of exclusives still, and so it’s the easiest target. Nobody gets paid off for a review score anywhere, least of all at the big outlets (we’d get fired in an instant). The fact that we can’t even make a joke about that in a video in which the PlayStation editor’s dog decides whether MW3 or BF3 is better is surely testament to how deep that fallacy runs.

    Disclosure: I work for IGN now, in case that wasn’t obvious, having spent the previous six years freelancing for everyone from this site to Eurogamer to most of the UK games mags. Every outlet has its problems, obviously, but editorial integrity isn’t one of IGN’s. It’s taken enormously seriously, as I’d hope would be self-evident.

    #62 2 years ago
  63. The_Red

    @51 Keza
    My post / complaint earlier about the need for Dark Souls to get more 10s wasn’t directed at you. I actually think really liked your review. I love it though not as much as Demons. Dark S doesn’t DESERVE a 10 or even a 9.5 but then, no game this year does. I’d argue that Demon’s Souls is the only game of this generation that deserved (near) perfect score.

    (Just saying that I’m not among those that hated the Dark Souls review. Goes back to Blighttown to keep trying to get past those Rock throwing monsters).

    #63 2 years ago
  64. manamana

    Dear Patrick, you are constantly evolving in your journalistical storytelling and the insights of the gaming industrie. Thats what makes this site very special to me, thank you very much for that and keep your critical instincts up and running, so you can write them down for us.

    #64 2 years ago
  65. OrbitMonkey

    “Most gaming communities put far more emphasis on the score that we ourselves as critics do”.

    Well that pretty much sums it up :)

    I’ll take the word of a decent review over a arbitrary number anyday.

    #65 2 years ago
  66. SplatteredHouse

    Some times when I saw reviews on IGN, I’d regard their content as unhelpful/lacking and this became more noticable, so I just didn’t visit anymore. In the run-up to MW3 I hadn’t really been following it at all, and I noticed that particular top dog feature from Youtube and just wondered what it was about so gave it a watch. I was anticipating some kind of animated studio debate format between rival groups of IGN editors, not quite what I found! :)

    I definitely saw (and see) the humour of the editor’s dog making the decision (and while trying to find that particular clip to be sure I hadn’t just imagined it or something, did turn up a similar feature where the audience had their say through video hits after seeing clips of both games), but I just think it’s mainly a pity that suggestion/belief exists that such a joke to be made from, to begin with.

    #66 2 years ago
  67. DSB

    @61 Don’t get me wrong, I don’t need every reviewer under the sun to agree with me, I just need them to stop consistently agreeing with eachother. What happened to the second opinion?

    When Tom Chick slammed Deus Ex on Games Domain, they printed a second opinion from a different writer to balance it. Why doesn’t that go for a 10 review, or a 9 review? Why are there no (or so few) second opinions to be found?

    I can’t rule out the possibility that I’m totally weird and no reviewer is ever going to cater to my tastes, but that still doesn’t explain just how similar it all becomes every time a major release hits the street.

    One of the things that gets me is that this doesn’t happen for movies. Sometimes it almost seems like movie critics do everything they can to disagree with eachother, and their reviews certainly don’t all read the same. So how can that be the status quo for games?

    Disagree with me any day of the week, but don’t tell me that all games reviewers all love the same games completely irregardless of who they are. If that’s truly the case, then we need to get Stalin on the job and purge the fuck out of that profession.

    #67 2 years ago
  68. Samuel

    It’s unbelievable to me that anyone would think an 8 is a bad score. The actual fact is that it’s not, it’s a good score. To think otherwise is foolish and that opinion should be disregarded. But that can’t happen only because somehow, so many people came to this false conclusion.

    #68 2 years ago
  69. Ireland Michael

    Brilliantly written piece, but I couldn’t disagree more.

    I think it’s pretty absurd to blame someone else and make excuses for journalists unwillingness to have a spine.

    I’m so glad that I’ve never been one of these journalists. Me and my guys have always been completely honest. If we receive a game for review, I will review it honestly, whether it’s a 10, a 7, or a 4. If developers feel the need to stop sending us games because of simple, professional honesty, then so be it.

    We put disclaimers in all our reviews, so there’s complete transparency from our readers as to how we obtain our reviews copy, to help show that are scores aren’t “bought” by generous scores or advertising.

    You say that consumers don’t need to pre-order games or buy them on release, and should wait for more “independent” reviews, yet you also say that websites are obliged to rush reviews and content out day one, even when forced into restrictive and tightly scheduled NDAs?

    Surely that’s a serious double standard? You do seem to be shoving a lot of the blame off of the media and on to everyone else. At leas that’s how it reads to me.

    Everything else I agree with, especially regarding gamers. Gamers seriously need to get some perspective, and start thinking for themselves a little more. I have never, ever understood the obsession with pre-ordering games myself.

    Assassin’s Creed: Revelations is the perfect example of this. It got mostly glowing reviews across the board, yet as soon as a few 7s started appearing, people wanted to run and cancel their pre-orders. What ever happened to personal preference? Weighing your own personal tastes against the positive and negative points raised in the reviews? It’s like people can’t think for themselves anymore, like it’s too much effort or something.

    In the US, half the big stuff can be rented from a RedBox, or you can just pick it up from a friend. Why do people feel such a desperate urge to have everything so quickly? Most of it will be half price in month anyway. Recessions my ass… the way some people spend, you’d find it hard to believe there is one.

    #69 2 years ago
  70. mavis_beacon

    Does this whole thing prove that gamers are too stupid to read?

    This is the greatest online review there has ever been and I would like to see any gaming website have the balls to do similar and bear in mind that this is one of the biggest music websites on the planet – Pitchfork.com

    #70 2 years ago
  71. OlderGamer

    “Everything else I agree with, especially regarding gamers. Gamers seriously need to get some perspective, and start thinking for themselves a little more. I have never, ever understood the obsession with pre-ordering games myself”

    I have two problems with preorders. Neither is a person knowing that they will like a game and wants to have it dayone.

    I dispise day one DLC incentives being linked to preorders. I preordered BF3, and I did so because I felt that it would be cheaper for me in the long run, because th game came with an online pass(that I would have had to buy if I picked the game up second hand), and it came with the Back To Karkand DLC. Now the DLC isn’t out yet, but when it hits, i will have it covered because I bought the preorder copy.

    The second reason I have to hate preorders are the retail chains. I know my local gamestop(been told by a real life friend – whom happens to be the stores manager), that they will restrick ordering numbers of a new game to the preorders plus one or two. If you want a game a couple of weeks after the launch, you will not find a new copy on the shelf. Instead your stuck buying a used copy directly from GameStop – at an increase of profits for said store.

    Both of these things are wrong, and really part of the same problem. Pubs knew that the first couple of week window can make or break a games sales – so they push hardcore for preorders with incentives and marketing/PR(like trying to boost review score outcomes).

    I find the best place to get a feel for a game before I buy it is asking people on my friendslist that I see playing the game. And websites like Gamefly where you can read actual gamers and members of their site write actual first hand reviews. Their numbers aren’t important, but things they say are. I can not take the review of any reviewer working for a site or mag that makes its living selling advertising space to publishers of said game being reviewed at face value. In that context, a score means nothing anyways. If I am interested in a game to begin with, I want to know about stuff like controls, dificulty, gamelength, glitches/bugs.

    It all comes down to my fav saying in life. It is what it is.

    #71 2 years ago
  72. djmattyg007

    You know something? If you base a purchase purely on the score a game gets, you’re an idiot.

    I highly recommend you all check out the show Good Game. It airs on ABC2 here in Australia, and you can vodcast every single show since it started a few years ago. Just google “Good Game ABC” and you’ll find the site.

    Yes, they give scores at the end of their review, but each review is anywhere between 7-15 minutes long. They take you through what the game is, the game mechanics, the baseline story, and what they thought of each part of the game, as well as it as a whole. And you know something? They don’t use numbers to quantify it. Yes, it’s purely subjective, but they use plain English, and that, combined with what I hear generally from friends and others from a myriad of sources, helps me form a good idea about the game. A score out of ten can’t do that, and anyone who thinks it does is retarded. Ergo, Metacritic is retarded.

    #72 2 years ago
  73. manamana

    Thing with preorders is: you have it in your box when its out. And thats great, as it happened often that around launchday of a game I dont have the time to drive around and I’m stucked at work. Plus there are bonuses that makes sense, like Karkand DLC – in a game I already knew I will play for months (despite a shite campaign).

    So, did I preorder Zelda? Yes.
    Do I build my gamingpreference on scores or reviews? No.
    Do I like to read some of them? Of course.

    Next preorder station: The Last Guardian.

    #73 2 years ago
  74. Joe Musashi

    @69 I think it’s pretty absurd to blame someone else and make excuses for journalists unwillingness to have a spine.

    I think that’s putting it a bit strongly. However, as I mentioned in my earlier comment, it simply feels wrong for the writers to point the finger at gamers and games publishers when the article that they have written is not well received.

    A lack of accountability combined with finger-pointing simply comes across as making excuses. This does nothing to instil confidence in those who are being scrutinised – in fact it does more damage. Every profession demands accountability.

    Furthermore, this importance of a number – if the media wishes to make a stand about it then either use it properly (score 1-10, not 7-10) or don’t use it. It’s being misused and there’s nobody to blame for that than those that write the score at the end of the review.

    Then there’s the talk of why are they so important. Why do gamers rely on them so much? It’s a fair question in isolation. But in the context of this site which habitually aggregates review scores on high-profile games releases (not the review comments and not for games that haven’t already been hyped) it comes across as a weak point of view. In such cases, this behaviour is only justifying the demand for a number more. There’s one article against game scores here – but there’s many articles on high profile games justifying them. At least three in the last week.

    So where is the consistency? What line of argument is the reader expected to care about? This site alone promotes the use of game scores again and again and it then produces an article like this. There’s a contradiction in those behaviours and it’s not one that can be laid at the feet of gamers or games publishers.

    Seeing others from the games media chime in in agreement doesn’t sit well with me either. It’s a totally one-sided view. We all do it – we all bitch about the way we have to work and speak on behalf of those that make our job more difficult. But that doesn’t mean it’s a balanced view. It’s the games media sympathising with the games media and telling readers why the games media is right and those that aren’t the games media are wrong. A far more balanced effort would be a roundtable with representatives from all links in this particular chain.

    After all – would you trust Fox news to tell you why Fox news is right and others are wrong when there’s been a dispute about how Fox news conducts itself? Hopefully not – you would expect a more neutral view.

    Criticising game scores and the practices behind them in one breath and promoting and justifying them in another is not very convincing.

    And as (sadly very few) others have suggested – you want to make a change then go ahead and make it. You can’t change how others behave, you can only change your own behaviour. That’s where a change will start. But that means you have to first take responsibility for your output and have the confidence and guts to buck the trend consistently. Not with one article in a sea of thousands that contradict it. Blaming others and not changing yourself will not make any difference.

    JM

    #74 2 years ago
  75. Ireland Michael

    @74 No accountability is being taken by anyone, unfortunately.

    The publishers want to see their games sell, the journalist need the hits to “feed their children”, and the gamers are too lazy to think for themselves.

    #75 2 years ago
  76. DSB

    Nutshelled.

    #76 2 years ago
  77. Joe Musashi

    @75 Indeed. But one would hope that those bringing this to the attention of others in such a way would be practicing what they preach – otherwise what’s the point?

    JM

    #77 2 years ago
  78. Rambling Johnny

    The easiest solution is simple stop including a numerical score! That will force peoples to actually read what you have to say about Gungo-Fuckers 3 instead of trusting arbitrary numbers that clearly can’t even be trusted now anyway.

    #78 2 years ago