The Peanut Gallery: why can’t developers answer critics?

@49 Yeah, but then you can also start talking about the guys who sign with the lowballers.

That’s the problem with a press corps made up mostly of enthusiasts. There aren’t a lot of ground rules. It’s every man for himself.

@50 I couldn’t agree more. I’ve turned down games writing for the same reason.

I just don’t really see a solution, though?

I remember where one publisher who was more than willing to keep sending us review copies despite us slamming most of their games. I felt kinda bad for that. =(

Another… completely shut us when we gave one of their games a 2, even though they were a big developer and it was widely accepted that this game was just a lazy cash-in. All their other games we had reviewed incredibly positively.

Sometimes you need to bite the hand that feeds you. The problem wi the gaming industry is that the hand is practically supplying you every meal.

@46 He writes on my site, which is entirely voluntary. A lot of our staff are already professional journalists and webmasters elsewhere, but we use it as a platform provide honest and generally professional content (especially in the area of reviews) without any PR manipulation or people looking over our shoulders.

I’d love to see bitsnark go onto something bigger, because the guy has some of the greatest passion for writing I’ve seen in any journalist in years. He definitely deserves to reap the rewards.

@51 that’s just it man, there’s no real catch-all solution to the issue.

@52 ‘I remember where one publisher who was more than willing to keep sending us review copies despite us slamming most of their games. I felt kinda bad for that. =(‘

I feel bad when that happens too, but I have to be critical or else I’m not doing my job right. One of the harsh truths of the job.

One ‘solution’ – albeit not a very popular one – is to only review games after they’ve been released – and that you’ve actually paid for.

Which is why my most trusted resource for opinion (outside previews) is forums – where I can read the opinions of people who are playing and, critically, paying for their games…

@55 forum reviews (and other “free” reviews) are great for fans of the series or genre. But the issue with non-writers writing reviews is largely that a reviewer (at least a good one) tries to deliver a snapshot of the experience of the game that’s as impartial as possible – and that’s usually the polar opposite of a forum review.
So, professional reviews done well i) don’t carry baggage from one game in the series to the next, ii) carry baggage from rival series, iii) assume that one mode or element of the game takes primacy over another, iv) assume the reader has loads of prior knowledge or understanding of the series, game mechanics, genre etc. And all while, ideally, demonstrating understanding of the game, its genre, the series, the historical and broader context.
In other words, for a lot of people, reading a review of a FF game or a CoD game or a whatever by fans of that game is nigh-on useless at telling them what experience they’ll get. It can be, very useful, if you’re already a fan of the series, of course.
A good example was Keza Macdonald’s review of Demon’s Souls – that enticed readers in to actually play a game that they might otherwise have passed over. It clearly marks out the good and bad, but also draws the reader in to a new world. I just don’t see that on many forums. But perhaps that’s the forums I hang out on!

@56 I’m not sure I could disagree more with that.

Arguably the best position a critic could find himself in, is filling the role of the trusted friend. That neccesarily means calling it as you see it, even if it means losing someone, or maybe not living up to anothers standards. If you think the previous entry is relevant to this one, go with it. If you think a game is relevant to an entirely different game, go with it.

You should use whatever you can if it makes your point easier to understand. Clear communication is the essence of good reporting in my opinion.

You can only critisize as the person you are, with the references and preferences you have, not the person you think people want you to be, or even the person you aspire to be.

I think one thing everybody needs to accept for themselves, before they take on something like criticism, is this: You’re not ever going to reach everyone. Trying to do that won’t help you.

If you have 10,000 readers, then chances are that maybe 3000 are going to agree with you. The rest are going to be split between the people who want to bitch at you, and the people who respectfully disagree. Likewise, some people will catch the reference, others won’t.

That’s essentially why someone like Nathan Grayson was so good when he was on this site. He actually called it the way he saw it, and he didn’t see any reason to play the “professional”, which meant that his points came through unusually loud and clear. And that’s precisely what makes me trust one writer over another.

@57 – hah, I largely agree with you.

Of course you have to call it as you see it. And of course you should “use whatever you can” to make yourself “easier to understand”. That’s the point of trying to reach as broad an audience as you can. Sure, you won’t get everyone – but the job of a professional critic is to get as many as we can. To be as passionate, opinionated, divisive, interesting, but also fair, balanced and clear as possible. And there’s a great difference between professional-seeming writing and great professional writing. Great writing, for money, doesn’t have to be bland. In fact it never should be. But…

Any of my mates can tell me whether I should go and see Prometheus or not. Most in words of one syllable. Most in one word. They don’t need to give me the full picture. They don’t need to tell me what it’s about. In the same way, fan writing doesn’t need to paint a picture of the world – it just needs to tell me if the multi-player is any cop, or if the level design has been sorted out this time.

Ultimately great, professional critics are guides not friends. They show you worlds you didn’t even know existed, they make places you think you might not like interesting, and they find links between stuff you know and new stuff you don’t. When I’m mountain biking, it’s great to go with mates. But sometimes, if I’m going somewhere new and gnarly, I’d like a professional opinion – based on experience and their ability to size me up. I don’t need a mate telling me “it’ll be fine”, I’d like a guide saying “take it careful on the first right”.

There is, IMHO, a clear but fine line between matey writing and fan writing, between a guide and a friend, between what should be the best of professional criticism and the best of forum reviews. And the two should be different from each other – not much at times, but always a bit.

Yeah, everybody has a different methodology.

“the job of a professional critic is to get as many as we can”

I only agree with that in the most literal sense.

Ultimately it’s not your job to get readers, that’s the owner of the publication’s problem. Your job is to write some quality reviews and articles, and the editors job is to make sure that you do. The story is the important bit, not the reader.

Some may be the “critics critic” writing stuff that loses the layman, and some may be the harmless middleground, where everybody can kinda nod their heads.

To me a creative work isn’t a set of parameters. I came into writing after leaving music as a profession, and all I ever learned from dealing with it technically, was that it was never about technique at all.

Any cow can learn how to sing as well as Mariah Carey or Whitney Houston, but it’s far from any cow that’s born with a voice like Nina Simone, or the set of balls it takes to use it right. To put it in provocative terms.

I’m kinda reminded of this Penny Arcade stump:

http://www.penny-arcade.com/2009/02/06

That’s pretty representative of my own views. Intuition, the grasp of a subject, beyond simply technique or knowledge or achievement, is something that you can’t teach people. It’s unique to every person, and that’s your real advantage.

I can’t guide you through a piece of music, anymore than I can guide you through a game. There are too many things going on at once. All I can do is distill that into a piece of text, describing exactly what an album or a game does for me.

That’s my method. It’s not the only one, but that was really the only point I was trying to make. There are millions of words to use, and there are millions of people, with millions of ideas for rules on how to use them.

I definitely think I’ve become a better writer for studying the basic journalistic ethics and methods, but I’ll never be a guide, I’ll just be the guy who’s there, telling a story of what I see.

Wow, lots of opinions from journalists here, makes for interesting reading.

Firstly, THIS IS WHAT NEOGAF IS FOR. This happens every single day there, albeit indirectly and discretely. You just need to keep an eye out for it. This isn’t a bad thing, it’s actually an very agile way of responding to feedback and incorporating key learnings.

Secondly, Dave – I really like your content on VG247, I think it’s great and if/when we meet IRL, I’d like to buy you a beer. That said, I remember reading your GamesTM review of Mortal Kombat and I have to say it was pretty poor. Granted it’s your own personal opinion and, IIRC, your point that all the characters animated at the same speed is valid. That said, the breadth of content offered on the disc and the depth of the core combat mechanics more than made up for it. Okay, so you could spam certain attacks, but that’s true for virtually all one-on-one fighters. Personally, I reckon it’s a zillion times better than Marvel vs Capcom 3, which is attested to by most of your peers, however GamesTM scored that 8/10 and MK9 got 6/10. Maybe you didn’t score that game, but IMHO you ought to have been aware of the rest of GamesTM’s reviews so as to put your own in context. I genuinely think you were in error, but that’s just IMHO and it’s not like this is personal in any way. Please don’t take this the wrong way!

Which brings me to my next point.

EVERYONE IS ENTITLED TO THEIR OPINION, BUT…

…and I appreciate that Jim Sterling and Pat get this already, however…

…METACRITIC SCORES AFFECT WAGES. When a games gets a rogue 4/10 when all the other review scores are an 8 or a 9, bringing the Metascore below a publisher-defined reward threshold, real living-&-breathing people lose out. They’ve slogged their guts out making a game for two or three years on not very great pay and all of a sudden their big reward is snatched from them by some random cunt with a blog and an opinion. How the hell is that fair? Yes, you could lay the blame at Metacritic for allowing shonky sites on their system (and that really is a big problem), but I honestly don’t know how people like that can sleep at night. Both the journos and the publishers that tie bonuses to Metascores instead of sales.

(As an aside, what really fucks me off is when people get called “lazy developers”. Honestly, the hours that devs put into this industry, frequently uncredited and often both underpaid and undervalued… breaks my heart, and I mean that. You think writing to multiple deadlines is hard? Try working as a line programmer on a AAA game.)

Is it somebody’s responsibility as a journalist to recognise this fact and have it affect their review? Absolutely not. Is it what I’d expect from any decent human being, let alone a passionate gamer? Fuck *yeah* it is!

If a game is genuinely shit, fair enough. But too often I’ve seen small or no-name sites give a game an anomalous low review score – presumably to draw attention and attract hits – leading to a bunch of hard-working devs missing out on the bonus that would have made the previous nine months of crunch worthwhile.

As for the scores – can we please move to five start ratings already? I saw a site give a 3 and half star review the other day. What the fuck is the point of that?!?! That’s a ten point scale!! Marking out of five – just like the movies – makes sense. Everybody would buy a 3/5 star game – 60%? Notsomuch.

As for this whole five being an average thing – FFS wake up, it hasn’t been like that since 1993. Recognise the reality of the situation instead of living in some fantasy best-case-scenario dreamland. We all know that 7 is average… well it was until 2009, now 8 is average and all of a sudden a 7 means your project’s sequel gets canned. This is unfortunately the result of their being (a) too many games competing in the same space and (b) too many whiny entitled gamers on the internet. Myself included, I’m no exception.

Shatner was being a dick when he said journos wren’t “a part” of the industry, but I’ll tell you something… Just like PRs, there are some journos who *are* industry and others who definitely are not. This is usually the point where I’d say “fuck PR!”, but I’ve met some great PRs recently and it’s made me readjust my attitudes. That said, if I had to deal with a videogames PR who had more of an interest in shoes and clubbing in Ibiza than gaming – one who, for example, didn’t even know how to plug in a console, let alone demo the game they’re meant to be promoting – I’d probably want to hand out a shitty score just because of the bad experience and I’d probably feel justified too.

Back on topic – if a reviewer can potentially affect a games sales and or dev earnings by a rogue, out-of-whack score, then not only do I think devs and PRs have the right to reply, I think Metacritic should take its responsibilities far more seriously and remove them from the site.

I think you might be the first person I’ve ever seen that thinks Mortal Kombat is a better fighting game than Marvel 3.

That completely boggles my mind.

Check Metacritic or GameRankings, I’m clearly not the only one.

Great Post Gheritt White. Really I mean it. Nice.

@AbsoluteZ

I also think the MK games are at least as good as Marvel. The reason isn’t gameplay for me. Almost all fighters boil down to button mashers, I think that is a given. But the reason is value. the sheer number of unlockables in some of the MK games were killer. But I can’t really speak for the latest releases from either franchise. I own both, but the DLC crap muddels it for me.

@64 The recent Mortal Kombat had some downloadable characters after launch and a few classic costumes to buy, but as far as sheer content goes, the game was absolutely bursting at the seams with it.

The same certainly couldn’t be said for Capcom’s effort…

No journalist has ever taken a penny from a developer. Blaming a reviewer for a practice that he has nothing to do with, is just ridiculous.

If game developers were somehow slaves who had no choice in what they did, who they worked for, or under what conditions, then maybe I could see your point. I wouldn’t like to cause a slave to get whipped.

They aren’t though.

You’re talking about well paid, well educated people, who chose to work for cunts. That’s their problem, not the reviewers, and working for cunts can’t possibly mean that you should get preferential treatment.

Metacritics problem has nothing to do with the sites they allow, but with the model they use. If a site uses a 1-5 scale then that inexplicably gets converted into Metacritics own 1-100 scale, which means that a 2 somehow becomes a 40, and a 1 somehow becomes a 20.

It boggles the mind that the industry, the audience, and most of all Metacritic don’t see a pretty obvious problem with that. It’s patently idiotic, and misrepresentative by design.

Calling for a press where reviewers get boycotted the moment they dare to disagree is just fucking ridiculous, and even though I respect you Gheritt, I think it’s that sort of backwards logic that keeps games from being respected as a whole.

It’s simple: Look at movies. Most movie critics disagree with eachother. The same thing goes for music. There is no consensus, and people are proud of that. Somehow games is the only place where critics acting and thinking independently, somehow becomes a huge disservice to their audience. No one seems to care about the merit of a review, only about how well it matches their own opinion.

And movies also have a great aggregate in Rotten Tomatoes, which simply tallies reviews according to whether they’re positive or negative. That way each reviewer only gets one vote to add to the hat, regardless of score, and no one is ever able to cause a massive swing one way or the other.

Once a movie gets 60% positive reviews, it’s considered “Fresh”. Which also means that if you want to tamper with the score, then you’ll need to buy a lot more than one journalist.

“Almost all fighters boil down to button mashers, I think that is a given.”

Abuh guh?

What? Theres a reason that Marvel was the main event at Evo this year and Mortal Kombat was a side-line event, its like claiming Starcraft 2 is a lesser game because Company of Heroes has more content.

I stand by that one AZ, sorry. Means nothing tho, just my two cents. But I have seen skilled vets get their asses kicked by some noob pounding buttons.

@ 65: I get your point (…apart from the buit about respectngme, that’s just crazy talk).

“You’re talking about well paid, well educated people, who chose to work for cunts. That’s their problem, not the reviewers, and working for cunts can’t possibly mean that you should get preferential treatment.”

That’s a gross oversimplification, especially as these practices aren’t the exception, they’re the norm. Yeah, somebody could leave, but where would they go to that’s any different whatsoever? It’s the standard rule. Devs don’t get paid all that well (unless you’re super-senior) in comparison to other sectors of the market (your passion is frequently leveraged as a means of enforcing unpaid overtime) – you’re big bonus is the carrot. How this went from being measured against sales to being measured against a Metascore is beyond me, but that’s where we are for all sectors of the market aside from iOS/Android devs and indies. Won’t be long until Apple user reviews are employed in the same way, mark my words. It’s a travesty, but it’s too, late to turn the clock back. So, you have to respect the fact that if you want to work in console dev, you will be subjected to this, no exception.

I agree the Metascore model is flawed and broken and Rotten Tomatoes is *much* better, But now the publishers have a taste for it, if it wasn’t Matacritic it’d just be GameRankings… or even GameStats (p’too, ick).

Calling for a press where reviewers get boycotted the moment they dare to disagree is just fucking ridiculous, I think it’s that sort of backwards logic that keeps games from being respected as a whole.

I agree with you – I hate the way certain games like Halo and GTA just get nines and tens by rote, fuck that noise. But you *know* what I’;m talking about – smaller sites using lo scores on big games to make a name for themselves. I understand the reasoning, but much like a software pirate they’re unaware how their action affect people’s livelihood in very real ways.

While journalism is a noble profession, I don’t think games journalists can be oblivious to the all-too-tangible ramifications of their actions just because, in an ideal world, this set of circumstances would never ever occur.

@68 When I’m in the US, I’m staying in the South. I see people working up to three jobs, none of which they enjoy, some join the army just to survive, some are working just to go to college, which still doesn’t save them from running up a massive debt, which is so incredibly far from anything I see back home in safe, comfortable Denmark.

I simply cannot take it seriously when people with perfectly good jobs stomp their feet for losing a bonus, even if I can see how it sucks not to get it.

Ultimately, it’s in their court. If they’re underpaid, they need to do something about it. They can’t just pass the buck to someone who isn’t even a direct part of the industry. Do something about it.

As for critics, it’s just something I see thrown around a lot whenever someone sticks out, even a little bit. It’s like in the Penny Arcade piece I linked earlier.

People aren’t looking at merit, they’re looking at how much they agree with something. Gaming is the one scene where everybody agrees. To me having someone disagree would be a sign of progress. And if they’re a contrarian, then it hardly matters, because they’re well and truly alone in that.

Most reviewers happily slap an 8 onto something they’ve forgotten about next week. I don’t know whether to laugh or cry when I see one of those guys go “This is one game I’m actually going to keep playing after this review”. Most of them have zero credibility. It’s just god-awful.

Okay, I get your point that devs aren’t destitute, but they’re highly skilled labourers – I assume you know about crunch and the ins and outs of dev cycles. EA wives is very much alive and kicking in 2012, i’m sorry to say. A contact of mine recently said that he thought agile and scrum has been effected just as a means of extracting ever more unpaid overtime from devs – that all these managerial drives only increase efficiency at the cost of a workers work/life balance. I <3 agile, but I'm sorry to say he might have a point.

What I'm trying to say is this – have a heart, FFS. We all know that being a journo is a tough gig even when you're at the top (…Rupert Loman aside, j/k). It's certainly no less tough being in dev, so as I said, have a heart. Remember that your critical analysis doesn't exist in a vacuum or a purely philosophical realm – *you* (as in journalists) have the power to fuck up somebody's career… and I don't mean the design/art/programming leads, I mean the grunts on the shop floor working for a break.

As for reviews, they need to split into two types – critiques and buyers guides. Critiques should be long-form and have no score. Buyer's guides should be short (say 800 words) and be marked out of five stars. Equally, Metacritic and GameRankings should readjust to the Rotten Tomatoes formula. These two actions would have an enormous positive impact in people's lives. Okay, so we're talking first world people, but it's something that people I've actually met have the power to do.

Why will this never happen? Because – and here's the kicker – journos need to eat too and those scores and controversial headlines is what puts food on *their* table.

I've essentially just argued myself into a corner, haven't I? Still, it irks me to see people who can only be described as talentless hacks (I'M LOOKING AT YOU, N'GAI CROAL) make a killing while entire studios suffer because of a rogue score or two.

I get your point though, but I also don’t see how it’s possible.

You can’t afford to be best buds with people, when you’re covering commercial products. It directly impacts your credibility. I think journalistic integrity becomes twice as important when money is involved.

If you’re considering wives, kids, tuitions, working conditions when you’re reviewing a game, then it all detracts from what you should be doing, which is relating to the game.

There just can’t be any mercy (or cruelty) if you want to be a reviewer or a reporter. You can’t add redeeming qualities that don’t come with the disc itself. You should always be fair though, and you should always remember that you’re dealing with peoples hard work. It’s the least you can do.

Objective journalism is an ideal, not a reality. I’ve reported on people I’ve really, really liked, but I’ve never reviewed them, and I can’t say I would bury or even angle a story a certain way for their sake. Ultimately you back the story, not the person.