Spector: journalists and players may be “just mediocre” at assessing success

Tuesday, 14th May 2013 02:31 GMT By Brenna Hillier

In a lengthy musing on how we should measure success, Deus Ex and Epic Mickey creator Warren Spector has cast aspersions on the ability of critics and gamers to judge whether games are any good or not.

When we put our faith in Metacritic as an impartial, scientific measure of quality, we should probably ask ourselves whether the crowd – the crowd of journalists as well as players – is really wise or just mediocre, incapable of recognizing and rewarding the new and different.”

“Gamers have every right to evaluate games themselves, and to share their opinions with other gamers. It would be insane to argue against that, or even to think about it too much,” Spctor wriote in his now-regular column on GamesIndustry.

“Reviews and reviewers? That’s a very different story. We – gamers, developers, publishers and reviewers themselves – need to think more about the purpose of reviews. Only by doing this will we get better reviews and understand how best to interpret them, as business people, creators and consumers.”

Spector said that reviewers assess games to say whether they’re good or bad, but fail to address any of the criteria which he considers important to measuring a game’s success – something he believes reviewers in other media – like film – do.

“Great critics don’t focus on questions of good or bad. Their assessments of a film’s success or failure are supported by an underlying philosophy they apply to all films. It doesn’t matter if you agree or disagree with their assessment of an individual film – heck, they often disagreed with one another. What matters is that, reading them, you can weigh your own likes and dislikes to determine if you will like a film, based on their review,” he said.

“An enthusiastic ‘This game sucks!’ or ‘The AI is bad’ or ‘This game gets a 4 out of 5′ tells me next to nothing I need to know,” he added.

Moving on to Metacritic, Spector argued that by aggregating critical voices and reducing them to a number, the resulting average often fails to account for the kinds of games which reach niche audience, or innovate substantially.

“By this logic, Metacritic, at best, rewards games that are conventional and well understood by players and critics alike. New and challenging things are, by their very nature, disruptive and easily misunderstood,” he said.

“When we put our faith in Metacritic as an impartial, scientific measure of quality, we should probably ask ourselves whether the crowd – the crowd of journalists as well as players – is really wise or just mediocre, incapable of recognizing and rewarding the new and different,” he continued.

Ouch. Spector also took issue with Metacritic’s policies, saying it cherry picks specific reviews, the lack of transparency of its weighting system, Metacritic’s conversion of all scores to a 100 point scale. He also said that Disney’s Epic Mickey, one of his lowest-rated games, sold substantially better than his highest-rated efforts, Thief and Deus Ex.

“Reviewers and some gamers may have preferred Deus Ex and Thief (and I have the fan mail to prove it!), but I received substantially more fan mail – and more heartfelt messages of thanks, and more fan art, and more everything – on the Disney Epic Mickey games than on both Deus Ex games I worked on combined,” he said.

“Metacritic be damned – I’ll take an emotional connection with players and the praise of Disney fans any day of the week”

Spector ended with an exhortation fo developers to measure their success by other means.



  1. Jerykk

    Wait… is he saying that he thinks Epic Mickey is better than both Deus Ex and Thief? If so, his opinions are officially null and void. Obviously a licensed console game featuring Disney characters will have broader appeal than an original IP for hardcore PC gamers, but quantity isn’t everything. More people watch Dancing with the Stars/America’s Got Talent/random worthless reality TV show than Schindler’s List. According to Spector, that apparently makes them better than Schindler’s List.

    On top of all that, he contradicts himself. He says that reviewers don’t appreciate unique or niche games. That’s generally true but Epic Mickey was not unique or niche in any way. It was the exact opposite, catering to the lowest common denominator of children and casual gamers.

    #1 2 years ago
  2. Cobra951

    Popularity and quality are two independent concepts. I’m surprised he doesn’t seem to know better. I agree that it’s troublesome to aggregate scores from subjective reviews which don’t even use the same weighted scale. But without actually buying and playing every game that comes out, we need to trust someone to help us not waste our money. So I choose to overlook the issues, and use the evaluation tools available as best as I can.

    #2 2 years ago
  3. The_Red

    Wow, the last part about fan emails comparison with Theif / DE vs Epic Mickey. Wow, Warren, you’ve really lost it. First the unprofessional attack on new Wolfenstein (Yeah, because the market is filled with Nazi robots… wait, what?) and then this. By this sort of logic, Honey Boo Boo, Jersey Shore and Bay’s Transformers trilogy are better than Breaking Bad, The Wire and The Godfather trilogy.

    #3 2 years ago
  4. silkvg247

    It’s clear he’s been hurt on the criticisms of EM, but to be honest I actually agree with his overall point.

    Time and again I’e ended up absolutely loving a game that scored very low overall. Games like shining in the darkness (50%ish), Fatal labyrinth (30%ish) still rank as some of my favorite oldies. Then some AAA 90%+ masterpieces feel absolutely “meh”; like Skyrim or Farcry 3.

    So surely he has a point? Just like films.. one mans love is another mans hate. Games should be reviewed with that in mind.. so yeah.. I agree actually.

    #4 2 years ago
  5. Old MacDonald

    Mediocre films also sell well, without getting great reviews. So I don’t see how the situation is so different there.

    #5 2 years ago
  6. sh4dow

    @4: Wow! I think this must be the first post I’ve ever seen of somebody aside from myself who said that they didn’t care for Skyrim! Now I gotta check out Shining in the Darkness and Fatal Labyrinth ;)

    #6 2 years ago

Comments are now closed on this article.