Failing grades: why review scores are stupid

FailingGradesDesistSmile

The degree to which video game review scores are taken seriously astounds me. I don’t mean that the reviews themselves are lacking in insight, though they sometimes are. I’m referring to the score itself, an anomaly oftentimes based on nothing at all, that sows confusion amongst legions of game players.

How does one review a game? To write the review, you think about the game you played: its standout moments, the tightness of control, the depth of ideas on display. Maybe you write about the visuals: they were technically quite basic, but relied on an abstracted style that set the tone of the work.

And then you slap a number on that review and call it a day.

GodHandAerial

There’s more to it than that, obviously. Most sites and magazines review games on a scale of ten, or on a scale of stars and half-stars that might as well be ten. There are protocols for each site, guidelines as to what each number means, what’s fair criteria for marking up or down. They try to be completely objective about it.

Roger Ebert just died. No matter what you think of him, he was a reviewer who said exactly what he thought about a film. He cared about artistry and craft, but knew there was more than just technique involved in creating and experiencing film. There was a connection between artist and audience.

I love Kick-Ass. It’s one of the greatest action movies I’ve ever seen. Ebert hated it. He gave it an abysmal score, saying that he couldn’t justify praising a movie so repugnant, a movie that, if he scored it well, might be seen by impressionable minors. He thought it was too senselessly violent, and said so right there in his review. He spoke his mind. He reviewed. I respect him for that.

Roger Ebert never seemed bound by review scores. They’re just an at-a-glance rubric that leans on the review to tell the full story. But when I see videogame reviews, I feel that reviewers are getting too tied up in their scores.

Uncharted2

At the very least, I know that I personally have no mind for it. I don’t care for Uncharted 2. It looks fantastic, with incredibly gorgeous colors, and the third-person combat is technically very accomplished.

But I don’t like the way it feels, its entire ethos that everything should be so incredibly simple, to the point where every action just slips from my fingers with the slightest input. It feels too helpful, like it’s pushing buttons for me, and there’s a disconnect between what I do and what Drake does.

Having to assign a numerical score to Uncharted 2 would undermine everything I feel about the game. I don’t care about it at all, but, were I to review it, I’m not sure I could give it less than an eight. It’s completely bland, but objectively it’s completely solid, right? It’s just so polished.

I think that’s the operative point we look for when we review games. Polish. In other words, is there anything really wrong with it? If not, how can I give it a low score, and how can it be boring? I don’t know. It just is. To me.

In honesty, my opinion of Uncharted 2 puts it in a subset of games that I don’t have any interest in playing. It doesn’t really matter the degree to which I don’t care about it, because it just doesn’t thrill me. Is that a seven or a five? Four three two one?

KaneandLynch2

Whatever you do, if you’re reviewing for a widely read site, don’t give any big name game a low score. That’s just asking for trouble. Remember Twilight Princess? Kane and Lynch 2? You don’t want to be that guy. That guy gets in trouble, gets fired, and starts his own highly profitable gaming website, then gets bought out for big bucks. Don’t be that guy.

Lately, people are obsessed with the idea of the paid-off reviewer. A mediocre game gets a good score, and all of a sudden there’s a mystery to be solved. The publisher must be offering the reviewer heaps of money for that score, right? Absolutely not. A junior bacon cheeseburger will suffice.

In seriousness though, the idea is ludicrously stupid. Like absolutely just dumb. I don’t mean to belittle anyone who holds this idea, but it really is delusional. I won’t say that it never happens, because things that are technically possible are technically probable.

But why would a publisher or developer take the risk of bribing reviewers? All it takes is one disgruntled employee to say their website takes paid reviews. All it takes is one website to say that someone offered them a bribe, and throw into doubt every other website and all the positive scores that game has received.

Of course there’s an unsavory relationship between (operative word coming up) some games journalists and publishers. Publishers fly journalists out to parties where they play new games in an insular environment, where they might be biased to give the game favorable coverage and reviews. Previews of games are often little more than optimistic shilling. Websites are offered early review privileges. All clear problems with the industry. But to say they’re outright bribed is complete naivete.

DeadlyPremonition

Games journalists can do no right anyways. They’re accused of bribery if they give a positive review score, and they’re crucified if they give a negative review score. There’s no winning. They’re too stupid to do their job, says the internet.

I know how it feels to see your favorite games get low review scores. Nobody ever in the world is giving Cave’s bullet hell games positive anything. I read the reviews and see the scores and it’s clear to me that they just don’t get it, man.

Except that it’s totally their opinion and nobody else’s. Maybe they were bound by reviewer guidelines to mark it down (or up!) a little more than they would have otherwise, but they still generally did or did not like that game, and nobody else should be telling them how to review.

The onus is on the reader to either accept the review or stop reading it, assuming they’re reading it in the first place and not just complaining about the score. We need more honesty, not hand-holding feel-good eight out of ten everything.

There are certainly aspects of reviews that can be criticized, same as anything else. But in large part, reviews are just opinions. That’s all. One person’s opinion with some sidebar fluff and a score.

Diablo3

Negative reactions to scoring are holding back discourse in all mediums. The Dark Knight Rises received favorable reviews, and before it even came out, fans harassed reviewers to no end because the reviews weren’t positive enough.

I saw The Dark Knight Rises, and it was the most overwhelmingly mediocre movie I think I’ve ever seen in theaters. I sat there completely astounded that this was a Christopher Nolan film. I was almost in awe at the incredible boredom I was feeling. It didn’t even feel like a Nolan film. I don’t even think he tried, though maybe he tried very hard.

Review scores are a large part of the problem. They’re too inflated, and they’re taken too seriously as an overall reflection of the quality of a work, rather than the opinion of one person. Since anything remotely enjoyable gets an eight or higher, it’s taken as a massive affront when a reviewer just doesn’t enjoy it.

Why does anybody even use the ten point scale? There’s nothing to be gained from it. In college, me and my friends jokingly rated everything on the Game Informer scale. 2001: A Space Odyssey? Uuuuuuugh I dunno it’s soooooo good… 9.75 Game Informer scale. Dokapon Kingdom? Absolute garbage with a great premise; 2.5 Game Informer scale, fasho.

Everything was incredibly high or incredibly low. A mediocre work would get a “meh” and a seven or eight. Seventy-five percent of the scale was devoted to things we didn’t even like.

DragonAge2

There’s really no point to all the fluff in-between. When we take review scores seriously, we’re basically closing ourselves off to anything that scores below a seven, and the numbers become a scale to determine how crap a game is.

In my mind, there’s no reason to have anything more than a four star scale. Four stars, no in-betweeners. When scoring games, these are the points that matter:

  • One star: An unenjoyable game which doesn’t display any general merit
  • Two stars: Didn’t appeal to me, but I can see its merits and maybe you’d enjoy it
  • Three stars: Has merit and I enjoyed it, but it didn’t blow me away
  • Four stars: Awesome, and will likely stand as one of my favorites of its kind

And you can reduce further. Three points of interest: unenjoyable, recommended, and great. Or maybe just two, recommended or no. But then you might as well just not score them at all.

SimCity

I don’t score the games I review, for a couple of reasons. Firstly, I’m not beholden to anyone, so I mostly just play whatever I want and write up a review if I feel strongly about any aspect of its design. If I assigned scores, they’d mostly just be positive anyhow. I’m not likely to play a game I hate for long enough to inform an opinion on it.

But also, I really don’t want to be bound by my scores. It’s not cowardice: I’m definitely not afraid of any of the four people who read my blog on a given day. You guys are nice. You guys are the greatest.

When I write a review, though, I want to be able to talk about any aspect of a game. If I find that the visual design is the most interesting part of a game, I want to be free to write about that for four hundred pages. Which is really antithetical to giving it a fair review score.

Guilty Gear X2 Reload has awesome character designs and lots of them. Nine out of ten.

Of course, no website in their right mind would do away with review scores altogether, because then they won’t be aggregated on Metacritic, right? It’s a sad state to be in, but you gotta make that money. There’s really no way around it.

Femshep

I guess what I’d really like is for people not to care so much about scores. I want reviews from mainstream websites that get at the heart of the experience, even if that means the review is completely unbalanced toward one small aspect of the design. I’d like people to be accepting of that reviewer’s subjective experience.

Subjectivity is a dirty word in the industry, but reviewing is far from objective. There’s nothing in the code that makes a game good; there’s only our reactions and interactions, our sense of a game’s style and how well it’s pulled off. Ask two people how good Call of Duty is, and you’re bound to get two wildly different answers.

What makes a review interesting isn’t objectivity. It’s the reviewer’s sense of style meeting their ability to verbalize why they do or don’t like a game. The opinion isn’t as important as the reason for the opinion, because that what allows readers to measure it against their own tastes.

Roger Ebert had some occasionally crap opinions, but he told me enough about his experience that I knew whether or not it applied to me. He often talked about small segments of a work, to really get at the heart of his experience rather than just a broad view. To me, that’s good reviewing.

When you review, write about what’s important to you. You’ll likely end up writing far more interesting reviews, ones that give readers a better idea of what the game might mean to them. Maybe you’ll end up writing something that can be read or enjoyed on its own instead of just in tandem with the work itself.

There’s more to reviews than just recommending games. Video games are an incredibly complicated medium with a wealth of topics to talk about. And there’s certainly more to talk about than whether a certain website likes a certain game, right? Right.

Now, if you’ll excuse me, I’m gonna go look up review scores for Devil Summoner 2.

Advertisements

Tags: , , , , ,

10 Responses to “Failing grades: why review scores are stupid”

  1. Squire Grooktook Says:

    I have to agree. I really don’t feel right about rating games on a number scale. Games are an experience for me, and how can you rate an experience? You might like more then others, but you’re ultimately comparing apples to oranges.

    If I’m interested in a game, I try to find out what it’s really about. What kind of experience it is. What it feels like to play. What kind of skills it demands or develops. If someone could answer me those questions about a game, that would be a whole lot more meaningful to whether I invest in it or not then knowing if they thought it was a 7 or an 8.

    And oh yeah, Dark Knight Rises. I was disappointing by that movie. I didn’t think it was bad, in fact it was probably quite good. I just felt like, aside from the ending, it didn’t have the power or the emotional depth that I came to it expecting. It just felt like it wanted to be an “epic” but fell short of achieving anything of the sort.

    • catstronaut Says:

      About the game experience and feel, I love when a reviewer says something incredibly specific about the core systems of the game – that there’s an incredibly satisfaction to the forward boost or that it has the most satisfying shotgun sound in gaming history. Sometimes, there’s these incredibly small little nuances that can completely make or break a game, and we sometimes miss out on relating them if we’re trying to be fair and balanced.

  2. karlosmorale Says:

    Another voice in agreement with your post. Review scores have little use IF YOU ARE A GAMER, even a casual with more than a passing interest. I can understand using a star rating in a newspaper or a general interest magazine that is writing for people who maybe buy a couple of games a year – after all they don’t have time (or inclination) to find a reviewer they trust or find gameplay videos, play demos etc. A score might be a useful shortcut for that audience.

    • catstronaut Says:

      Yeah, I agree that it could be helpful for an outside audience, though hopefully a review geared towards them would be kept short, simple, and free of jargon. In the case of newspapers, I think review scores would be of primary interest to people who don’t play games but want to buy for somebody else. But for people who only play a few games per year, I find that the games they play tend to be incredibly stable. You know approximately how good the next Madden, Call of Duty, or Halo game will be. There’s the occasional popular upset like Resident Evil 6, but a lot of these games barely even need reviews. You know what the new Call of Duty is gonna be, so there’s hardly even any real reason to read about it, is there? Of course, having informed reviews for every game is always good, but I don’t think people would be significantly influenced either way unless a game received phenomenal or terrible reviews.

  3. Paige Six Says:

    I genuinely loved this article! You make some great points and address a lot of the hurdles one has to face when trying to objectively review a game. I often miss the old Playstation Magazines which offered 3 different perspectives and scores from 3 different kinds of game journalists with their own tastes. Just because I loved one game doesn’t mean my opinion and hence base score will be accurate for you. This is something the online community just needs to fix, especially for larger sites with the funds and who hold the weight/reputation and the ability to make or break a game.

    • catstronaut Says:

      Thanks! I always liked that format too, and I think having multiple reviewers on the same game encourages them to provide more of their own experience. I also miss the days when games magazines weren’t so damned serious – have some fun with it, guys and gals.

  4. Albedosrighthand Says:

    Great article. For me I cant stand review scores. No matter how they get scaled they just fail to capture the essence of the game. I use a simpler system. I say my feeling on the game I played and add a note after it like: “I enjoyed the game, but If anime isn’t your thing then stay away” or “I couldn’t stand the game, but there was this one awesome moment when…” etc etc. Basically Iam trying to show both sides of the coin, so to speak.

  5. Squire Grooktook Says:

    You know, thinking about this article, there’s another thing that struck me.

    I think part of the problem that leads to this is that games are a very interpretive genre, and the different aspects of a game are a lot easier to compartmentalize.

    With a movie, if the story sucks, then that’s it. It would be perfectly acceptable to review it and give it a below average score with the reason being “the story sucks”. But with a game, maybe not everyone is playing it for the story.

    Maybe people a good portion of players don’t care about the story, and are interested in the gameplay. Maybe a good portion of them do care about the story, and the gameplay is irrelevant. Maybe a good portion of them don’t care about either, and are just buying it for the multiplayer?

    Maybe these players like the feeling of empowerment and destruction in this action game. Maybe these players prefer a feeling of vulnerability and caution.

    Maybe these players want to play a hard game for the achievement of victory? Maybe these players want to play a hard game for the excitement of the moment to moment gameplay and don’t care too much about winning? Maybe these players don’t care about difficulty at all?

    When a single game might have several different appeals or interpretations, and when aspects that the player doesn’t like or simply doesn’t care about can easily be ignored or compartmentalized, it becomes a lot harder to give a review without pissing people off, and a lot easier to just cop out and score the game based on whether it’s technically solid or worth the price tag.

    • catstronaut Says:

      I definitely agree – there’s so much that goes into a game that reviewers really have to hone in on exactly what is or isn’t interesting about an experience. There could be a bunch of fluff that doesn’t matter in the end, because of that one mechanic that’s just nailed perfectly, or that one story beat that really resonates.

      I do think this applies to other mediums, but maybe not to the same extent as in videogames. Movies like Stalker and Begotten have a near-total lack of story, but are praised by some critics for their visual technique. Some indie comics have simplistic art but receive accolades for their affecting characters.

      Videogames may not be the “ultimate” medium, as some like to say, but they are incredibly involved, and any major slip-up can have a massive effect on how the game is perceived. Oftentimes, we also have incredibly basic reasons for a dramatic shift in feeling. While we might not directly compare fantasy novels to autobiographies, we often see people skipping out on entire genres of videogames because they don’t reach the same playlength as others, which is both dumb and smart at the same time.

      Both the overlying and underlying character of videogames are incredibly strong, and it’s very hard to be drawn in if either is lacking. There’s so much to take stock of that it can be mind-boggling. On the other hand, we may be quick to forgive major failings in certain areas, as long as our core concerns are met. So maybe there’s a little fudge room, as well.

      I think it’s important that reviewers let readers know what those core concerns are, saying what works for them and what doesn’t matter. The reader will make their own informed decision, but first they have to know.

  6. qishmish Says:

    1-10 system differs from 1-5 grade.
    8/10 is not 4/5. 8, actually, is “excellent”, so 10/10 is not 5/5 but smth like 5++/5

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: