Login | Signup

A Quick Word About Review Scores... And Why They Don't Matter

Author:
Jonathan Lester
Category:
Dealspwn
Tags:
Reviews, Scores, Stars

A Quick Word About Review Scores... And Why They Don't Matter

Reviewing games is one of the most important things we do here at Dealspwn - hopefully empowering our readers with the knowledge they need to make an informed buying decision. However, many gaming sites (us included) frequently come under fire for the little number we put at the end.

Guys, look. Review scores are primarily aimed at publishers and developers who rely on the data as the only quantitative measure of a game's quality as opposed to raw sales figures. Shareholders and executives are driven by raw numbers - it's all they understand. We put great pains into making sure that our review scores are as representative and accurate as possible, using a strict set of marking criteria behind the scenes.

But you guys aren't a spreadsheet or an algorithm. You're unique individuals whose gaming preferences are based on your own gaming experience and tastes. A number can't take that into account. We painstakingly write thousands of words that are designed to give you as broad a look as possible into the game in question, as well as pros and cons that sum it up. This is the important bit; this is the bit that matters. To gamers, a score should be just a number - and if it's above 5, you should probably check it out, read around and see if it will appeal to you. In effect, the scores shouldn't matter... and they're definitely not worth taking too seriously.

Add a comment5 comments
hurrakan  Sep. 21, 2011 at 17:27

Reviews that are well-written and objective ARE important to me - but sadly not many are. And the journalism/writing in most reviews is not to a professional level.

Recently there seem to be more reviewers that are too young - they aren't old enough to remember and discuss older games that might be relevant. And they don't bother to do any proper research either.

My advice is to find a few reviewers who you trust and who seem to have similar interests. Then only listen to those reviews and ignore everything else.

I find that I am frequently in extreme dis-agreement with the reviews on Eurogamer - so now I completely ignore them.

Metacritic cannot be trusted - but it can be useful to get a loose idea of how good a game is.

Reviews that I currently listen to:

bit-tech.net
dealspawn
kotaku

Late  Sep. 21, 2011 at 17:43

I give your quick word about review scores 6/10

JonLester  Sep. 21, 2011 at 18:23

We need pros and cons too, Late. ;)

MattGardner  Sep. 22, 2011 at 01:00

My advice is to find a few reviewers who you trust and who seem to have similar interests. Then only listen to those reviews and ignore everything else.


^This

We do try to ensure that games are given a fair shout here on Dealspwn. By that I mean that all of our writers appreciate different things more than others. We wouldn't ask Dave (who loathes JRPGs) to review a Final Fantasy title, for example. And I would be completely out of my depth attempting to review a sprawling MMO. Where there's appreciation there tends to be a knowledge base, though it must be said that careful background research is conducted where possible.

Historical research is paramount. Reviews can only really be conducted by precedent and comparison. It is only really by looking back at the benchmarks of the past that we can try to move forward. Reviews, once written, essentially become bookmarks for posterity. You wouldn't, for example, review the original Mass Effect from the same perspective now as you might have done four years ago. It would be impossible to do so.

Finally, the fact that something as arbitrary and subjective as 'fun' or 'enjoyment value' comes into play in reviews should highlight the fact that what we preach is not exactly gospel.

fanpages  Sep. 22, 2011 at 07:55

I was discussing this with another HotUKDeals member a few days ago (who writes reviews for TheAverageGamer). Just a fact; not an attempt to advertise the site.

I asked if he consciously does not rate/score (out of 5, 10, 100) the titles reviewed, or whether that is a site policy, or even a personal preference?

His reply was that it was part of the site review policy; No scores, just the review, & whether it is worth getting or not.

He was mindful that in doing so the site does not get mentioned by Metacritic & that the resultant visits to the site may well increase because of inclusion there, but it would, & I quote, "also open a whole new can of worms in itself"

He also added that he thought it was "actually quite refreshing not to have scores" & "...the whole scoring thing is pretty flawed anyway".

My response was that I appreciated all review scoring is flawed (as nobody is truly unbiased on any subject as their personal preferences/experiences influence the way the review is approach), but I was thinking in terms of quoting some form of (site-based) scoring system mainly for a quick reference guide to compare similar game titles across all the site reviews.

If you do not have a summary of each review, a visitor to the site cannot quickly compare one game to another. For example, "I'm in the mood to buy a first person shooter... I'll just see which recent game InsertSiteOfChoiceHere prefers...". There is no "headline" score with snippets (sound bites) of key quotes from the review text to draw the reader in. Just "walls of text" that are difficult to compare/contrast if the reviewer is different &/or the structure/presentation of the reviews change.

When I write reviews I also prefer not to score them, but structure each in a similar manner so that all can be compared side-by-side (if desired).

As mentioned above, & as I have discussed a few times (over at Dealspwn.com sister-site HotUKDeals.com), I also adopt the approach of reading just one or two preferred sources based on past similarities to my findings after seeing or playing a particular game. If I find I can "trust" a reviewer then I will readily accept their opinion in the future; although, that does not necessarily follow. If those industry journalists move to other organisations (or are freelance) I look for their new outlets rather than sticking to the same sources where previous work was published.

BFN,

fp.

Email Address:

You don't need an account to comment. Just enter your email address. We'll keep it private.