This is constructive criticism, I'm not simply nagging. But anyone else find that Touch Arcade's scoring system for game reviews is uninformative and of little meaning/value/distinction because of the base fact that TA ALWAYS gives ratings that are very high. Almost everything is either 3.5, 4, or 4.5 stars. When you rate 90% of games in this small and high range, there is no distinction between games, and thus the ratings have no meaning/substance. I mean, I don't think anything lower than 2 stars even exists in TA's rating system. And TA consistently rates games that aren't very good and that they criticize a fair amount in the text of a review as 3.5 or occasionally 3. These are games that deserve 1-2.5 stars. Saying 'everything's at least pretty good' as the ratings indicate does nothing for a consumer looking to single out a particularly good games. TA seems to always give benefit of the doubt to games and in general seems scared to rate games lower than 3.5. THE CURVE IS TOO HIGH. Again, when you rate almost all games as 3.5-4.5, it takes meaning away from the ratings, and highly indicates that TA wants to give as many games as possible inflated ratings (why this reason is, I'm not sure, it could well be not wanting to upset developers or sponsors, being scared to be too critical, generally wanting to support the platform so giving most everything overly complementary ratings, etc.).
We only post about games that are good, or otherwise noteworthy. You don't see any 1 star reviews because assigning a writer a game that is clearly terrible is a waste of their time and a waste of the reader's time. People seem to come to TouchArcade for recommendations of what to download, not what to avoid. It's not difficult to find junk on the App Store, you don't need a web site for that. The result of only reviewing good games is a top-heavy scoring system, you're totally correct. Personally, I think review scores are completely pointless, and fought very hard to keep them off TouchArcade. Unfortunately, for as much as I tried, those who wanted review scores pushed back just as hard. I really don't understand what it is about people that requires some kind of quantitative metric to compare things, but, Metacritic wouldn't be the juggernaut that it is if it wasn't for a demand for what they offer. My suggestion is to just read the review and see if it sounds like something you're interested in regardless of how many stars are attached to it. Hell, some of my favorite movies barely break the 6/10 mark on IMDB, but that doesn't stop me from loving them. Edit: And remember, this is how we describe our star rating system: "Our final scores are not the product of any traditionally objective measures such as graphics or sound, but simply reflect the games we would most recommend to others." A three star game is still a "good" game, at least in my eyes. And really, I've had a ton of fun with even the most flawed games. I think Spell Tower was one of the best puzzle games of 2011, but its interface on the iPhone is clunky as hell. Regardless, I've dumped days into it.
Sure, we don't use the full scale. But, like President-In-Chief Eli Hoddap just said, that's because we don't review garbage games. Telling you that a game sucks on a platform clotted with bad games isn't doing anyone any favors. Giving you a heads up on a good one on the other hand? That's what we're all about. I'd say sit back, read the review, and don't even look at the rating. If someone sounds good, just grab it. EDIT: Also, I'm not a big fan of people accusing us of inflating stuff. Go read a magazine and get back to me on inflated scoring.
Agreed. Go ahead and read Gameinformer in particular. Then you will see a "Paid Review" and inflated scoring.
See. If you're really interested in a game or not sure to try/to buy you check other sites as well (yes, there are other iOS-gaming sites - but not for me ).
TA Rating System I tried posting this as a comment to the lastest "So You Just Got an iPhone 5... Let's Get Started" article, but either Disqus won't let me or I'm not allowed to comment on TA anymore (I hope not). Either way, I think the current rating system needs to be adjusted. My original comment is below. -- If you don't give out ratings less than 2 stars, you should move to a 4-star rating system or re-weight the rating distribution over 5 stars. Ratings are only valuable to users when they can be used effectively to convey or compare the quality of games. As it is right now, most of us who frequent TA already know that: 5 stars = Definitely worth checking out 4.5 stars = Very good if you can overlook some flaws 4 stars = An "average" game By "average" game, I mean average amongst the games on the app store with at least a decent amount of polish, not average in terms of every game on the app store. (There is a lot of unplayable stuff out there, and most games would be 4.8 ~5.0 stars if rated on an absolute scale.) Anything under 4 stars we already know not to bother with, be it for poor IAP models or clunky controls. What I'm saying is: positioning TA as a game discovery site is great (it's exactly what many of us use it for) but weighting review ratings higher actually hurts this purpose rather than helps it. We want to know how games compare to other decent games, and it's harder to get this kind of information from a rating system that is skewed by games that even the article admits is not worth bothering with. When ratings are weighted the way they are, savvy TA readers can tell it's in effect a 3-point system with the appearance of a 10-point system.
There is a relevant thread with posts from administrators here: http://forums.toucharcade.com/showthread.php?t=121844 And there has been endless discussion of this topic in many random game threads. Basically it sounds like they only begrudgingly include ratings at all and wish they had never started using them.
It's actually way better than the console world where acceptable score ranges are 8.5/10 to 10/10. But, yeah, I wish we never had to implement review scores at all. Sort of a necessity with the way the app works. I think what causes my intense review score fatigue is how commenters take review scores about 10,000 times more seriously than we do when we're assigning them.
I think most of the time you're never going to please everybody. Give a 5 star review? A vocal minority will critisize the iap "on principle" or hate on the "super short story". Give a lower score? Everyone will rail on you for hating on a game or the devs (as in dead trigger or shadowgun) or proclaim their love for the game adamantly. Dammed if you do, damned if you don't. Of course I've never reviewed anything, but that's the impression I usually get.
I'm not familiar with all the console game ratings systems, but I frequent GiantBomb.com and they have an excellent 5-point rating system with no half-points. But even 8.5 to 10 (effectively a 4-point scale) is more indicative than a scale of 4 to 5 stars (effectively a 3-point scale). I actually would prefer a move back to the old reviews that had no scores. It places an emphasis on the review itself and could lead to better review writing and some interesting discussion. Your review score is your endorsement for a game/product. I'm not sure you should be taking them 10,000 times less seriously than readers. Developers will quote your review scores in their app descriptions, just like restaurants will quote Zagat scores and ratings. The scores should always be something you stand by. If not, just don't implement them.
I said "could". But that isn't even the main point. I won't reiterate as I think my previous posts were pretty clear.
I don't have the time to play lots of games these days. I still have a queue of really great games sitting around that I haven't gotten to. But I do enjoy "keeping up" with gaming news, regardless. Generally if a game is getting 3.5 stars or lower on TA, I know I won't be missing much. Definitely not a GREAT game.
TA needs a new rating system I'm starting this thread out of concern for the current rating system for reviews. Where I don't think it's totally broken, I do sometimes scratch my head when they give a game a perfect score. I truly feel a 5 star rating should be giving maybe a couple times a year. If that. Max Payne mobile was by no means a perfect game and that got a perfect score. That's just one example. I think Eli and the team really need to come up with something a little better. One suggestion would be maybe a 1to10 rating system. This will give you more room to rate a game and you could save the 10 score for true gems that deserve it. Or maybe even go to half stars as well which will give you more room to work as well. I've been a long time supporter of this site, but truly feel that some of these ratings are not warranted, and with the amount of people you have coming to this site for a accurate rating I think we deserve better. I know I'm not the only person that feels this way and I hope people chime in to express there concerns if there are any. If you go to the TA APP and look at all time 5 star reviews, you'll some that really don't belong there. Please TA, save five star scores for truly amazing games. A five star review should not be that easy to get, especially when in the review there ate complaints about the game. Thanks for listening guys! Btw the TA app is awesome.
The home page reviews do have half stars, but agreed, I like 1 to 10 more, I find them easier to uderstand.
A 1-10 scoring system would give a wider spectrum for scores, making the more understandable and, well, more variety Actually, I never even bother checking games with less than 4 stars reviews
Let's adopt a rating system where everything is 9.X where X is the actual rating that way everyone is happy.
Who are you responding to? The consensus in this thread is that, on average, scores should be lower, not higher.