Wednesday, 28 October 2009

What makes a good rating scale?

From time to time I have to fill some form where I have to rate items in a scale such as agree-disagree, like-dislike, etc. And more often that not, I end up cursing the creators of the form for not providing options I needed.

The first obvious problem is when you have an even number of options, e.g.:
Totally disagree / somewhat disagree / somewhat agree / Totally agree.
Where the hell is the "meh" option there? There should always have a neutral, "I don't care" option. For that effect, the number of options should always be uneven.

Then there's those that use stars or a number-based scale with lots of possibilities, such as iMDB which uses 10 stars or something. Sure, that gives you a finer control over the rating you give item, but don't you end up wasting too much time deciding whether you'll give this movie 6 or 7 stars (especially considering you gave that other one 8 starts, but this one is definitely better)?

Too few options is not good either: with a 3-options like/neutral/dislike scale you are too constrained and can't express distinction between something that's only above the average, and something that's absolutely excellent. And a 2-options love/hate one is even worse in that aspect.

Of course, in the crowd-sourced rating of items point of view these work very well, since you just have to do some math with the positive and negative votes, and you'll get a fairly balanced rating of that item once you have enough votes. But is the voting system satisfying to the voters?

The most used system seems to be the 5 steps scale: hate / dislike / don't care / like / love. That's what YouTube uses, for instance, even though they chose their own labels. But recently I read somewhere that they should get rid of it altogether and use the favorite system as a binary rating system.

Personally, I believe the 5-star scale is good, but would work much better if something else than stars was used. A few days ago I was doodling during a conference, and came up with the following:
>(    :(    :|    :)    :D
This is very similar to what mashable uses. I believe the smileys might convey a much more intuitive grasp at what each rating means, thus reducing the bias that happens when people who don't like something give the lowest possible rating, and vice-versa. Besides, it is a very compact representation since one single symbol represents the rating (as opposed to 4 or 5 stars -- what is one star worth anyway? It's rather subjective...) and doesn't need a label to explain its meaning. That's why mashable manages to use them inline, actually.

I am not sure this is the best system, but I think it offers a very acceptable compromise between the satisfaction of the rating process (especially a good balance between flexibility and unambiguity) and the representation of a combined rating sourced from many votes.

1 comment:

Paulo Ricca said...

This is really cool because we have an emotional reflection. We are already used to relate emoticons to moods so it's easy to pick the one we feel more connected to!
Good Post :D