Over the years of watching game ratings at BoardGameGeek, you start to see some trends. To put it plainly, these trends reveal not only the flaws in the rating system for BoardGameGeek users, but also illustrate flaws in Internet ratings of entertainment products in general. Some of the sources of these flaws are hard to determine, and the biases I discuss in this article are based on reflections by using scholarly articles.
Game ratings on BoardGameGeek and Amazon skew higher due to the fact that mainly people who own the game are rating the game. Usually, people who purchase a game are the ones who feel they have enough experience with the game (some rate even if they don’t) to give a rating. The reason this occurs is because of the need to psychologically affirm our choices. Game expansions and variants also gather higher ratings because the people predisposed to extend their product purchase are the ones who enjoy the game the most. This one aspect, I suspect, has the largest effect on game ratings.
Another bias comes from genre self-selection. This occurs because people usually only play games that they are predisposed to like. If you are a hardcore Euro gamer and are presented a heavy theme social deduction game, odds are you’re not going to like it. Therefore, the most popular genres get the most ratings and invariably smaller genre-specific games will draw a much smaller sample. This effect is also present in movie ratings.
The final aspect of bias comes into play due to the rating scale used. BoardGameGeek uses a 1-10 point scale that also attaches a qualitative measure to each point. Much work has been done by BoardGameGeek to make sure the qualifiers accurately represent a user’s feelings, but often users do not use the qualifiers and judge merely by number, especially in the game rating page where the qualifiers are not visible.
As a previous computer game critic, it is right and necessary that honest, thoughtful reviewers take the time to review games. Their thoughts give the uninformed the truth as opposed to the marketing and hype that builds a game up. For many publishers, they really just want you to buy the product, that’s all that matters. For designers, they hope you enjoy the game, but they also might want to know whether one of their designs is better than another. Using valid reviews and opinions gives consumers and producers much more information than anonymous rating scales alone.
At the end of the day, amongst all the ratings, comments, and rankings (Top 10’s are pretty popular), be sure to keep everything in context and use other sources besides BoardGameGeek or Amazon ratings for accurate information on quality and entertainment value.