Some Statistical Analysis on the Compo Results

… or: A Guide on How To Score High Next Time.*

Recently, ashdnazg posted a handy CSV containing all Compo and Jam entries that received a rating. Being somewhat of a data-nerd, I decided to take a deeper look at the results. In particular, I was interested in one question:

“Do ratings in one category influence the ratings in another category, and if yes, how much?”

One way to answer this question is plotting the ratings in category A against the ratings in category B using a scatter plot. If A depends on B (or vice versa), we will roughly see a ‘line’. The line will become more ‘blurred’  the less dependent the categories are.

You can also put a number on the dependencen using MATHS. More specifically: Pearson’s correlation coefficient. The correlation coefficient measures the correlation (dependence) between two variables. As result, it will give you a value between -1 and 1, where -1 and 1 translate to “highly correlated” and 0 means “no correlation”.

Before jumping to conclusions, it’s time for a disclaimer: 

“Correlation does not imply causality” – i.e. high correlation of ‘Theme’ and ‘Mood’ can mean that a high rating in ‘Theme’ will lead to a high rating in ‘Mood’ or the reverse. Or it can mean there is no causality at all.

 

Anyway, here are the results of my analysis (correlation coefficient is shown in the lower right of the plots):

“Overall” Rating

As suspected, the overall rating is highly correlated with fun:compo-Overall-Fun

Surprisingly, it is less correlated with good graphics or innovation:
compo-Overall-Graphics
compo-Overall-Innovation

Here are the remaining plots. As you can see, the overall rating is almost independent on the humor rating, but surprisingly very correlated with the mood.

compo-Overall-Humor
compo-Overall-Mood Audio and Theme have a moderate correlation with the overall rating.
compo-Overall-Audiocompo-Overall-Theme
Conclusions:

There are two things we can take from this:

1. If you want a high overall rating, make you game (a) fun and (b) pretty. If you have time, try to evoke some intense feelings (mood).

But perhaps more important:

2. The Fun and Overall rating could be turned into one combined rating (“overall”).

 

Bonus:

This post is too short to show or even analyse all the correlations of each category with each other category. However, I created a zip containing all the graphs (including jam results) so you can draw your own conlusions. You can download the file here.

 

* Not really though. See the disclaimer.

Comments

20. Sep 2013 · 08:33 UTC
Nice work! These all make sense as far as I can see. Gameplay is the number one thing that should be valued in a game, good to see that others seem to feel that way. And as you mentioned, mood correlates to emotion, and nothing is better at evoking positive feedback from humans than emotion.
20. Sep 2013 · 08:37 UTC
That is a very nice set of graphics you made there! Maybe you could try making correlations between other pairs of variables?
20. Sep 2013 · 08:47 UTC
@caranha

Well yeah, a lot of times I found that I didn’t know how to rate mood because the game didn’t give me any feelings whatsoever (which is perfectly fine). In those cases I just left the rating out (I did the same for humour on almost all games).
20. Sep 2013 · 09:01 UTC
It’s nice to see statistics are consistent!
20. Sep 2013 · 11:40 UTC
I think it’s a problem that people are not taking production values enough into consideration when voting for the Overall category. It makes me sad when I see games that obviously took a lot of work and skills to make beaten by simplistic games.