{"assets":[],"author_link":"author\/crowbeak\/","author_name":"Crowbeak","cat":"LD #32","categories":["LD #32"],"comments":[],"epoch":1431212700,"event":"LD32","likes":5,"metadata":{"p_key":"83752","p_author":"Crowbeak","p_authorkey":"0","p_urlkey":"294491","p_title":"Looking at my rating habits with the LD Personal Voting Analyzer","p_cat":"LD #32","p_event":"LD32","p_time":"1431212700","p_likes":"5","p_comments":"0","p_status":"WAYBACK","us_key":null,"us_name":null,"us_username":null,"event_start":"1429228800","event_key":"27","event_name":"LD32"},"source_url":"2015\/05\/09\/looking-at-my-rating-habits-with-the-ld-personal-voting-analyzer\/","text":"

\"lda1\"<\/p>\n

Liam \ud83d\ude00<\/a> made this awesome Ludum Dare Personal Voting Analyzer<\/a>. You go to the play and rate page while logged in, copy everything in the table showing your voting scores for all the things you rated, and then paste it into a box on the voting analyzer page. The tool then takes the data and pops out a bunch of charts and graphs about your votes.<\/p>\n

I am super happy that this tool was made. For several LDs, now, I\u2019ve been wondering how good my internal milestones for each rating are, and although my rating is balanced in some areas, it turns out that it\u2019s pretty heavily skewed in others. I\u2019ve had the feeling my judging was skewed for a while now, and this tool has given me the data I need to know how badly skewed it is and in which areas so that I can adjust how I vote for next time. The reason I care is that I feel like 3 stars should represent the average and that if I\u2019m scoring right, I should have a nice bell curve across all my votes.<\/p>\n

The pie chart above shows how many of the things I rated were Jam, Compo, or Other games, where Other are games on which I commented without rating anything because the game was unplayable. As you can see, about half the games I rated were Jam and half were Compo. This is the only pie chart the tool shows, and aside from telling me that I have good sample sizes for both Compo and Jam games, I find it less useful than the line graphs.<\/p>\n

<\/span><\/p>\n

\"lda2\"<\/p>\n

I\u2019m pretty happy with my distribution of Overall ratings. It\u2019s not exactly a bell curve on the right hand side, but it definitely looks like a nice mountain. I give a game its Overall rating last, taking into account my ratings in all the other categories. The slight skew to the right seen here is influenced by the heavy right-hand skews in a couple of other categories.<\/p>\n

\"lda3\"<\/p>\n

Innovation is one of the categories where my skew is the worst. When rating, it\u2019s very hard to get a 1 out of me on Innovation; the game basically has to be a straight-up clone of something. Since I want 3 stars to be average, I then set 3 stars as standard game genre. To get a 5-star rating, a game has to be unlike anything else I\u2019ve seen before. 2 and 4 stars have thus been based on how different it feels from standard. I think that to rectify this, I\u2019ll have to set 2 stars as standard genre. This will give me more room for variations on doing better than that.<\/p>\n

One thing I noticed while doing my voting during this LD is that my Innovation and Theme ratings tend to rise and fall together, though if you look at the Theme graph below, you\u2019ll see that it has much sharper ups and downs.<\/p>\n

\"lda4\"<\/p>\n

This one is slightly skewed to the left for Compo, but a very pretty curve for Jam. I\u2019m okay with this one. My 3 star rating for Fun means that that was pretty fun and I\u2019m glad I played it this one time for rating it. 1 star indicates that I didn\u2019t really enjoy the game at all and 2 stars is somewhere in between\u2026 it wasn\u2019t terrible, but I could have done without playing it. 4 stars is this was really<\/em> fun to play this one time, and 5 stars indicates that I am seriously considering keeping this game to play more later.<\/p>\n

Given that rating system, it makes sense that Compo games would have a different skew. It\u2019s harder to make a game by yourself with no outside assets in two days than it is to make a game on a team with outside assets allowed in three days. One could argue that I should go easier on Compo games, and in some of the other categories I do, but not this one. These two curves are visually dissimilar, but the number of games that got 3 stars or higher out of me in both divisions is about the same.<\/p>\n

\"lda5\"Holy strange skew, Batman.<\/p>\n

Every time a theme is announced, there\u2019s generally an obvious and easy to implement way of interpreting the theme. For An Unconventional Weapon, for example, you can just take any regular whacky\/slicy or shooty game genre and just replace the weapon with something strange. I always set that obvious implementation as my 3-star rating, give anything that doesn\u2019t seem to fit the theme at all 1 star, and anything that blows my mind 5 stars. The 2- and 4-star ratings, then, are just\u2026 tweens.<\/p>\n

I think that to rectify this, I\u2019ll need to set the obvious implementation as 2 stars. This will give me more room (as with Innovation) for variations on anything that does better than that. So to use An Unconventional Weapon as an example again: a straight up regular platformer where the weapon difference is strictly a difference in visuals (a pickle instead of a sword, but it still acts like a sword) and maybe sound would get 2 stars from me; a game that is mostly a regular member of its genre but the unconventional weapon\u2019s behavior is slightly different would get a 3; and then games that went out of their way to be offbeat would get 4 or 5 depending on how far out they were.<\/p>\n

I have yet to analyze prevous LDs\u2019 results, but I find myself wondering if this Innovation graph will be considerably different from those of previous LDs. About three games into rating, I came across a game which used a weapon which appeared to be normal but acted entirely un-normal, and as an idea my team and I hadn\u2019t considered and the perfect fit for the word \u201cunconventional\u201d, it sort of set the bar for what 5 stars in Theme was gonna mean to me this time around.<\/p>\n

\"lda6\"<\/p>\n

My graphics ratings are also skewed to the right. I\u2019ve always set 3 as being good enough to convey all details necessary for gameplay, but as with innovation and theme, I think I\u2019ll need to move that down to 2 stars to give myself more room for variety in doing better. Things that I rated as 2 stars in Graphics this time would then fall to 1 star.<\/p>\n

\"lda7\"<\/p>\n

I feel pretty much the same way about my voting for Audio as I do about voting for Graphics. The only 1 star rating I\u2019ve given out here was for a game whose audio simply grated on my nerves, but I think having 1 star be insufficient audio (or nerve grating) and setting the barely acceptable bar at 2 stars will help here. I suspect my Jam curve will still have a dip at 2 in the future, but many Jam games have a dedicated audio person, so that makes sense.<\/p>\n

\"lda8\"<\/p>\n

I\u2019m not sure what to do about this cowboy hat. I voted on humor for far fewer games than I rated in total. Humor is dependent on so many things that it\u2019s hard to quantify, and it doesn\u2019t help that a lot of things most people find funny just aren\u2019t amusing to me. On the other hand, just doing 1 star or 5 depending on whether I found it funny or not wouldn\u2019t work, either. So\u2026 yeah. Cowboy hat.<\/p>\n

\"lda9\"<\/p>\n

This one is skewed to the right, but not horribly so. Nonetheless, I think setting \u201call right\u201d at 2 stars and taking advantage of more options to the right of that would benefit here, as well.<\/p>\n

Conclusion<\/h2>\n

There are other graphs at the bottom of the tool, showing how strong or weak the correlations are between pairs of individual categories, but as long as my overall category graphs are so skewed, I don\u2019t think they\u2019re very useful to me. As such, I\u2019m not going to cover them here.<\/p>\n

It\u2019s really nice to have these graphs, though, because they\u2019ve helped me visualize how my mental images of what each star rating turns out to be translates into how I vote. I am not going to go back and rerate the 115+ games I\u2019ve voted on this time, but I will be looking back at these graphs for next time.<\/p>","time":"May 9th, 2015 11:05 pm","title":"Looking at my rating habits with the LD Personal Voting Analyzer","title_was_empty":false}