| |
BGonline.org Forums
Method of scoring a BG quiz
Posted By: Jason Lee
Date: Thursday, 27 September 2012, at 12:52 a.m.
Let's say you're going to create a BG quiz. We'll use a bot as gospel. Assume that answering any particular question is optional -- this is important, I'm not willing to let it go. How should it be scored?
I'll take a stab... everybody gets assigned a score on each problem... we compute the average EMG error on the problem among those people who answered. Then your score for the problem is: Average Error on the Problem - Your Error on the problem
If you don't answer, you get a zero. If everybody answers the same thing, everybody gets a zero.
More concretely, say we have a problem for which 20 people answer. 15 people choose the "right" answer (remember, I'm just taking the bot as gospel). 3 people choose a play that is an 0.02 error, and 2 people choose a play that is an 0.05 error.
The average error is (3*0.02 + 2*0.05)/20 = 0.16/20 = 0.008. Then the scores gets assigned:
15 people get 0.008 - 0 = 0.008
3 people get 0.008 - 0.020 = -0.012
2 people get 0.008 - 0.050 = -0.042
Then if we have a 20-question quiz, each player can sum their scores over the 20 questions to get their total score for the quiz. Notice that the sum of everybody's score on each problem is zero, so you can kind of measure your progress against the field by taking a total score of zero to be "average."
One obvious problem is that if you choose one real stinker in your guesses, you'll ruin your score. On one hand, that sucks. On the other hand, backgammon PR is graded like this. You can play a great match but make one really horrendous play and your PR goes down the tubes.
I now ask you all to shoot some holes in my concept. I want to know where it goes wrong.
JLee
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.