| |
BGonline.org Forums
"Easy" improvement to PR calculations
Posted By: Henrik Bukkjaer In Response To: "Easy" improvement to PR calculations (Timothy Chow)
Date: Monday, 15 April 2013, at 10:08 p.m.
>> And once you grant that, then I don't see the point of tinkering with the details of the definition of ER/PR. In the medium run all these details tend to average out anyway.
The point, of course, is to shorten the run needed for this to average out. Otherwise we could just use win rate and leave out any error assessments completely - given enough matches, the better player will win more!
I know "reducing variance in PR" was not the best choice of words for what I was trying to say. Your view on the now and then 20+ game as an outlier is correct. You'll have more of them looking at PR instead of ER. But they will be there for ER also, given that there are games which is brought to an end by an early drop.
Also, if you look at a novice player, the very low ER games that can be produced by Snowie when you have a very high number of forced moves, could be considered outliers! :-)
What I meant to say was, that the numbers produced by the revised PR measurement, would be closer to the actual "perceived performance" by that player in those games/matches.
Looking at very small samples (such as single games with few decisions), we should expect variance! If it is not there, I'd be skeptical of the method used.
----------------
Now, you keep saying that we simply shouldn't use ER/PR for all sorts of things. Or at least we shouldn't look for ONE good "fits all" measurement. I agree (to some extend). But error rates ARE the preferred metric of choice, when talking about player performance in individual matches or even tournaments. So why not improve the measurement for that purpose if possible?
----------------
In another thread (the one about error-rates and complexity in chess), you mentioned something about analyzing a heap of backgammon matches, in order to compensate the error-rate calculation (or something like that) for complexity. I assume categorizing positions into position types, and then analyzing which types typically were difficult to play for humans, etc.
I wonder how you would use that information (if gathered/derived) without conflicting with the views you put forward in this thread? I'm curious to know exactly how (and for what) you'd use the output from that position/error analysis? could you elaborate a bit on that?
In fact, I think your analysis of errors/position types would be very close to what I considered for this idea. I split into "early opening book", non-contact, and bear-off (all of which I would consider "non-complex" to various extend), and for all the other position types, I just used the roll combined with any forced or semi forced moves as the basis for the move complexity evaluation (to more options you had, the more difficult the move, roughly).
Cheers,
Henrik
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.