| |
BGonline.org Forums
Scale of Open/Intermediaate Players
Posted By: Jason Lee In Response To: Scale of Open/Intermediaate Players (Stick)
Date: Tuesday, 28 April 2009, at 11:14 p.m.
D. Zare: Another problem with gnu's method is that your error rate can be ridiculously high if you encounter few unforced decisions in a match. This happens frequently with the cube decisions, which makes gnu's cube error rate unreliable. A high error rate often indicates that someone had few decisions rather than that the player lost a lot of equity. In fact, a low cube error rate often means the player had no real decisions, but gnu thought the player had many unforced cube decisions.
Alright, technically, Doug is right.
Here's my counter argument: please name for me the last time you heard somebody report to you their ER on cubes in a particular match. That's right, you've never heard it.
I keep tabs of my ER in aggregate -- so I'm not measuring averages of averages. I keep track of checker play errors, # of checker plays, cube errors, # of cube errors. Then in sum, I can determine my overall error rate appropriately. Thus, it is irrelevant if I had a fat 0.086 cube error in just one decision in the match -- that 86 millipoint per cube decision is swallowed up in the sea of cube decisions.
If forced moves were to happen at random, and were not affected by previous plays, then considering only unforced moves would be a clear improvement. However, upon closer inspection, the distinction we would like to draw is not between forced and unforced moves. It is between trivial moves and nontrivial moves. You can't make an error while dancing, but you also have little real opportunity to make an error when you are moving your checkers around the board after getting closed out, when you have negligible chances to win or to get gammoned. It is also not a real opportunity to err if you roll an opening 3-1.
Bob Koca alluded to this by suggesting a quick fix by removing from the denominator all plays where the equity was the same no matter what legal move you made. This goes only a tiny way towards fixing this problem, but it is a sensible first step.
However, this problem is smoothed out in the aggregate as well. I've played over 30 practice matches against a colleague over the last year, and recorded them all. We put them into GNU and check our error rates. Because I'm a stronger player, I've gotten the lower ER every single time. In one memorable match, we both got absurdly low (for us) ER, but mine was still better. He lamented that the one time he got a great error rate that I still outplayed him. I pointed out that part of the reason was that the entire match was played with fairly simple positions -- for example, every single game happened to be played to conclusion, so we "benefited" from making lots of racing checker plays where it's hard to err. Our ER "lucked out" from easy to play positions.
So the argument against GNU style error rates is that they can be misleading -- error rates for a single match are misleading anyway, for the reason I stated above and because there's a sample size problem!
GNU's ER formula is not perfect, but it specifically eliminates some of the peculiarities of Snowie's ER.
JLee
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.