| |
BGonline.org Forums
Error Rates
Posted By: Chuck Bower In Response To: Error Rates (Stick)
Date: Tuesday, 28 April 2009, at 4:24 p.m.
For argument's sake let's imagine you have some sick puppy who cares about error rates for an inexplicable reason, he has the time and computer power to roll out all his matches and decides to roll out every play that's close to statistical significance when possible, or at least killing off most of the variance and getting it to where the plays may be 'too close to call', good enough?
No! While rolling out plays once you get a statistically significant result and the play made OtB is clearly correct you can stop the rollout at that time, it doesn't matter for the error rate how far back decision B is. However, if you're rolling out your blunders (of which even the best of us have numerous in any given match) and you roll it out a minimum number of games, let's say 108, and after that 108 games it is statistically significant are you finished? Nope, not if you want to be sickly accurate. You need to overroll the plays you got wrong to know how large your error/blunder is. After only 108 trials it may be a .15 blunder and not the correct play by 3.5 JSDs so you'd think on to the next, but that .15 blunder could be a .22 blunder or a .08 blunder with all the variance that's left in that type of rollout.
I'm that sick puppy. Been here and done all that, exactly as you say. I roll out positions that the human is right until I get statsig (usually > 3 but if very long RO, > 2) and when the human is wrong I roll it out until I get the SE's at least below 0.005 and preferably the j.s.d.'s below 0.005.
But I agree we need a standard that doesn't require so much computer time. And I agree it must be a STANDARD. I.e. everyone conforms to that method.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.