| |
BGonline.org Forums
Got Variance?
Posted By: Nack Ballard In Response To: Got Variance? (Daniel Murphy)
Date: Monday, 24 May 2010, at 5:52 a.m.
I don't know, maybe for every pair of rollouts that gets scrutinized for which the odds of such a change are 1 in 120, there are 119 other pairs of rollouts for which the change wouldn't be so "wild" (that's "W," right?) but nobody's looked.
Well, that's the point. My observations, in looking at pairs of results of identical positions rolled out (often by accident, or double-assigned) or in watching extensions, that early game results are more wild than they should be according to the way Snowie reports the distribution of results are supposed to be, in my perception. I admit that I haven't logged anything; if it were important enough to me (relative to other projects) to have those perceptions confirmed or denied, I would have taken the time to do so.
Now that Snowie and Gnubg have been reporting their 95% confidence intervals for several years, perhaps those CI's should be put to the test. Hopefully with a reassuring outcome. Because if you can't trust a bot when it says CI ± 0.100, what about ± 0.040, ± 0.020, ± 0.010, CI ± 0.005, CI ± 0.002 .... Think of the literally thousands of positions that have been discussed for which one and only one rollout was ever done. That's a lot of (potential) misplaced confidence.
Yup. I can only speak for (my perception of) early game rollout results, though. I have not observed what happens for positions after the first few rolls. That said, if it is definitely confirmed that CI ranges are understated for whatever class(es) of positions, one can still look at old rollouts and make the necessary mental adjustments, including extensions when the trials are intolerably few; that's basically what I do already, acting on my perceptions (or hunch, if you want to call it that). At least the data isn't wasted.
Nack
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.