| |
BGonline.org Forums
statsig and j.s.d. question
Posted By: Chuck Bower In Response To: statsig and j.s.d. question (Matt Cohn-Geier)
Date: Saturday, 1 November 2008, at 1:16 p.m.
You can extend the rollout result. Just click rollout again.
(I'm sure Matt knows this, but it needs to be clarified for others.) You need to change the number of rollouts you want run in the RO window. For example, if your first rollout was for 1296 trials. That window still shows this number. That is the total number of rollouts at the point it will stop. If you don't set this number higher, (you'll quickly find that) when you try to continue/extend the rollout it won't run. If you want ANOTHER 1296, set the value at 2592. When it starts up, after it's first newly completed trial, the onscreen counter will show 1297.
There is a potential problem with setting a rollout to end after a particular J.S.D. value (which has been discussed on GammonU in the past): there is a bias.
Since the rollout itself is a random process, the J.S.D. SEPARATION between the equity of the two candidates varies as well. Sometimes it's higher than it should be and sometimes it's lower. By setting a specific value of J.S.D.'s to stop at you are likely stopping at a lower value than you want.
As far as what the proper setting is, I've never seen a study (analytical or emperical) to tell us. The longer the rollout, the less the fluctuation on statistical significance.
I often use the second option "stop rollout of move when JSD of best move is better by..." and the values I use are 3.5 j.s.d. and 216 trials minimum. But if after a long time I check the RO and it shows no hope of getting there (after > or >> 1296 trials), I stop, keeping in mind that the true j.s.d. difference is not really as large as it says.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.