| |
BGonline.org Forums
Rollout Result/Gnu/Post on Forum
Posted By: Daniel Murphy In Response To: Rollout Result/Gnu/Post on Forum (Seth)
Date: Monday, 29 March 2010, at 6:43 p.m.
For my money this rollout has no value. If I were rolling it out, even just for my own personal studies, I would (a) use at least 2-ply checker, (b) not truncate, and (c) run more trials.
(a) Even in my own study, I use 0-ply only for the simplest of positions, while being aware that even for such positions 0-ply and higher play results can differ significantly.
(b) Although I occasionally use truncation for my own analysis, that is only when I really need a quick answer, when the play choice is not close, and when the position is likely to be resolved within the truncation depth. Published truncated rollouts I generally ignore. The fundamental problem with truncated rollouts is that the accuracy of the result depends on the accuracy of the evaluation at the end of the truncation period. Something else to be aware of is that the low Standard Deviations (and consequently the low Confidence Intervals) reported by a truncated rollout can be misleading. For example, Seth's rollout has SDs of 0.014 and a difference of 0.108. If this were a full rollout, those numbers would indicate a significant result -- that play B is unlikely to be better than Play A. But in Seth's rollout the SDs are small only because the rollout is truncated. If Seth wanted to lower the SDs even more without increasing the number of trials, he could decrease the truncation depth -- but would that rollout be more accurate?
Gnubg presents an additional problem with truncation because of odd-even oscillation. If you must truncate, best results seem to be attained when ply plus truncation depth is an even number; when ply plus depth is an odd number, results can be very wrong. But this isn't always so.
(c) I want to stress that there is no magic number of trials sufficient for all positions. The number of trials needed in order to attain a significant result depends on the size of the SDs of the plays relative to the difference in equity between the plays. If for instance someone recommmends 5,000 (or more trials), realize he has in mind positions for which so many (or more) trials are necessary, such as rollouts of opening plays, or when two plays are extremely close. Or if someone recommends 1296 trials, that person may be more interested than you in attaining small confidence intervals and thus more precise ranges of both absolute and relative equities. And realize that at the conclusion of 1296 trials, either (1) the result is significant and reliable or (2) more trials are needed or (3) the result while not yet significant may still indicate that the plays are close, which may be all the information desired. 1296 trials may be enough, 7776 (or more) trials may not be enough, and 648, 432 or even 108 trials may be all that are needed, depending on the variance, the closeness of the plays, and the student's interest in precision.
That said, I certainly have from time to time used low ply, truncated, and "too short" rollouts in my own backgammon studies. The parameters you choose to use will depend on your interest in the position, your computer's speed, your relative desire for quick results versus accurate results, your interest in attaining precise absolute and relative equities, and your tolerance for the greater chance of inaccuracy of rollouts that use less than ideal paramaters.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.