| |
BGonline.org Forums
Play A "loses more gammons"
Posted By: Timothy Chow
Date: Sunday, 17 January 2010, at 4:33 p.m.
Recently, Paul Weaver commented (regarding an opening-move rollout) that he was surprised that a certain play lost more gammons than the alternative.
For day-to-day practical work, taking the bots' gammon-win percentage at face value in order to make such judgments is fine. However, in the context of reference positions where we're rolling out a position tens of thousands of times, or perhaps even hundreds of thousands of times, and trying to make fine distinctions, I think we can do better.
The first thing I want to point out is that, at least for GNU and Snowie, the reported gammon wins are estimated cubeless gammons. As I recently learned from Daniel Murphy, GNU has the following surprising (to me anyway) behavior: Even when it does a live-cube rollout, it does not report the actual number of gammons won as the "gammons won" number. Instead, for a trial where the cube is dropped, GNU performs an evaluation to try to estimate the cubeless wins/gammons/etc. that would have resulted if the game had been played to a conclusion. (It's especially odd that GNU does this when, during the rollout trial, it plays checkers ATS instead of cubelessly, but whatever—this is what it does.)
Knowing the cubeless gammon wins has some value. It is certainly of legacy value, if you want to compare results with Snowie, for example. It is also of value if you want to compare the rollout results with a direct evaluation of the position, since bots currently rely on cubeless estimates as a stepping-stone in order to arrive at a cubeful evaluation. I would argue, however, that typically we're more interested in actual gammon wins as opposed to cubeless gammon wins. When calculating whether to take a cube ATS, one takes wins and adds/subtracts gammon wins/losses multiplied by the appropriate gammon price to compare with the raw take point; in this calculation, actual wins and actual gammons are what we should really stick in and not cubeless gammons. (Here I'm ignoring recube vig, but even if recube vig is taken into account I believe my point still holds. However, that would be a long discussion that I prefer to skip for now.) Even for money games, there is a distinction between cubeless gammons and actual gammons.
Do others agree with me that for the purposes of building up an opening book of heavily rolled-out reference positions, actual wins and gammons would be more valuable than estimated cubeless wins and gammons?
GNU already has the capability of reporting these actual numbers, via the "View Statistics" button. (Well, that's a little bit of a white lie, because the "View Statistics" numbers don't have any variance reduction applied. Some work would be needed to calculate the effect of variance reduction. I haven't thought this through so I'm not sure how involved it would be, but my guess is that it wouldn't be too bad.) If people agree with me that actual wins and gammons are of interest in building opening books, then that would argue again for more support for this feature of GNU (and of XG?) than currently exists.
By the way, I also noticed that the XG rollout displays that people have been posting don't seem to give confidence intervals for the wins/gammons/etc. The only confidence interval I see is for the overall equity estimate. It's important to remember that when making a claim that "Play A loses more gammons," one has to look at the confidence intervals for the gammons and not just for the overall equity. A rollout that is statistically significant as far as comparing equities is concerned may still fail to be statistically significant as far as comparing gammon wins/losses is concerned.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.