| |
BGonline.org Forums
What "cubeful" equities are and how to compute them
Posted By: Timothy Chow
Date: Friday, 8 January 2010, at 5:20 a.m.
There have been several recent threads about what cubeless/cubeful/live-cube equities mean and how the bots compute them. There was some back-and-forth discussion which got quite confusing at times. I thought it might be useful to collect the information into a single post.
The first thing to note is that one needs to distinguish between evaluations and rollouts. The reported equities in the two cases are calculated differently. I will concentrate on rollouts because that was what Ray Kershaw asked about initially, though I will say something about evaluations later. I will also assume that rollouts are full (and not truncated), meaning that all games are played to completion.
The simplest type of rollout to understand is what I will call a live-cube rollout. In this type of rollout, the bot does what you would commonsensically expect it to: it keeps track of the position of the cube as the game progresses, and makes cube decisions on every roll. Some games terminate with a (re)double and a pass, while others continue until one side has borne off all checkers. This is the kind of rollout that GNU does by default. (I think that the other bots also have this capability—although I think that in the case of Snowie you have to say specifically that you're interested in a live-cube rollout—but I'm not sure. The rest of my comments about live-cube rollouts are definitely true for GNU; I believe they're true of the other bots too but again I'm not sure.)
GNU reports wins/gammons/backgammons for each side. These mean what you think they mean: double/pass counts as a simple win, and (for example) the figure for gammon wins counts the total number of gammon wins regardless of what value the cube happened to be on when the gammon was achieved. Now, by adding the number of single wins to twice the number of gammon wins to thrice the number of backgammon wins, and subtracting the analogous figure for losses, we can obtain what GNU calls the cubeless equity of the rollout. In my opinion, this figure is misleadingly named (because it is derived from statistics from a live-cube rollout) and it is also useless, because what we really care about is the value of the game taking the cube into account. I will call this value the true equity. The cubeless equity is not an estimate of the true equity; in fact, it probably shouldn't even be called an "equity" at all, but the term is entrenched now.
To get an estimate of the true equity, we need to know not just the w/g/bg totals, but also how many gammons (for example) ended on a 2-cube, a 4-cube, etc. This information is not displayed by default, but Daniel Murphy pointed out to me that GNU has an option that lets you display this information. With this extra information, one can compute an estimate of the true equity, by taking the number of single wins with the cube on 1, plus twice the number of gammon wins with the cube on 1, plus three times the number of backgammon wins with the cube on 1, plus twice the number of single wins with the cube on 2, etc., minus all the corresponding figures for losses. This calculation will give you what GNU calls the "cubeful equity" (and what I believe Snowie calles the "live-cube equity") of the rollout, provided you have variance reduction turned off.
Variance reduction is a large topic in itself that I don't want to go into here. Suffice it to say that the idea is to get a better estimate of the true equity than the "obvious" calculation above yields, by estimating how lucky the rolls in each trial game were, and to adjust the equity estimate accordingly. This is a pretty tricky business and you can't expect to reproduce the calculations manually. Thus, if variance reduction is turned on, then even if you are armed with the w/g/bg figures and the information about what value the cube was on in each of the trials, you will still unfortunately be unable to manually reproduce GNU's "cubeful equity" number. Note, however, that whether or not variance reduction is turned on, the "cubeful equity" represents an attempt to estimate the true equity.
So much for live-cube rollouts. Are there any other kinds of rollouts? Yes. Snowie can do what I believe is called a cubeless rollout. In a (full) cubeless rollout, games are played to completion because there is no cube in play and hence no such thing as terminating a game with "double/pass." If you do a cubeless rollout with Snowie, then you can accumulate w/g/bg numbers easily enough, and from these you can compute the cubeless equity of the rollout as before. Since I pooh-poohed cubeless equity above, you might expect me to do the same here. Well, it's true that the cubeless equity is useless by itself. However, for starters, the terminology is at least not misleading, since we are in truth doing a cubeless rollout. Also, historically, there was a time when live-cube rollouts were not trustworthy because the bots could not be trusted to handle the cube correctly during the rollout, so trying to extract information from cubeless rollouts was the best option available.
But the fact remains that a cubeless equity is not what we care about. We want to estimate the true equity. How can we do that if all we have at our disposal is a cubeless rollout? Enter Janowski. Janowski came up with some clever formulas for estimating the true equity from a cubeless rollout. These formulas are rather subtle and again I will not go into them here. (In a recent post, NJ gave a sketch of the ideas behind Janowski's formulas.) Despite their cleverness, Janowski's formulas are inherently imperfect because a cubeless rollout just doesn't contain all the information you really need for a precise estimate. Nevertheless, Janowski's formulas work pretty well in practice. The result of applying Janowski's formulas is also usually called a "cubeful equity," because it is an attempt to estimate the true equity. It's important to remember, though, that a Janowski "cubeful equity" is an estimate derived from a cubeless rollout whereas GNU's "cubeful equity" is an estimate derived from a live-cube rollout. The calculations involved in producing them are very different, even though they're ultimately trying to estimate the same thing (namely, the true equity).
An alert reader might ask, if Janowski's formulas are pretty good, can't we apply them to the statistics accumulated from a live-cube rollout too, to get a "cubeful equity"? Well, you could, but I would argue that there really isn't any point. If you've done a live-cube rollout, then you have all the information that you need to estimate the true equity directly, as indicated above. Janowski tells you how to do the best you can with one hand tied behind your back (i.e., given only a cubeless rollout), but why would you voluntarily tie one hand behind your back when you have both hands available? I suppose it's possible that some bot does this, but it doesn't make logical sense to me.
All right, we have live-cube rollouts and cubeless rollouts. Is there anything else? Don't people talk about "cubeful rollouts"? It's true that people do use that term. I believe that they're using the term to mean what I've been calling a "live-cube rollout." In any case, I don't believe that there is any third type of rollout.
Finally, I promised to say something about evaluations. In an evaluation, you definitely don't have any direct information about how often the cube gets up to 2, 4, 8, etc. Instead, you have to work with the estimates that are hard-coded into the neural net. The neural nets in the current bots are trained on cubeless play, so something like Janowski's formulas have to be used to get "cubeful equities." Again I will not go into details, other than to point out some further terminological confusion. If you read the GNU documentation about how it does evaluations using Janowski's formula, you will see talk about concepts such as "cube efficiency" and "live cubes." The "live cubes" here are not to be confused with what I call "live-cube rollouts." Rather, the "live cube" refers to a certain idealized mathematical model of cube action.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.