[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

TP 23.54? GV 0.588? (Long)

Posted By: Jim Stutz
Date: Sunday, 29 June 2014, at 11:36 p.m.

The bots aren’t that good, and neither are we.

(Apologies if this issue has already been discussed here.)

It has become common practice to represent MET values, take points, gammon prices, etc., out to four significant figures. A good example is the Kazaross match equity table, used by Extreme Gammon and derived from its own rollouts, which expresses match equities out to hundredths of a percent – 26.68%, 19.17%, and so forth. I’m not a supporter of this practice. My objections boil down to the following: 1) As good as XG is, it’s not anywhere near that good. 2) Even if we were sure these multi-digit figures were true values, most (if not all) of the extra digits after the decimal point still amount to noise -- meaningless at best, confusing at worst -- for even the top human players.

When XG pronounces, for example, that the trailer’s match equity at 2-away, 6-away is 28.88%, it is representing a reasonable certainty that this figure is accurate to within 1/100 of 1%. In other words, it’s assuring us that the true value is closer to 28.88% than it is to 28.87% or 28.89%. It is well settled that XG, as great a program as it is, has its weaknesses and foibles in certain positions -- containment, backgames, and others. And those are the weaknesses that are in plain sight and within the scope of human understanding – I’m sure XG also has other weaknesses that are and will always be unknowable. XG’s rollout results, no matter how long the trial and carefully fine-tuned the settings, ultimately reflect the quality of its own checker plays and cube decisions. Can we really assume that XG’s inevitable mistakes will cancel each other out and yield a match-equity value, take point, gammon price, etc., accurate to within 1/100 of 1%? I think not. In fact, I think we can fairly assume the opposite: Most of the values in the Kazaross MET are wrong according to the rules of significant figures – not necessarily wrong by much, but wrong just the same. It follows that the gammon prices, market windows, etc., derived from the table are also wrong. These four-figure values imply -- indeed they guarantee -- accuracy that simply does not exist.

And even if we were 100% certain that these ultra-fine MET values were accurate in the context of Perfect Play vs. Perfect Play, I don’t see how this ultra-accuracy would ever benefit a human player. Not many humans can crunch four-digit values OTB. And even those capable of such calculations would then have to combine these super-precise figures with seat-of-the-pants judgments regarding the qualities of a position and the relative strength of the opponents. How often do I win from here? How many gammons for the other side if things go badly? Am I a stronger player than my opponent, and if so, what are the respective ELO’s? Maybe a top expert could estimate these values within one or two percentage points, but, even so, the best expert estimates are orders of magnitude rougher than the accuracy and precision implied in a take point expressed as 22.87%. Combine a known precise value with a rough estimate and the best you’ll get is another rough estimate, so it is a waste of effort to labor over the precise-value part of the equation.

Accordingly, I would be in favor of the following convention: All MET values, as well as take points, gammon prices, and any other value derived from a MET table, should be expressed as either X% or X.5%, where X is an integer. This would better reflect both the limit of XG’s accuracy and the limit of human players’ practical ability to efficiently and effectively utilize such data over the board.

Messages In This Thread

 

Post Response

Your Name:
Your E-Mail Address:
Subject:
Message:

If necessary, enter your password below:

Password:

 

 

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums is maintained by Stick with WebBBS 5.12.