BGonline.org Forums

Milking more out of the Depreli positions

Posted By: Timothy Chow
Date: Friday, 26 February 2010, at 4:19 a.m.

It is often debated on this forum whether an equity difference of 0.010, or 0.005, or 0.001, means anything, especially between two close opening plays.

One argument that a 0.005 difference can't be trusted runs like this. Suppose Play A leads to a backgame 3% of the time while Play B leads to a backgame 8% of the time. Suppose that a backgame lasts 33 moves on average and the bot's error rate for the defensive side of a backgame is about 3 millipoints per move higher than for the offensive side. Then all this will add up to a 0.005 systematic error in the bot's comparison of Plays A and B. Since these figures don't seem unreasonable given what we know of bot weaknesses, we should not trust equity differences of only 0.005.

Of course, I pulled those numbers out of a hat. Perhaps, though, the positions from Michael Depreli's Bot Comparison could be used to make the above argument precise. There is plenty of off-the-shelf machine-learning software nowadays; for example, SVM Light is one of my favorites. One could take all the Depreli positions where a particular bot makes errors greater than a certain size, along with an equal number of positions that the bot does just fine on, and dump them into SVM Light to see if it can learn to distinguish between the two types of positions. One might hope that it's a lot easier to do a reasonable job of recognizing the positions where the bot makes big errors than it is to correct those errors (i.e., to make the bot play better, which is clearly a difficult task).

Assuming that SVM Light can output a decent classifier of this type, one could then program the bot to run the classifier on all the positions that arise during a rollout. This would then allow us to produce estimates of the bot's systematic errors. The rollout would take longer, of course, but in practice SVM Light classifiers run very fast, probably significantly faster than a 3-ply evaluation.

Anyway, this is of course speculative, and requires quite a bit of work to implement, but I thought I would put the idea out there in case it strikes someone's fancy.

Messages In This Thread

• Milking more out of the Depreli positions
  Timothy Chow -- Friday, 26 February 2010, at 4:19 a.m.
  • Milking more out of the Depreli positions
    lenny -- Saturday, 27 February 2010, at 2:30 a.m.
  • Milking more out of the Depreli positions
    Matt Cohn-Geier -- Saturday, 27 February 2010, at 8:55 a.m.
    • Speaking of milking .....
      neilkaz -- Saturday, 27 February 2010, at 3:20 p.m.
  • Milking more out of the Depreli positions
    Gregg Cattanach -- Monday, 1 March 2010, at 7:25 p.m.
    • Milking more out of the Depreli positions
      Stick -- Monday, 1 March 2010, at 7:40 p.m.