BGonline.org Forums

xG 3-ply ROLLOUT Hmmmmm - 32 backgame

Posted By: Robert-Jan Veldhuizen (Zorba)
Date: Monday, 13 July 2009, at 6:04 p.m.

In Response To: xG 3-ply ROLLOUT Hmmmmm - 32 backgame (eXtreme Gammon)

Thanks for the reply, Xavier! That's good to hear, I guess it was a bug in variance reduction? Maybe similar to what Frank Berger experienced with BGB, a few months ago.

Anyhow, it's very good to see that my result was due to a bug in rollouts, and not due to xG's neural nets.

About variance reduction in rollouts: I think that, allowing for the statistical error margins (as expressed f.i. by the 95% confidence intervals), rollouts with or without VR should give the exact same results. VR can't introduce a (structural) bias in a rollout. At least, that's my strong impression from what I've read about the subject (by David Montgomery). The idea is that in the end (infinite trials), "luck" by definition goes to zero as you do more trials, so the VR (luck) adjustment can't be biased.

About the size of the standard error (CI) in relation to the number of trials, as a measure for the reliability of the bot: I lack the statisitical knowledge for the theoretical side of this idea (maybe Maik Stiebler or Douglas Zare can say something about this). In practice though, I've seen many examples of SE/sqrt(#trials) getting lower, sometimes much lower, in positions we know the bot plays very well. Races with GnuBG are an obvious example. The opposite happens in exotic backgamish positions where GnuBG plays very poorly. So, at least in some way, it seems to be a measure, perhaps more of consistency than accuracy, and more about relative accuracy (comparing move equities) than absolute accuracy (determining a take decision).

Messages In This Thread

• Test your favorite bot's evaluation skills on this 32 backgame
  neilkaz -- Tuesday, 30 June 2009, at 3:37 a.m.
  • Here's the position
    neilkaz -- Tuesday, 30 June 2009, at 3:38 a.m.
    • Here's the position
      Rich Munitz -- Tuesday, 30 June 2009, at 2:47 p.m.
    • Here's some evals
      neilkaz -- Tuesday, 30 June 2009, at 2:47 p.m.
      • Here's some evals
        Rich Munitz -- Tuesday, 30 June 2009, at 3:32 p.m.
        • Here's some evals
          neilkaz -- Tuesday, 30 June 2009, at 3:41 p.m.
    • Here's the position
      jdg -- Tuesday, 30 June 2009, at 4:09 p.m.
    • Here's the position
      Chuck Bower -- Tuesday, 30 June 2009, at 4:16 p.m.
      • Here's the position
        Marty Storer -- Wednesday, 1 July 2009, at 3:01 p.m.
        • Here's the position
          Chuck Bower -- Wednesday, 1 July 2009, at 3:56 p.m.
    • Here's the position
      Frank Berger -- Tuesday, 30 June 2009, at 7:21 p.m.
      • About the only thing wrong with BGB is VERY pessimistic backgame evaluations.
        neilkaz -- Wednesday, 1 July 2009, at 2:47 a.m.
        • About the only thing wrong with BGB is VERY pessimistic backgame evaluations.
          Ian Dunstan -- Wednesday, 1 July 2009, at 5:51 a.m.
          • BOT development and backgame evals
            neilkaz -- Wednesday, 1 July 2009, at 4:44 p.m.
            • BOT development and backgame evals
              Frank Berger -- Wednesday, 1 July 2009, at 9:03 p.m.
            • BOT development and backgame evals
              eXtreme Gammon -- Thursday, 2 July 2009, at 2:06 a.m.
    • a question of 3-aw 3-aw maniac
      aoleg1 -- Wednesday, 1 July 2009, at 1:05 p.m.
      • Zadeh MET=garbage *NM*
        neilkaz -- Wednesday, 1 July 2009, at 4:35 p.m.
        • Zadeh MET=garbage
          MaX -- Thursday, 2 July 2009, at 7:28 a.m.
  • xG 3-ply ROLLOUT Hmmmmm - 32 backgame
    Robert-Jan Veldhuizen (Zorba) -- Monday, 6 July 2009, at 4:13 p.m.
    • xG 3-ply ROLLOUT Hmmmmm - 32 backgame
      eXtreme Gammon -- Friday, 10 July 2009, at 3:00 a.m.
      • xG 3-ply ROLLOUT Hmmmmm - 32 backgame
        Robert-Jan Veldhuizen (Zorba) -- Monday, 13 July 2009, at 6:04 p.m.