BGonline.org Forums

Two things that need to be done with XBG

Posted By: MaX
Date: Tuesday, 30 June 2009, at 8:25 a.m.

In Response To: Two things that need to be done with XBG (Rich Munitz)

I would tend to say that the ultimate procedure to decide who's better is this:

- have bot A and bot B play head to head N matches and/or money seessions, with settings that generates comparable response times for the 2 (and keep track of the total time used by the two boths).

- check the overall outcome (who has won more).

- let each bot analyze (at the same level used for play) the other bot decisions and find plays where the two bots (at play level) disagree.

- for each play where the bots disagree, do a rollout (to statsig) with each bot and see if the two rollouts agree or not: * if they agree, one of the bots got it right (most likely) * if they don't agree, we won't know, unless the SM tells us the truth :)

Notice that it's not absolutely mandatory to have the two bots play head to head: we could just take any set of matches/sessions (even played by humans) and let the bots analyze them. Having the bots play head 2 head only provides the high-impact face to face result (like bot A won 32.5$ over 1000 games to bot B).

I'm not really a fan of a 0ply money cubeless comparison: it is surely fast and potentially interesting (assuming that the result is a sizeable difference between the 2 bots, which may not be the case). As far as I can see, match play is much more tricky and theory of cube handling is not at all that straightforward. The MET could be an issue, but gnubg can use custom METs: if XG has his own one (and they are willing to share) or if XG can use custom ones, there's no problem.

MaX.

Messages In This Thread

• Two things that need to be done with XBG
  neilkaz -- Monday, 29 June 2009, at 6:24 p.m.
  • Two things that need to be done with XBG
    Mislav Radica -- Monday, 29 June 2009, at 6:45 p.m.
    • Two things that need to be done with XBG
      Stick -- Monday, 29 June 2009, at 7:24 p.m.
      • Two things that need to be done with XBG
        Rich Munitz -- Monday, 29 June 2009, at 9:17 p.m.
        • Two things that need to be done with XBG
          Frank Berger -- Monday, 29 June 2009, at 9:36 p.m.
          • Two things that need to be done with XBG
            Ian Shaw -- Tuesday, 30 June 2009, at 12:30 a.m.
        • Two things that need to be done with XBG
          Bob Koca -- Monday, 29 June 2009, at 11:14 p.m.
          • Two things that need to be done with XBG
            Mislav Radica -- Tuesday, 30 June 2009, at 12:06 a.m.
          • Two things that need to be done with XBG
            Rich Munitz -- Tuesday, 30 June 2009, at 4:26 a.m.
            • Two things that need to be done with XBG
              Stick -- Tuesday, 30 June 2009, at 5:11 a.m.
              • Two things that need to be done with XBG
                Ian Dunstan -- Tuesday, 30 June 2009, at 6:17 a.m.
                • Two things that need to be done with XBG
                  Mislav Radica -- Tuesday, 30 June 2009, at 9:19 a.m.
                • Two things that need to be done with XBG
                  phil Simborg -- Tuesday, 30 June 2009, at 12:52 p.m.
                  • Two things that need to be done with XBG
                    aoleg1 -- Tuesday, 30 June 2009, at 1:41 p.m.
                    • Two things that need to be done with XBG
                      Bob Koca -- Tuesday, 30 June 2009, at 5:04 p.m.
                      • Two things that need to be done with XBG
                        Ian Dunstan -- Tuesday, 30 June 2009, at 9:24 p.m.
                        • Two things that need to be done with XBG
                          Matt Cohn-Geier -- Tuesday, 30 June 2009, at 9:34 p.m.
                          • Backgames
                            Ian Dunstan -- Wednesday, 1 July 2009, at 1:37 a.m.
                        • Two things that need to be done with XBG
                          Frank Berger -- Tuesday, 30 June 2009, at 9:38 p.m.
            • Two things that need to be done with XBG
              phil Simborg -- Tuesday, 30 June 2009, at 12:25 p.m.
              • Two things that need to be done with XBG
                Marty Storer -- Tuesday, 30 June 2009, at 2:23 p.m.
              • Two things that need to be done with XBG
                Rich Munitz -- Tuesday, 30 June 2009, at 3:47 p.m.
        • Two things that need to be done with XBG
          MaX -- Tuesday, 30 June 2009, at 8:25 a.m.
          • Two things that need to be done with XBG
            Bob Koca -- Tuesday, 30 June 2009, at 9:41 a.m.
          • Two things that need to be done with XBG
            playBunny -- Wednesday, 1 July 2009, at 5:08 p.m.
      • Two things that need to be done with XBG
        Frank Berger -- Monday, 29 June 2009, at 9:47 p.m.
      • Two things that need to be done with XBG
        Mislav Radica -- Monday, 29 June 2009, at 10:14 p.m.
  • Two things that need to be done with XBG
    aoleg1 -- Tuesday, 30 June 2009, at 12:59 p.m.
    • Two things that need to be done with XBG
      neilkaz -- Tuesday, 30 June 2009, at 5:58 p.m.
    • Two things that need to be done with XBG
      Stick -- Tuesday, 30 June 2009, at 7:57 p.m.
  • Two things that need to be done with XBG
    pontiac -- Tuesday, 30 June 2009, at 9:04 p.m.
    • Two things that need to be done with XBG
      Matt Cohn-Geier -- Tuesday, 30 June 2009, at 9:28 p.m.
      • Two things that need to be done with XBG
        Mislav Radica -- Tuesday, 30 June 2009, at 10:47 p.m.
        • Two things that need to be done with XBG
          Matt Cohn-Geier -- Tuesday, 30 June 2009, at 11:11 p.m.