BGonline.org Forums

Two things that need to be done with XBG

Posted By: Rich Munitz
Date: Monday, 29 June 2009, at 9:17 p.m.

In Response To: Two things that need to be done with XBG (Stick)

The goal is to determine how the two neural nets compare in strength. XBG's ability to produce GNU's results is not by itself a valid test, since GNU could be wrong.

Neil is right that a head-to-head long duration dueller type match-up (if possible) is the best test of relative playing strength. Ultimately what would be most revealing and most efficient would be a match up using the basic neural net without look-ahead (0-ply for GNU). This is not only the fastest to run, but ultimately the goal is to assess the NN strength and not the lookahead which may not always be apples to apples. And probably money games are the best test, and cubeless money games even better. IMO it is best to focus on how the NNs compare to each other and eliminate the other variables like METs, cube adjustments, etc.

Similarly, you can't just take two rollouts of "normal positions" and judge one better than the other (though it can help to instill confidence that it is trustworthy). What is more useful is to find cases where the bot's evaluations disagree and see which bot's evaluations the rollouts tend to uphold. And of course, the answer to that could be dependent upon position type. There is not necessarily a stronger bot in all situations and it is useful to know strengths and weaknesses. Another test is how close does each bot's eval tend to come to the rollout results.

Messages In This Thread

• Two things that need to be done with XBG
  neilkaz -- Monday, 29 June 2009, at 6:24 p.m.
  • Two things that need to be done with XBG
    Mislav Radica -- Monday, 29 June 2009, at 6:45 p.m.
    • Two things that need to be done with XBG
      Stick -- Monday, 29 June 2009, at 7:24 p.m.
      • Two things that need to be done with XBG
        Rich Munitz -- Monday, 29 June 2009, at 9:17 p.m.
        • Two things that need to be done with XBG
          Frank Berger -- Monday, 29 June 2009, at 9:36 p.m.
          • Two things that need to be done with XBG
            Ian Shaw -- Tuesday, 30 June 2009, at 12:30 a.m.
        • Two things that need to be done with XBG
          Bob Koca -- Monday, 29 June 2009, at 11:14 p.m.
          • Two things that need to be done with XBG
            Mislav Radica -- Tuesday, 30 June 2009, at 12:06 a.m.
          • Two things that need to be done with XBG
            Rich Munitz -- Tuesday, 30 June 2009, at 4:26 a.m.
            • Two things that need to be done with XBG
              Stick -- Tuesday, 30 June 2009, at 5:11 a.m.
              • Two things that need to be done with XBG
                Ian Dunstan -- Tuesday, 30 June 2009, at 6:17 a.m.
                • Two things that need to be done with XBG
                  Mislav Radica -- Tuesday, 30 June 2009, at 9:19 a.m.
                • Two things that need to be done with XBG
                  phil Simborg -- Tuesday, 30 June 2009, at 12:52 p.m.
                  • Two things that need to be done with XBG
                    aoleg1 -- Tuesday, 30 June 2009, at 1:41 p.m.
                    • Two things that need to be done with XBG
                      Bob Koca -- Tuesday, 30 June 2009, at 5:04 p.m.
                      • Two things that need to be done with XBG
                        Ian Dunstan -- Tuesday, 30 June 2009, at 9:24 p.m.
                        • Two things that need to be done with XBG
                          Matt Cohn-Geier -- Tuesday, 30 June 2009, at 9:34 p.m.
                          • Backgames
                            Ian Dunstan -- Wednesday, 1 July 2009, at 1:37 a.m.
                        • Two things that need to be done with XBG
                          Frank Berger -- Tuesday, 30 June 2009, at 9:38 p.m.
            • Two things that need to be done with XBG
              phil Simborg -- Tuesday, 30 June 2009, at 12:25 p.m.
              • Two things that need to be done with XBG
                Marty Storer -- Tuesday, 30 June 2009, at 2:23 p.m.
              • Two things that need to be done with XBG
                Rich Munitz -- Tuesday, 30 June 2009, at 3:47 p.m.
        • Two things that need to be done with XBG
          MaX -- Tuesday, 30 June 2009, at 8:25 a.m.
          • Two things that need to be done with XBG
            Bob Koca -- Tuesday, 30 June 2009, at 9:41 a.m.
          • Two things that need to be done with XBG
            playBunny -- Wednesday, 1 July 2009, at 5:08 p.m.
      • Two things that need to be done with XBG
        Frank Berger -- Monday, 29 June 2009, at 9:47 p.m.
      • Two things that need to be done with XBG
        Mislav Radica -- Monday, 29 June 2009, at 10:14 p.m.
  • Two things that need to be done with XBG
    aoleg1 -- Tuesday, 30 June 2009, at 12:59 p.m.
    • Two things that need to be done with XBG
      neilkaz -- Tuesday, 30 June 2009, at 5:58 p.m.
    • Two things that need to be done with XBG
      Stick -- Tuesday, 30 June 2009, at 7:57 p.m.
  • Two things that need to be done with XBG
    pontiac -- Tuesday, 30 June 2009, at 9:04 p.m.
    • Two things that need to be done with XBG
      Matt Cohn-Geier -- Tuesday, 30 June 2009, at 9:28 p.m.
      • Two things that need to be done with XBG
        Mislav Radica -- Tuesday, 30 June 2009, at 10:47 p.m.
        • Two things that need to be done with XBG
          Matt Cohn-Geier -- Tuesday, 30 June 2009, at 11:11 p.m.