After learning over 10 years ago that "unbiased" TD-Gammon v1 had considered 63 a bad opening roll, I wondered if current bots misunderstood it and I started to experiment exploiting the bots with it.
As I discovered that how the games enfolded by how the bots played favored my style of play, I included 62 and 64 in my experiments also for being very similar plays.
During my experiments I noticed that different levels of GnuBG and XG played those rolls differently and I was surprised that bots could diverge so early in the game, right from the opening roll.
Below is a quick spreadsheet I created today, to visualize it better myself and to illustrate to you all how the bots differ. (I included BGBzzzt not because I played against it enough or care about it but just to see if there would be any obvious "genetic markers").
Hoping that you can understand my lazy-short comments, let me ask the question again: When bots diverge like this, how do you determine which one is right?
And a bonus question: how much inconsistencies and inaccuracies would it take for you all to finally say that bots may be significantly unreliable, beyond the tip of the iceberg that you are able observe?
MK