Rollout: who to trust - Gnubg or XG

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

Rollout: who to trust - Gnubg or XG - backgame

Posted By: Taper_Mike
Date: Tuesday, 26 April 2016, at 12:49 p.m.

In Response To: Rollout: who to trust - Gnubg or XG - backgame (sebalotek)

I am glad you found my post "Simulating XG Roller++ in GNU Backgammon." It describes how to create a truncated rollout in GnuBg that closely matches the way an XGR++ eval works.
It is very good at what it does, but it also has many limitations. For complex positions such as a backgame and/or containment game, stronger settings are needed. The b.rol trunc looks ahead only 5 moves. A full rollout will "look ahead" all the way to the end of a game. Clearly, a full rollout is stronger.
Another issue is what GnuBg calls the Move Filter. In XG, the same thing is termed the Search Interval. It determines how many plays are examined for each move. The XG help system explains, "For instance in 3-ply, the normal search interval will analyze up to 4 moves in the top level. If you change the search interval to huge, it will check up to 8 moves. If you pick gigantic, then 16 moves can be analyzed."
Backgames, and the containment games that often result after a late hit, are both known weak areas for bots. One way to improve the performance of a bot in these kinds of games is to increase the size of the Search Interval. To get the best results, you must do both: increase the Search Interval and do a full rollout.
Full rollouts can take a long time. When you increase the Search Interval, they will take even longer.
Even a long rollout, however, is not a guarantee that XG and GnuBg will rank plays in the same order. Their neural nets are optimized to put the "best" play at the top of the list. They are not intended to accurately rank all the candidate plays.
In your position, the results for GnuBg and XG are not all that far apart. Below I used my b.rol for GnuBg and XGR++ for XG. Notice how close the plays are. Both bots picked the same three plays. All are within 16 millipoints of the play picked as best.
Match to 7 points - Opponent 0, Player 2
Opponent 210

Player 126
Position ID: sCMJ2BaDu8FDAA Match ID: QYnyAAAAEAAE

# Ply Move Equity

1 R 13/9 13/8 -0.379

0.453 0.249 0.006 - 0.547 0.126 0.003 +0.022 -0.379

0.002 0.002 0.000 - 0.002 0.001 0.000 0.005 0.007

Truncated cubeful rollout (depth 5) with var.redn.

360 games, Mersenne Twister dice gen. with seed 775378628 and quasi-random dice

Play: world class 2-ply cubeful prune [world class]

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Skip pruning for 1-ply moves.

Cube: 2-ply cubeful prune [world class]

Different evaluations after 2 plies:

Play: 1-ply cubeful prune

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Cube: 2-ply cubeful prune [world class]

2 R 17/12* 6/2* -0.393 ( -0.014)

0.452 0.264 0.007 - 0.548 0.135 0.004 +0.027 -0.393

0.001 0.001 0.000 - 0.001 0.001 0.000 0.005 0.007

Truncated cubeful rollout (depth 5) with var.redn.

360 games, Mersenne Twister dice gen. with seed 775378628 and quasi-random dice

Play: world class 2-ply cubeful prune [world class]

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Skip pruning for 1-ply moves.

Cube: 2-ply cubeful prune [world class]

Different evaluations after 2 plies:

Play: 1-ply cubeful prune

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Cube: 2-ply cubeful prune [world class]

3 R 13/9 7/2* -0.395 ( -0.016)

0.455 0.244 0.006 - 0.545 0.140 0.005 -0.001 -0.395

0.001 0.002 0.000 - 0.001 0.001 0.000 0.005 0.007

Truncated cubeful rollout (depth 5) with var.redn.

360 games, Mersenne Twister dice gen. with seed 775378628 and quasi-random dice

Play: world class 2-ply cubeful prune [world class]

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Skip pruning for 1-ply moves.

Cube: 2-ply cubeful prune [world class]

Different evaluations after 2 plies:

Play: 1-ply cubeful prune

keep the first 0 0-ply moves and up to 8 more moves within equity 0.16

Cube: 2-ply cubeful prune [world class]

White is Player 2

score: 0
pip: 210

7 point match

pip: 126
score: 2

Blue is Player 1

XGID=-BabbbCCB---aDa-aA-cb-----:1:-1:1:54:2:0:0:7:10
Blue to play 54

1. XG Roller++ 17/12* 6/2* eq: -0.486
Player:
Opponent: 44.40% (G:25.71% B:0.68%)
55.60% (G:14.28% B:0.65%)
2. XG Roller++ 13/9 7/2* eq: -0.499 (-0.013)
Player:
Opponent: 44.17% (G:24.80% B:0.68%)
55.83% (G:14.58% B:0.66%)
3. XG Roller++ 13/9 13/8 eq: -0.501 (-0.015)
Player:
Opponent: 44.16% (G:24.10% B:0.64%)
55.84% (G:14.33% B:0.64%)

eXtreme Gammon Version: 2.10, MET: Kazaross XG2

Note: In GnuBg, I made my printout by 1st choosing "Setting > Export..." and then changing the value for "Show at most xx moves" to 3. Then I used the command "Edit > Copy as > GammOnLine (HTML)." Finally, I switched over to the message box at BgOnline, and used Ctrl+V to paste it in.
Mike