XGR++ vs. 1k Rollout
Bob asks, “It sounds that given the choice you normally would rather have the XG++ result than a 1296 game 3 ply with roller for cubes. Is that correct?”
In this analysis of the Deprelli Study, Xavier concludes, “Note that a full 3-ply 1296 game rollout appears as strong as a XGR++ analyze (0.115 and 0.117 PR respectively).” What that means is that either one is just as good at selecting the best move. My statement was a little different. It regards the variance in the reported margins between plays. In general, an XGR++ eval does a better job gauging the margins between plays than a full 1k rollout. So, yes, in most cases, I prefer an XGR++ eval to a 1k rollout.
Bob also points out that, “In this particular case the difference from roller is .018 and from RO is .002 so the actual difference would need to be greater than .010 in a long RO for your statement to be correct. How likely is that given the confidences of only .004 and .004 from the RO?”
My answer is, “Not very likely!”
After posting, I recalled something Tim brought up, namely that full rollouts, even short ones, often perform better than XGR++ in endgame positions. Tim wrote, “I wonder if your experience is skewed by excessive exposure to early-game rollouts.” Perhaps that is true to a degree, and perhaps it is also true of the Deprelli Study. In any event, the rollouts below show that XGR++ and 1k both did a decent job in this position.
Mike
Comparing 1k Rollout (left) and 10k Rollout (right)
If you widen your window, the two rollouts below will appear side by side. Otherwise, the 1k rollout will appear above the 10k rollout.
The 1k rollout does a good job at identifying how close the top two plays are. It also comes within 0.0027 of the 10k rollout on the third- and fourth-best plays, although puts them in the wrong order. Given that Ah_Clem only posted three digits in his 1k rollout, this margin might be a bit smaller or larger if the four-digit result were available.
Comparing XGR++ Eval (left) and 10k Rollout (right)
Once again, if you widen your window, the two rollouts below will appear side by side.
In its assessment of the second-best play, XGR++ does not come as close to matching the result of the 10k rollout as the 1k rollout does. That said, it is off by less than 0.006.
XGR++ does a better job than the 1k rollout on the third- and fourth-best plays. Unlike the 1k rollout, XGR++ ranks them in the correct order. The largest error on these two moves (relative to the best play) is 0.0014, almost half of what the 1k rollout produces.
