| |
BGonline.org Forums
Have you ever seen anything like this?
Posted By: MK In Response To: Have you ever seen anything like this? (Timothy Chow)
Date: Friday, 3 May 2024, at 3:14 a.m.
After posting my previous reply on this, I kept thinking about it, especially reconsidering your comments, and came to think that it is more likely to happen during rollouts than during my experiments, for the following reasons:
> This type of position doesn't come up enough during training
How do you know? Since the training is by random self-play, any and all possible positions should be equally likely to occur. You can't say that certain positions, i.e. "this type" doesn't occur enough.
Also, it's important for you to understand that the first position that I posted is not the first position, i.e. the cause, of the long sequence of weird moves.
The sequence probably started with a barely noticeable unusual position much earlier (that I wasn't able to recover and save).
The other positions I posted are also just a few I picked as examples from several hundred that I was able to capture and save. I can post them if it would be of any use for any reason.
> (and when it does come up, the time horizon is too long)
Could you explain this a little, preferably without distracting too much from the main subject at hand.
> for the neural net to learn how to play well, so it makes all
: kinds of strange-looking moves.I can agree with this for one or only a few moves at some positions that the bot didn't learn well but it should then correct itself before it goes on to making hundreds more of strange moves, no?
> Eventually, just by random chance if nothing else, the checkers
: will move past each other, and the position will turn into a pure
: race, which the bot will understand (at least in the later stages).The key concept here is "understanding a position", for both humans and bots.
Beginning and ending game stage positions must be the easiest to understand, at least for bots since those would occur most frequently during training and thus the bot would learn to play them well.
The games in my experiments start from the beginning and go to the end with some possibility of unusual and complex positions during the middle stages of games that bot may not have learned well.
In contrast, it seem to me that you guys are doing rollouts of positions that either you don't understand well or that you suspect the bot doesn't understand well, which are more likely to be unusual and complex positions during the middle stages of games.
With some possible rare exceptions, bot's plays at those positions will not be totally berzerk but just suspicious enough for you to do rollouts.
The critical point here is that many strong human players may very well correctly detect some positions that the bot hasn't learned to play well but that doesn't play too obviously badly either. The position may be just odd enough to be the seed from which a tree will grow after a few moves down the road. In other words, I think "this kinds of positions" are much more likely to occur during rollouts that are triggered by suspect plays than during boringly common bot-vs-bot games.
Since all those individual moves during tens or hundreds of thousands of trials during rollouts aren't visible, there is no way to observe and know if and how often this may be occurring, effecting all equity, error, etc. calculations in rollouts, (maybe even in shorter evals/mini-rollouts also).
MK
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.