|
BGonline.org Forums
Using Noisy Evaluations to Measure Performance
Posted By: Bob Koca In Response To: Using Noisy Evaluations to Measure Performance (Chris Yep)
Date: Thursday, 31 March 2016, at 2:03 p.m.
The partial decisions idea was proposed here a few years ago. I forget by whom.
"N = standard deviation of noise; it shouldn't be too small or too large; my guess is that it's better to err on the high side; maybe use something in the range of 0.04 to 0.08.
Note that to better mimic humans, N should be smaller in certain types of positions (e.g. bearoff checker plays) and larger in other types of positions (e.g. complex prime vs. prime checker plays)."
Is there also an issue with dependence? For example suppose there are two ways too race and too ways to stay and shuffle checkers inside. A human might rate one type of play incorrectly and both the plays would be too high. The bot though might have one too high and one too low.
I've played games against bots using the noise to simulate an intermediate player and I thought that overall it plays equally well as an intermediate but the errors did not realistically match a human intermediate.
|
BGonline.org Forums is maintained by Stick with WebBBS 5.12.