[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

Rollouts

Posted By: Daniel Murphy
Date: Sunday, 21 October 2007, at 8:20 p.m.

In Response To: Rollouts (Stick)

it was mentioned on either my forums or Kit's forums how ppl see a '1296 game rollout' and sort of blindly trust the results, this is just one example of why you shouldn't.

Indeed we shouldn't, which is why some of us (we know who we are!) repeatedly remind readers to pay attention to Standard Deviations and Confidence Intervals.

Play CF after 1296 trials 1 SD +- 1SD +- 2SD +- 3SD CF after 5184 trials
24/22 24/21 -0.0177 0.0187 from -0.0354 to 0.0010 from -0.0541 to 0.0197 from -0.0728 to 0.0384 -0.0455
24/22 13/10 -0.0578 0.0171 from -0.0749 to -0.0407 from -0.0920 to -0.0236 from -0.1091 to -0.0065 -0.0532

After 1296 trials, the equity for 24/22 24/21 was -0.0177. After 5184 trials, the equity was -0.0455. The difference of the results was 0.0278. In contrast, the equity for 24/22 13/10 changed by only 0.0046.

Should we be surprised by the change in equity for 24/22 24/21?

Emphatically, no. Look at the gray box, which is a range of numbers from two standard deviations below to two standard deviations above the 1296-trial equity.

That range tells us that, having run one 1296-trial rollout of 24/22 24/21, if we had proceeded to run more and more rollouts of the play (not more trials, but more rollouts with different seeds), about 95.45% of the time we should expect any one result to be somewhere between -0.0541 and 0.0197. The result in the 5184-trial rollout, -0.0455 is in that range.

What would have been quite surprising is if the result had been greater than 0.0384 or less than -0.0728, three SDs below or above our first result. But even that would not have been impossible, only very unlikely.

Should we be surprised that after 5184 trials the difference between 24/22 24/21 and 24/22 13/10 is only 0.0086 (rounded) instead of 0.0401? Again, no. -0.0455 is between -0.0541 and 0.0197. -0.0532 is between -0.0920 and -0.0236. No surprises there. Nothing in the original rollouts tell us that should assume that the equity values must remain centered in the narrowing confidence intervals as we extend the rollouts, or that the difference between the plays should remain about the same no matter how many trials we run.

http://en.wikipedia.org/wiki/Confidence_interval

http://en.wikipedia.org/wiki/Standard_Deviation

Messages In This Thread

 

Post Response
Your Name:
Your E-Mail Address:
Subject:
Message:

If necessary, enter your password below:

Password:
 

 

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums is maintained by Stick with WebBBS 5.12.