[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

Confidence

Posted By: Timothy Chow
Date: Sunday, 12 June 2011, at 10:49 p.m.

In Response To: BQ: The bot says there is only one play *ROLLOUT* (leobueno)

There are couple of things to note about stopping when you reach a certain JSD threshold.

1. The reported JSD is only an estimate. In statistics one usually doesn't know the truth and must rely on estimates; this is the case here. With a low number of trials, the estimate could be off. Therefore it's good to roll out some minimum number of trials before trusting the JSD estimate. I usually use 1296 as a minimum but if you're impatient you could use something smaller; beware, though, that the smaller your threshold, the more risk you have of an incorrect estimate.

2. Confidence refers to the confidence you have in rejecting the hypothesis that the play is just as good as the top play. It doesn't give you any "confidence" in the value of the equity or equity difference. That is, you might be tempted to say, "I'm 99% confident this is a whopper." But 99% confidence as reported by the JSD doesn't allow you to draw this conclusion, because "whopper" is a quantitative estimate of the equity difference.

3. Stopping when the JSD reaches a threshold means that the confidence level of the experiment is actually lower than what you would get by naively looking up the JSD value in a table of Gaussian tail estimates. See this post for more information.

Messages In This Thread

 

Post Response
Your Name:
Your E-Mail Address:
Subject:
Message:

If necessary, enter your password below:

Password:
 

 

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums is maintained by Stick with WebBBS 5.12.