[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

How to design a statisctally significant bot head-to-head?

Posted By: Maik Stiebler
Date: Monday, 11 January 2010, at 4:08 p.m.

In Response To: How to design a statisctally significant bot head-to-head? (Ian Shaw)

You can look here and here for foggy references to an experiment that successfully employed duplicate dice streams. In your case I suspect it might work a little less well, unless your contestants do not only play nearly equally well, but nearly equally. But still, I think for static evals duplicate dice streams without conventional variance reduction is the way to go, because conventional variance reduction should take about 20 times as long as playing the actual games.

BTW,

In my example, I have a estimate that Bot A is 0.005 ppg better than Bot B. So its probability of winning is 50.0025. Putting this into Timothy's formula I get 4/(50.0025 - 50)^2 = 640,000 games.

you got me. It should be 50.25%, and Timothy's formula would go 4/(0.5025-0.5)^2 = 640,000 games.

Messages In This Thread

 

Post Response
Your Name:
Your E-Mail Address:
Subject:
Message:

If necessary, enter your password below:

Password:
 

 

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums is maintained by Stick with WebBBS 5.12.