[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums

Rollouts of Robertie's 501 problems: Pos 129 Snowie Eval

Posted By: Timothy Chow
Date: Friday, 26 March 2010, at 3:35 p.m.

In Response To: Rollouts of Robertie's 501 problems: Pos 129 Snowie Eval (Chuck Bower)

Chuck asks why rare, high cube values can cause variance estimates to undershoot.

Here's a crude way to get a feeling for the situation. Let C be a number (technically, a random variable) that can take on the values 1, 2, 4, 8, 16, .... Suppose that the probability that C = 2n is 2/(3n+1). That is, C can take on exponentially high values, but with exponentially low probability. You can check that the expected value of C is 2, but that its variance is infinite. Now imagine randomly sampling C to try to estimate what is going on. In practice, you're never going to see those very high values of C, and so your sample variance is going to be some small finite number. You're not going to realize that something is fishy unless you have the ability to take a very large number of samples and see that whenever you triple your sample size you suddenly see this unusually high value of C that drives the variance up.

The above is a caricature of money-game backgammon, but a similar effect is potentially at work. Exponentially rare, exponentially large values drive the variance up, but don't show up in small samples, so your sample variance will, with high probability, be an underestimate.

Messages In This Thread

 

Post Response
Your Name:
Your E-Mail Address:
Subject:
Message:

If necessary, enter your password below:

Password:
 

 

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

BGonline.org Forums is maintained by Stick with WebBBS 5.12.