| |
BGonline.org Forums
Variance reduction is unbiased but may be skewed
Posted By: Timothy Chow In Response To: Variance reduction is unbiased but may be skewed (David Rockwell)
Date: Saturday, 11 April 2015, at 4:06 p.m.
David Rockwell wrote:
To the extent that the evaluation function is imprecise, the variance reduction will be imprecise. I've seen no reason to believe this potential error term is significant or biased. But, to me it is a bigger issue than error due to skew.
It might help to use a bit of technical jargon here. A probability distribution is described by its moments. The first moment controls the average value, the second moment controls the variance, and the third moment controls the skew. Higher moments control increasingly subtle features of the probability distribution.
The imprecision of the evaluation function means that the luck does not have exactly the right probability distribution. What I'm trying to get at here is exactly how that probability distribution is imprecise, by understanding how the higher moments might be off.
The fundamental fact about variance reduction is that the first moment (i.e., the average value) of the luck is exactly zero, which is what it should be. This is what is meant by the saying, "luck is unbiased." This follows just from the way luck is defined. It doesn't matter how wildly wrong the evaluation function is; the average value will be correct. In the long run, this means that a VR rollout will converge to the correct value.
Unfortunately, the second, third, and higher moments could be off, and this can affect the results we see. Roughly speaking:
1. If the second moment is off then the reported confidence intervals may not accurately reflect the range in which the non-VR equity (probably) lies.
2. If the third moment is off then the non-VR equity may be more likely to lie at the high end of the confidence interval than at the low end (or vice versa).
All I'm observing here is that the fact that doublets tend to be lucky may be a reason to worry about the third moment.
Your intuition may be telling you (in my language) that inaccuracy in the second moment is more of a practical concern than inaccuracy in the third moment, because it could mislead us into thinking that we have a sharper estimate than we really do. I agree with this sentiment. Unfortunately I don't currently have any interesting insight to report about the second moment of the luck.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.