| |
BGonline.org Forums
Calculating confidence values
Posted By: Timothy Chow In Response To: Calculating confidence values (Tom Keith)
Date: Sunday, 30 August 2009, at 6:40 p.m.
Yes, you can certainly play games with the priors in this way. However, before you get carried away with that, I would caution that there are a lot of statistical pitfalls along the way. If you generate some data, look at the results, then use those results to decide what to do next, then you can easily trick yourself into thinking that your results are more significant than they are.
The statistically pure thing to do is to lay out your full strategy ahead of time. "I'm going to do this many trials, then I'll follow Plan A if I get this result; otherwise I'll follow Plan B, and then after I see the results of Plan B I'll..." Only then can you properly calculate the probabilities of various things. If you start collecting data and then halfway through get an idea about what to do next which you hadn't planned at first, then all your statistical significance figures will be suspect.
These comments apply not just to backgammon, of course. If you're not already aware of how sloppy the statistical procedures of even top scientists are, you would be shocked. They're constantly making things up as they go along, and doing things that would give a professional statistician apoplexy. Anyway, the point is that I would avoid the temptation to try to take shortcuts by monkeying with the priors unless you're sure you know what you're doing.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.