BGonline.org Forums

Minimum # of Trials and Confidence

Posted By: Timothy Chow
Date: Friday, 15 October 2010, at 11:08 p.m.

In Response To: Minimum # of Trials and Confidence (Stick)

Daniel certainly overstated the case for short rollouts. For an analogy where common sense should help us cut through the mathematical fog, suppose I have a coin and I want to estimate the probability that it comes up heads when I flip it. No one in their right mind would suggest that a single trial is enough, even though the estimated variance from a single trial is 0 so (blindly following the formulas) we can be 100% confident in the result.

All those confidence percentages are implicitly based on the normal approximation. The normal approximation is correctly only asymptotically, i.e., when the number of trials is large.

Messages In This Thread

• Minimum # of Trials and Confidence
  Stick -- Friday, 15 October 2010, at 10:21 p.m.
  • Minimum # of Trials and Confidence
    Timothy Chow -- Friday, 15 October 2010, at 11:08 p.m.
    • Minimum # of Trials and Confidence
      Tom Keith -- Saturday, 16 October 2010, at 1:31 p.m.
  • Minimum # of Trials and Confidence
    Daniel Murphy -- Saturday, 16 October 2010, at 2:22 a.m.
    • Minimum # of Trials and Confidence
      Stick -- Saturday, 16 October 2010, at 5:32 a.m.
      • Minimum # of Trials and Confidence
        Daniel Murphy -- Saturday, 16 October 2010, at 6:34 p.m.
        • Minimum # of Trials and Confidence
          Neil Robins -- Sunday, 17 October 2010, at 5:32 p.m.
          • Minimum # of Trials and Confidence
            Stick -- Sunday, 17 October 2010, at 6:19 p.m.
            • Minimum # of Trials and Confidence
              eXtreme Gammon -- Sunday, 17 October 2010, at 7:27 p.m.
              • Minimum # of Trials and Confidence
                Daniel Murphy -- Wednesday, 20 October 2010, at 5:55 p.m.
    • Minimum # of Trials and Confidence
      Timothy Chow -- Sunday, 17 October 2010, at 4:53 p.m.
      • In defense of single trial rollouts
        Tom Keith -- Sunday, 17 October 2010, at 7:46 p.m.
        • In defense of single trial rollouts
          Maik Stiebler -- Tuesday, 19 October 2010, at 11:42 a.m.
          • In defense of single trial rollouts
            Tom Keith -- Tuesday, 19 October 2010, at 2:54 p.m.
            • In defense of single trial rollouts
              Maik Stiebler -- Tuesday, 19 October 2010, at 3:56 p.m.
            • In defense of single trial rollouts
              Bob Koca -- Tuesday, 19 October 2010, at 7:18 p.m.
              • In defense of single trial rollouts
                Daniel Murphy -- Tuesday, 19 October 2010, at 8:42 p.m.
              • In defense of single trial rollouts
                Tom Keith -- Wednesday, 20 October 2010, at 11:17 p.m.
    • Minimum # of Trials and Confidence
      Tom Keith -- Sunday, 17 October 2010, at 8:19 p.m.
  • OT Minimum # of Trials and Confidence
    svilo -- Saturday, 16 October 2010, at 8:46 a.m.
  • Minimum # of Trials and Confidence
    higonefive -- Saturday, 16 October 2010, at 11:25 a.m.
  • 324 is basically my minimum.
    neilkaz -- Tuesday, 19 October 2010, at 4:50 p.m.