| |
BGonline.org Forums
A better approch? [LONG]
Posted By: MaX In Response To: A better approch? (Timothy Chow)
Date: Thursday, 5 November 2009, at 3:04 p.m.
I've read the whole discussion (Tim, Tom and others) with interest.
My understanding (please correct if wrong) is that the advantage of the Bayesian approach is the fact it gives the direct answer to the question "How sure we are the top play is really better than the others". The jsd approach (or confidence interval approach) gives an answer to a somehow related question (but not exactly to our question).
Besides that, doing what Tim suggested or integrating the multidimensional normal cdf to find the final probability to be shown to the user (whatever the exact definition of that probability is) is equally difficult. Tim has an intrinsic discrete approach, Xavier/Maik have gone MonteCarlo.
Each method is "correct" under his own assumptions.
I've a python script that tries to play with the bayesian approach and compares to the current jsd approach. You enter N equities (true equities of the N plays to compare), and the number of trials. It then generates the "random" results of each trial (just win/lose, like in Tim's simplified DMP case). Finally applies Tim's procedure to get an estimation of the probability of play 1 being better than all the others, and computes jsd and confidence intervals of play1 being better than play2, play1 being better than play3 etc.
Here's an example of the output:
---------------------------------------------------------------------------------
Plays equities : [0.6, 0.5, 0.5]
N tirals : 400
MC size : 10000
Pbayes = 89.7943751777 %
Min CI = 90.0194955301 %
Pmc = 90.3 %
MUs : [0.58999999999999997, 0.54500000000000004, 0.47749999999999998]
SDs : [0.024653298500968295, 0.024960946500396732, 0.025037267842161228]
JSDs : [0.034865029097301659, 0.035083243538383296, 0.035137585403353579]
CIs : [0.50000000050000004, 0.90019495530127791, 0.99931683943276561]
---------------------------------------------------------------------------------
INTEPRETATION:
3 plays with real equities 0.6, 0.5 and 0.5, 400 trials (10000 MC samples for integration).
Pbayes is computed with Tim's method (discretization of pdf have 1001 points). it's 89.79%.
Min CI is the minimum confidence interval: for P1vsP2, P1vsP3, a confidence interval is computed via the classical jsd method, then the min of all is taken. Its value is 90.02%.
Pmc is computed via monte carlo integration of the 3 independent normnal distributions (like what Xavier/Maik did).It gives 90.3%.
Otherlines are the raw "rollout" results (means, stdev, jsds and confidence intervals).
Now I'm wondering if keeping the current approach but showing the CI% instead of the jsds is not just enough for our goal.
I would just do this: rank the plays by decreasing equity and show a percentage aside of each play:
- for the top play, the percentage is the min of all the confidence intervals of the other plays (being worse than the top one)
- for the others, the % is the confidence interval of top play being better than this play.
In the above example the output would be:
P1 Eq=0.5625 CI= 91.08% (min of the two below)
P2 Eq=0.5150 CI= 91.08%
P3 Eq=0.4650 CI= 99.71%
Notice that in general, it may happen that a play with a lower equity has as smaller CI (that's why I'd take the min).
I can send you the python code if you ant to play around, just ask. It's just 300 lines (most of which are empty/commment/trivial).
WARNINGS:
- I'm not a good python coder. it's far from optimized but runs pretty fast anyway.
- I didn't want to use NumPy, hence it really looks like C code.
MaX.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.