| |
BGonline.org Forums
PR and luck in live matches
Posted By: Jeom
Date: Wednesday, 20 September 2017, at 12:15 p.m.
I have fetched 936 actual live matches from Backgammon Studio and created a dataset including the PR level of two players and the winner of the match. 501 matches are to 11 points, 360 matches are to 17 points, and 75 matches are to 25 points.
Each match consists of Player 1 and Player 2. I created a variable containing the difference in PR (var_prdiff) by subtracting the PR of Player 2 from the PR of Player 1. I then created a dummy variable (var_win) indicating wether or not Player 1 won (1) or lost (0) the match.
I then did a logistic regression with var_win as the dependent variable and var_prdiff as the independent variable.
After creating the model, it is possible to predict the winning chances of Player 1 based on the difference in PR with Player 2. A negative value of var_prdiff indicates that Player 1 have played with better PR than Player 2.
Given that the PR level accurately measures the skill of a player (not the point for discussion here), the probablity of winning when playing at exactly the same PR should be 0,5. This is not the case when looking at the data, most likely because of the small sample (I would lean towards this before blaming PR to be a bad measure of skill). This can however be 'solved' by including a mirror of the matches in the dataset (i.e. adding all matches between Player 2 and Player 1 as additional data points, increasing N to 1872), artificially reaching a probability of winning at PR-difference 0 of exactly 0,5. If anyone that is more skilled than me when it comes to statistical methods have an argument against this step, please let me know.
The results of running a logistic regression of the model including the 'double dataset' (model3) are attached below.
Predictions are as follows: 0 = 0.5 -0.5 = 0.5116552 -1 = 0.5232978 -1.5 = 0.5349152 -2 = 0.5464947 -2.5 = 0.5580242 -3 = 0.5694914 -4 = 0.5921922 -5 = 0.6145063 -6 = 0.6363491 -8 = 0.6783219 -10 = 0.7176001 -14 = 0.78678 -18 = 0.8427269 -25 = 0.910557
In other words, playing at a PR level being 2 points better than your opponent should give you a 54.65% chance of winning the match, and so on and so forth.
Now some questions: 1) Knowing the methodology for calculating PR and luck in XG, it should be possible the corresponding predictions without using atcual match data. Has it been done? (i.e. how does these results from live matches compare to the expected results) 2) Given that the PR does not get worse when playing worse players (as has been discussed here recently), would you say that these numbers are what you would have expected? I sometimes hear people talking about a rate of winning of about 60% for really good players, which would indicate that they on average play opponents being 4-5 PR worse than they are. Sounds reasonable, right?
Please criticize my methodology in a civilised manner. I know that the sample is small and that it would be more interesting to see differences in results between matches of different length. I am no stat-wizz, so it might even be that it's all rubbish, but I don't think so.
By the way, I want more data! Anyone? (and this study has not been sanctioned by Terje, but I hope he does not disapprove :)
> summary(model3)
Call: glm(formula = var_win ~ var_prdiff, family = binomial(link = "logit"),
data = ma3)
Deviance Residuals:
Min 1Q Median 3Q Max -1.628 -1.159 0.000 1.159 1.628
Coefficients:
Estimate Std. Error z value Pr(>|z|) (Intercept) -9.336e-17 4.685e-02 0.000 1 var_prdiff -9.326e-02 1.348e-02 -6.916 4.65e-12 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 2595.1 on 1871 degrees of freedom Residual deviance: 2544.4 on 1870 degrees of freedom AIC: 2548.4
Number of Fisher Scoring iterations: 4
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.