| |
BGonline.org Forums
Another issue with excluding "obvious" moves from the PR calculation
Posted By: Timothy Chow
Date: Saturday, 21 January 2017, at 9:35 p.m.
Various people, such as Tom Keith and Neil Robins, have articulated some reasons why Xavier's decision to exclude "obvious" or "forced" moves from the PR denominator can be questioned. Here I would like to articulate another issue that I don't think has been stated before, at least not in quite this way.
Let me begin by recalling that, during his brief appearance on BGOnline a few years ago, Wilcox Snellings once posed the question, What is your error rate on tough problems? If we think about this question for a moment, we see that any "error rate" calculation is going to be very sensitive to the set of positions that we are averaging over. Even if we could agree on objective criteria for what constitutes a "tough problem" and could amass a lot of examples and test people on them, the "error rate on tough problems" would clearly be very different from what we normally think of as error rate. It is not simply that "easy" problems have been excluded; many types of problems that are notoriously difficult (pay now or later, prime versus prime, match cube actions, etc.) will show up disproportionately often in any compilation of "tough problems" compared to how often they show up in actual play.
The conclusion that I want to draw from this is that the intent of the usual ER or PR calculation is to normalize the size of your errors according to how often those errors occur in actual play. If you consistently make a certain type of error on the second or even first roll of the game, then the magnitude of the equity loss might seem somewhat small when considered in isolation, but since the decision comes up a lot, it gets more heavily weighted in an ER/PR calculation. Conversely, if you make a 0.250 blunder in an Othello quiz problem, but the position is so unusual that it almost never shows up in actual play, then it's not going to figure heavily in your ER/PR. This is as it should be, since performance in actual play is what we want ER/PR to measure.
Now let me come to the issue of excluding "obvious" or "forced" moves. I claim that one important effect that this has is to skew the relative weights of various kinds of decisions. For example, consider a type of checker-play error that tends to lead to your getting blitzed and closed out a lot. All those dances from the bar are eliminated from the denominator of the PR calculation, and so your error, in effect, gets weighted more heavily than it would if those forced moves were included. To put it another way, the artificial exclusion of the forced moves makes those kinds of checker-play errors appear to occur more often in practice than they actually do.
By the way, I'm not advocating that the definition of PR be changed. As they say, the nice thing about standards is that there are so many to choose from. I think it's a good thing that the backgammon community seems to be converging on a single standard. It is better to have a single standard, even if it is flawed, then to tinker with it endlessly trying to make it "better" (whatever that means). However, I think that it is good to be aware of the limitations of the PR calculation, and not to blindly believe that it is necessarily better than ER.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.