| |
BGonline.org Forums
The wisdom of crowds and the OLM
Posted By: Timothy Chow
Date: Sunday, 27 November 2011, at 8:16 p.m.
Buried in a recent thread, there was some discussion about which of the following strategies would produce better team performance against the bot in the OLM:
Follow the Giant (FtG): Decide who the best player is and have that player make all the moves.
Wisdom of Crowds (WoC): Vote blindly and independently and make the majority play.
Intuitively, it might seem that FtG is the way to go. (This was Stick's intuition, for example.) The argument runs as follows. I, as a participant in the OLM, can either evaluate the position myself, or I can look at what the Giant's vote is. Since the Giant has a higher probability of picking the right play than I do, surely I do better by imitating the Giant's play than by giving my own play.
The point of Surowiecki's book, though, is that this intuition is not always correct. Empirically, one finds that in some contexts, WoC outperforms FtG. Here's an oversimplified model that helps illustrate how this could be possible. Suppose that a bunch of tough pay-now-or-later problems arise in the OLM. Suppose that the Giant's batting average is 0.800 (i.e., the Giant gets 80% of the decisions right) whereas the typical BGO Donkey's batting average is 0.600. Finally, suppose (and this is the critical assumption, which is not complete realistic) that the Donkeys' performances are independent, meaning that a particular Donkey is no more or no less likely to get a particular decision right if the other players get it right. (The reason that this is unrealistic is that if other players get it right then that probably means the decision is a relatively easy one, so that the particular Donkey's chances are probably better than 60% in that case; however, as long as the Donkey doesn't consult the other players, the independence assumption is not completely unreasonable.) If there are enough Donkeys voting independently, then the probability that the majority of them will get the answer wrong is much less than 20%, so WoC will beat FtG hands down.
Now as I've said more than once already, the independence assumption is oversimplified. However, as Surowiecki's book details, WoC does work surprisingly well for many real problems. For example, fans of Who Wants to Be a Millionaire? will undoubtedly have noticed that asking the audience is an extremely reliable way of getting the right answer to a trivia question.
Once one accepts that it's not obvious whether FtG or WoC works better for the OLM, the question arises as to whether there is some hybrid strategy that will work better than either strategy by itself. For example, the way the World Team organized itself in Kasparov versus the World was to have a few experts narrow down the candidates first, before opening up the general vote. The way that that particular chess match unfolded was an instructive illustration of the strengths and pitfalls of such a system.
Unfortunately, we don't really have the resources on BGO to conduct a proper study of different OLM strategies. So any discussion will necessarily remain at the level of opinion and conjecture. Still, I find it more interesting than debates about rulings on tournament irregularities.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.