| |
BGonline.org Forums
Wisdom of the Crowds for Ranking
Posted By: Casper Van der Tak In Response To: My take on the Giants list (Casper Van der Tak)
Date: Tuesday, 23 February 2016, at 10:23 a.m.
I watched it now completely, and it was very interesting and balanced. Thanks Matt!
One thing that struck me is that the Wisdom of the Crowd has usually been applied to number estimates, and not to ranking estimates. I am not sure that the conclusions would easily translate to the latter. Averaging rankings is problematic, because it is not clear whether the difference between rank 1 and 2 is the same as between rank 2 and 3. It is not clear that averaging is meaningful (but that should not stop us from averaging; see below).
One thing you mentioned is the weight of the votes. In the case of number estimates, you just take the average, and that is simple. In the case of averaging ranked lists, the procedure is less straight forward. What you describe in the video is how it is done in the Giant's list: 32 pt for number 1, 31 for number 2, etc., and then average. This creates the possibility for a few outliers to greatly effect the results, as you pointed out.
I am wondering whether doing it slightly differently would create more meaningful results. For example, Sqrt(32) for number 1, Sqrt (31) for number 2, etc., and then average. My guess is that this would lead to more reasonable results.
That said, I think the top of the Giant's list is very reasonable and hard to argue against. Seems like the right players are in the top-10; perhaps the order "ought" to be different, but who knows?
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.