Arnold Kling  

The Wisdom of Crowds?

PRINT
Hours Worked In the U.S. vs. E... The Success of Failure...

Are "predictions markets" more effective than other mechanisms at making forecasts and decisions?

This issue has been given a lot of publicity, starting with a now-scotched proposal to set up a terrorism futures market. Professor Bainbridge weighs in.


As for whether such markets will "revolutionize corporate forecasting and decision making," as GMU economist Robin Hanson claimed in the Time article, remains to be seen. I remember when quality circles were going to revolutionize the world. And I remember when self-directed work teams were going to revolutionize the world. I also remember that some of the people who predicted that self-directed work teams would revolutionize the world were some of the same people who had said precisely the same thing about quality circles a few years earlier.

Bainbridge offers a Hayekian view of how prediction markets might work better than bureaucratic decisions.

I find this more plausible than the "wisdom of crowds" argument, although I have not read James Surowiecki's book yet. I have strong memories of being against the crowd (when I wrote about the Internet Bubble, for example), with subsequent vindication.

I may be suffering from selective memory--a tendency to remember when I outsmarted the crowd, and vice-versa. But I think that the success stories for predictions markets may suffer from selective memory as well. Up to now, we've been hearing the best stories from the most enthusiastic supporters. As more efforts are made to establish and debunk predictions markets, my prediction is that the reputation of predictions markets will suffer. It's a bet I'd be willing to make.

For more on predictions markets, see Tyler Cowen.

For Discussion. If companies were to use predictions markets to make decisions, in what ways might these markets be manipulated by employees with special interests?


Comments and Sharing





COMMENTS (9 to date)
Brad Hutchings writes:

It seems to me there are two ways that these information markets can give meaningful information. The first is to capture a moving consensus. In a terror futures market, the consensus view would probably represent the hysteria of the community (whether influenced by news, intelligence, or the like). In political campaign futures markets, the consensus view would be driven by news, political ads, etc.

But the second, and probably most valuable and actionable source of information would be people gaming the system. In a terror market, it might be people with connections to terrorists who have a better sense of whether the hype is overblown or some real attack is imminent. In a predictions market, maybe there is arbitrage by some unit of the company. For $X invested, they could land $5X in additional funding. So it seems that the key to extracting such information, which is probably the highest value information such markets could provide, is to tolerate enough of the"cheating", detect it, and keep your mouth shut that this is the real purpose of the market so as not to discourage the cheaters who will provide you with the most information.

Andy Imboden writes:

I have worked as a meteorologist (the real kind, not TV) and I will attest to "the wisdom of crowds" in settings without monetary incentives. A consensus forecast is, on average, much better than individual forecasts, even if there is no communication between participants. I don't think that creating a market for forecasts in any arena will fail to do a good job of forecasting and decision-making when applied to a corporate setting.

Execution of the plan however is a different game altogether. Even the "optimal" forecast in a corporate setting may be useless if the ship cannot be navigated to the destination. Forecasts do not help in decision making if the consequences of being wrong, however slight, would be catastrophic. That requires management and bureaucratic oversight, checks and balances, and is beyond the scope of markets. Corporate forecasts and decision-making will be improved with efficiencies of market forces, but there will not be a revolution. A better handle on cost/benefits will not eliminate uncertainity, and that risk must still be managed.

I do not think that there is enough public information for a terrorism futures market to work as a predictive device. We certainly hear about "successful" terrorist acts, but the public is not informed of 100% of thwarted terrorist acts. My feeling is that if a terrorist futures market existed, the markets would be moved not by actors with specific inside information, but would be swamped by the crowds of people using the market as a sort of insurance policy (say if you live/work in Washington D.C.)

I am currently reading Surowiecki's book on the subway this week, I'll re-post if I change my mind.

Lawrance George Lux writes:

Perdictions markets are predicated on the concept that information is impacted at lower levels, and that the interpretation of information is always correct (trained professionals). This is reminisent of the Bush administration manipulation of the Public to get the Homeland Security Dept. and the Patriot Acts.

Information known at the Top can be easily sequestered, if such information is limited to a few knowledgeable individuals. Untrained non-professionals can be swayed by rumors--which can be artifically generated.

Belief in any Management decision which is not based on intense critical review by trained personnel, will always lead to disaster. lgl

Boonton writes:

I disagree, Surowicki's book demonstrates that the 'wisdom of crowds' does not depend on the members of the crowd being particularly good at interpreting information...nor does it depend on each member having all the relevant information. What's more important is that the crowd is independent & it's members feel free to speak their mind.

The example he used was an observation that Galton made of a hog competition. There people submitted guesses of a hogs weight with the person coming closest winning a prize. The average guess was remarkably close to the true weight.

The guessers were not especially knowledgable but they were independent. If 10 people guessed 400 lbs that wouldn't stop me from guessing 500 lbs. I'm not sure these characteristics are always available in a market.

Robert Schwartz writes:

Its the old horse race bettor’s problem. The odds on favorite to win is most likely to win and least likely to make you money. The way to make money is to bet against the crowd, but only when you are actually smarter than they are.

The same is true in the stock market. In March 1999, the Dow hit 11,000. I trimmed equity exposure in several of our accounts. For 2 years I thought I was an idiot. Now I feel pretty good.

The politicians (now there is a group of outside-the-box-thinkers) are blaming the CIA for overestimating Saddam Hussein and for underestimating Osama. The CIA is like the crowd at the track. They pick the favorites. AND NO CONCEIVABLE GOVERNMENTAL BUREAUCRACY WILL EVER BE ANY DIFFERENT.

Prediction Markets cannot solve the CIA problem, They can broaden the consensus, but it will still be a consensus. Case in point. The CIA consistently overestimated the economic and military strength of the Soviet Union. The only experts who saw a chink in the Soviet armor were demographers Murray Feshbach and Nicholas Eberstadt.

Cassandra was always right and no one ever believed her. If Cassandra had access to the NYSE, she could have retired rich, but she will not alter the consensus, which she will always oppose. But, the CIA or any similar bureaucracy will always spit her out (“not a team player”).

There you have it, a problem, not a solution. A contradiction in terms, a logical impossibility.

Lawrance George Lux writes:

Boonton,
You may be right, but I was thinking more on the lines of Decison series. I think the initial temporary decision may be correct, but each following decision will have significantly less probability of success. I have not read the book. Does it deal with the secondary fallibility issue. lgl

Steven McMullen writes:

Some corporate decision-making would not be appropriate for prediction markets, but some sorts of decisions work very well with markets. As an example it is worth checking out the work of Ely Dahan and John R. Hauser, who did experiments with prediction markets as an alternative to traditional market research in the area of product development. They found that they could get similar results with a small experimental market for potential sets of features and goods as they received with larger, much more expensive polls and surveys.

There will always be the possibility that people will try to manipulate the market results to bring about a certain end, but if the markets are controlled in an experiment setting this can be avoided. With larger markets which are open to the public you get the advantage of more expert input (potentially) and the draw of material rewards should theoretically create an incentive for those who know the most to put up the cash. In this setting, manipulation can be avoided only by making sure the market has enough trading volume to discourage any attempts to manipulate, or as with the Iowa Electronic Markets, ensuring that there is roughly equal incentive for manipulation on all sides, thus creating some sort of balance.

Barry Ritholtz writes:

If you take a detailed glimpse at the so-called prime data source, we can see how foolish this methodology can be:

The Iowa Futures markets, for example, has been used to predict both the elections and the stock market. The WSJ even had an op-ed stating that as the challenger’s chances for election improves, the markets then weaken.

Put this into context: The total dollar amount invested in this market is at present a grand total of $15,800.

Fifteen thousand, eight hundred dollars being played with primarily by a group of under-graduate and graduate students at Iowa University. From this “prediction market,” the author deduces the outcome of the $7 trillion capital markets in the United States.

If you want additional insight about the prognosticating accumen of the prediction markets, consider this last detail: The Iowa Futures had predicted a Howard Dean landslide in the Iowa primary. You may recall that he got clobbered.

Any other questions?

http://bigpicture.typepad.com/comments/2004/01/iowa_and_predec.html http://online.wsj.com/article/0,,SB109217833883588077,00.html
http://www.thestreet.com/p/_rms/rmoney/barryritholtz/10174814.html

Bibliophile writes:

Just two quick comments.
The futures prediction only works if there is diversity in the crowd. If the crowd is too heterogenous, the reasoning becomes group think rather than collected wisdom. This is illustrated by the success of rampant targeted marketing and fads that sweep through every few months.

I did not understand anyone behind this model, inculding Sorwiecki in regards to social issues, as saying that they are perfect. There will be instances where they will get it wrong. I do not remember anyone predicting that Dean would not get Iowa in the primaries.

I completely agree with Robert, this model will help broaden CIA intelligence but cannot completely help the problem.

Comments for this entry have been closed
Return to top