Bryan Caplan  

Where Superforecasters Start

PRINT
Bernanke's memoir: First impr... First-Order Effects Are First...
How can you improve your forecasting?  One of Superforecasting's most useful principles is "outside first."  Suppose you have a lot of details about the Renzetti family, and you're trying to guess whether it owns a pet.  It's tempting to dwell on the details, but:
The first thing [superforecasters] would do is find out what percentage of American households own a pet.

Statisticians call that the base rate - how common something is within a broader class.  Daniel Kahneman has a much more evocative visual term for it.  He calls it the "outside view" - in contrast to the "inside view" which is the specifics of a particular case.  A few minutes with Google tells me about 62% of American households own pets.  That's the outside view here... Then I will turn to the inside view - all those details about the Renzettis - and use them to adjust that initial 62% up or down.

It's natural to be drawn to the inside view.  It's usually concrete and filled with engaging detail we can use to craft a story about what's going on.  The outside view is typically abstract, bare, and doesn't lend itself readily to storytelling.
As far as I recall, I've yet to lose a bet I publicly blogged.  What's my secret?  I doggedly take the outside view.  When long-run trends say X, and the "latest news" says Y, I go with X.  When Democrats won big in 2008, I saw good luck, not a new political regime.  That's why, in 2009, I bet my former co-blogger Arnold Kling that the Republicans would regain control of one branch of the federal government by 2017.  I won in 2010.  During the Great Recession, I saw a bad business cycle, not a "new normal."  That why, in 2013, I bet Tyler Cowen that unemployment will fall below 5% by 2033.  I am very close to winning already.  When anyone forecasts any specific major disaster, I scoff.  Disasters do happen, but almost all specific predictions of major disasters do not come to pass.  And that's why I am eager to bet against such predictions

I sometimes joke that "news melts your brain."  What I really mean is that news amplifies our misguided impulse to take the inside view of things.  To be a superforecaster, of course, you have to use the news to tweak the outside view.  But most people would be better forecasters if they ditched the news and relied on base rates alone.




COMMENTS (10 to date)
Kevin Erdmann writes:

I can't believe Tyler made that bet with you, especially at those odds, in late 2013. If nothing else, demographics are highly in your favor. Workers at baby boomer ages have low unemployment rates.

Hilariously, the commenters on that post seem unanimously and strongly in favor of Tyler, except for the ones with some conspiracy theory about the data.

Nathan W writes:

In the tournament, I was able to score lots of big gains in areas where I knew others would suffer from prejudice or lack of knowledge (and having more personal knowledge of a given thing.)

For example, relating to China, Westerners very often assume the very worst of the government, and I was able to secure large gains from making informed bets on this (e.g., probability of violent military confrontation in the South China Sea in the next month or two - not 20%, more like 2% - it does happen from time to time). In another case, a lot of people attributed very high probability (30%) that the Shanghai Free Trade Zone would allow Facebook, but I know that Chinese officials regard Facebook as a potential locus of organizing a revolution, and hate the idea of US officials having access to extensive data on the social networks and blogging habits of Chinese citizens, and the chance they would do so is essentially 0%.

With respect to questions on the Middle East, I was able to score big gains against assumptions that Iran is always lying and can never be trusted and will never live up to their promises (granted, I don't trust them all the much) or about the inherent stupidity or tyranny of several regimes (on the matter of Syrian refugees, I knew it would be quite high because I'd met a very terrified man in Istanbul who had fled Aleppo in the first week of the conflict). In several other cases, the questions related to hypothetical specific loci of conflicts which had 0-1% probability because the hypothetical aggressor had essentially 0% chance to play a winning hand in that conflict (e.g., Syria attacks Israel, Iran blocks the Straight of Hormuz).

About Africa questions, people always seemed to bet heavily on the worst possible outcome, so lots of gains could be secured there.

A lot of other gains were easy to come by in cases of very low probability events, which people ascribed higher probability. E.g., probability of a war starting with the very short time period of just a few months (often pegged in the 10-20%, which is ridiculously high for such a short timeframe, so lots of gains in betting at 1-3% odds) or of a long term civil war coming to an end despite no credible peace talks (often pegged at 10-20%, whereas without credible peace talks this is more like 0-1% in the space of the following months). Or, the probability that some pandemic would set in (often betting at 10% within just a month or two, whereas historically this is less than 1%).

Other easy gains could be made on the backs of climate skeptics, who bet heavily against higher temperatures or lower arctic ice coverage (admitted, given natural variation, it's not sensible to bet strongly on these).

In questions about currency and stock markets, the easiest gains were not so much about knowing anything ... just check a day or two before closing and often people had not updated their expectations, so you could make gains without much long-term liquidity loss.

I don't really like the new format (I think the 3rd year was the best format), and also now it's a for-profit venture not an academic project, so I rarely participate anymore.

Wrong Species writes:

I think watching the news is better than you realize. Sure, if you don't pay much attention to the world and then one day see something about the stock market crashing you might naively assume that a recession is imminent. But if you frequently watch the news you realize that people make these kind of predictions all the time and you learn to ignore the doomsayers.

And it can also help you resist opportunistic political narratives. Right now, people on the left are outright ignoring the fact that the budget cuts in 2013 did not lead to recession(even though they claimed it would). And yet, many also claim that austerity is this great evil and needs to be stopped and they point to Europe as an example supporting their claims. History is a collection of selective facts that people use to support their story. The news has this problem too but paying attention allows you to more thoroughly see the details behind the narratives.

Phil writes:

"Did Tyler Cowen ever write out a rational for why he made that bet?

I get that they're generally in line with the premise of his book

but the whole thing seems absurd, the odds and timeline seems ridiculous

as do the stakes, which seem to override everything else, seems like $10 is a pretty light tax"

Floccina writes:

Since often both sides predict doom if their policy is not enacted I often find myself thinking both sides are wrong.

For example I doubt that atmospheric CO2 will be anywhere near as harmful most people concerned about it think, but I also think it would cheaper to get to zero net CO2 going into the air than most skeptics think (Assuming a CO2 tax with most of the proceeds paid out to people who remove CO2 from the atmosphere.) Pessimistic bias?

Pessimistic bias is very interesting to me because in their personal lives people often seem overly optimistic.

Daublin writes:

Floccina, can you show your work on getting to zero net CO2 being inexpensive? As one point that is influential on me, the Kyoto Protocol is widely assumed to be economically devastating to the U.S. had it been adopted, and yet is far short of zero net CO2 even for the U.S. alone.

Kudos for bringing the issue up and all. To my view it should be front and center.

If we look at the IPCC's actual estimates for the Social Cost of Carbon, we see it's "31–121 US$/tCO2" at the high end. That means a tax of $1 per gallon at most. This is a livable (but painful) amount.

It will also be temporary because a carbon tax that high will make various forms of nuclear energy both politically and economically unbeatable.

The real problem is that the problem will be used as an excuse for other regulations.

Kyoto is, of course, irrelevant because nobody pays any attention to it.

Nathan W writes:

Daublin

"can you show your work on getting to zero net CO2 being inexpensive"

Energy intensity in the economy is fair low compared to historical standards, and getting lower every year. Now, energy represents just a few percent of production costs in most businesses.

Achieving a carbon-free economy in 10 years time might double the cost of energy (sorry, too lazy to show my work), but that represents only a few percent change in GDP. Moreover, that's assuming that the policies required to engineer this change to a carbon neutral economy did not result in significant technological gains in energy efficiency and production.

Would it have been a disaster for 2014 if the economy in 2014 were the same real output as 2013? There would be costs, to be sure, but hardly disastrous.

I suggest that the alarmists on the one side of the story are nearly as unbelievable as the alarmists on the other side of the story.

I recall being at an energy conference, and listening to the representative of the International Energy Agency discuss the costs of change. He presented all figures in nominal terms. But some quick mental calculations reveal that this represented a level of annual investment equivalent to 1% of global GDP.

With global GDP growth in the 3-5% range, would 1% of global GDP dedicated to carbon free energy production be disastrous?

Here's the real problem. Energy expenditures are relatively inelastic. So a carbon tax will almost entirely come out of the pockets of fossil fuel industry shareholders. This creates a very obvious constituency to resist change.

I would have no objection to a carbon tax combined with a policy of getting out of the way of nuclear energy and repealing laws mandating CFLs, low-flow toilets, etc.

Fred Anderson writes:

@Nathan W.
You say, "Energy expenditures are relatively inelastic. So a carbon tax will almost entirely come out of the pockets of fossil fuel industry shareholders."

I thought energy demand was relatively price inelastic (e.g., Bernstein & Griffin, 2006). Your statement that energy expenditures are inelastic implies to me that, as prices rise, quantities demanded must fall correspondingly to keep total expenditures constant in the face of a price increase. Additionally, I should think that price inelasticity would imply that suppliers have pricing power; that is, they need not fear much loss in volume if they raise prices. Consequently, I should think fossil fuel suppliers would have little inhibition about passing a carbon tax on to consumers -- the opposite of your conclusion. (Consumers might try to shift to nuclear & hydro, now tax-advantaged forms of energy. But neither of these can be expanded enough to replace the approx. two-thirds of our energy supplies that are carbon based. So consumers must either drastically lower their consumption, or they get forced back to newly taxed fossil fuels.)

I am not trained as an economist. So what am I getting wrong?

Comments for this entry have been closed
Return to top