Scott Sumner  

How do we evaluate Robert Shiller's forecast?

PRINT
Godwin's Law in the Public Squ... Human Smuggling: What If Phila...

Nobel Laureate Robert Shiller recently made some comments about Bitcoin:

When it comes to economic bubbles, there is perhaps no single greater authority than Yale economics professor Robert Shiller. Shiller wrote a seminal book on speculation and its devolution into mania, called Irrational Exuberance. The book analyzes bubbles throughout centuries. Shiller has won the Nobel prize for his work in the area. Among his other contributions, he has created tools and indexes for measuring price levels and valuations. So when Shiller suggests that the bitcoin space might be a bubble, investors would be wise to take note. "Irrational Exuberance" and Bitcoin In an interview with Quartz, Shiller said that the "best example right now" of irrational exuberance is bitcoin.
Is it true that investors "would be wise to take note"? More importantly, how would we evaluate that claim? I don't think we as a society know how to evaluate these claims, and indeed I think we do so in the wrong way.

One solution is to simply wait until we see what happens. If the price falls sharply, then that would confirm Shiller's prediction. If it rises further, or even levels off for many years and decades, then he's wrong. But I see a couple problems with that common sense approach:

1. The returns are likely to be skewed. To take the most obvious example; it's much more likely that bitcoin will rise from $5000 to $15000, than fall from $5000 to negative $5000. Indeed there is a zero lower bound on bitcoin prices.

Now suppose that markets are efficient and bubbles do not exist. Also consider an asset with a very uncertain value, because its fundamentals are not well understood---like a new privately issued fiat electronic currency. Now go back to when the price of bitcoin was $30, and people were already calling it a bubble. Investors presumably knew there was some probability that bitcoin would catch on in a big way and soar much higher in value. They also knew that the price could never fall below zero. So it might go up to $1000 or more, but will definitely not fall more that $30. Sounds like a great investment, right? No, because if markets are efficient then the probability of it falling back toward zero must be many times higher than the probability of it rising up to $1000 or more. We just happen to live in a universe where a very improbable event occurred; its valued soared more than 100-fold (indeed more than 1000-fold if you go back even earlier in time.)

My point is that when people predict that bitcoin type investments are bubbles, they are usually going to appear to be correct in retrospect, even if bubbles do not exist. That's simply an implication of the skewness in the returns. A low probability of high upside, and a high probability of modest downside. So being "correct" actually doesn't prove very much.

2. A second problem is related to "data mining". To his credit, Shiller makes very public predictions that can be discovered using Google. Thus I quickly discovered a January 2014 article where Shiller claimed that Bitcoin was a bubble:

Shiller is convinced bitcoin is a bubble and he is bemused by the fascination surrounding digital currencies. He said he is amazed by how people are excited by bitcoin - and bear in mind that a man who won the Nobel Prize for his work in the field of behavioural probably isn't easy to surprise, let alone amaze. Shiller said:
"It is a bubble, there is no question about it. ... It's just an amazing example of a bubble."
It would be rather presumptuous to argue with a Nobel laureate, but luckily someone already did that. Back in December Forbes put Shiller's work to the test, comparing his findings with the bitcoin bubble.

Forbes contributor Tim Worstall argued that preventing a bubble from forming in the bitcoin market is not easy, since the market is not developed and lacks many tools needed to detect a bubble.

However, Worstall said it is "more than likely" that we are in a bitcoin bubble.


I don't have a big problem with Worstall's "more than likely" remark, given the skewness discussed earlier. But I do have a problem with Shiller's claim. At the time, bitcoin was at over $800. A year later it had fallen to $200. Good call, right? Not quite, because today it's over $4000. How can there be "no question about it", when three years later bitcoin at $800 looks like an absolute steal? Were investors "wise to take note" of Shiller's 2014 call? Not if the alternative was buy and hold until today.

To summarize, Shiller made a bubble call in January 2014. Most bubble calls for assets with highly asymmetric returns will look "correct" in retrospect, even if markets are 100% efficient and bubbles do not actually exist. And that's true of people who only make a single bubble call.

But Shiller is making repeated bubble calls, just as he talked about irrational exuberance in the stock market in both 1996 and 2000. So which prediction do we hold him accountable for? If Bitcoin falls in half over the next 12 months, does that prove his 2017 prediction correct, or does it show his 2014 call to be incorrect? (prices would still be far above 2014 levels.) How do we evaluate the record of bubble forecasters who keep making one bubble call after another, until the highly volatile asset finally has a big price drop?

Here's my claim. Bubble forecasts for highly volatile assets are not very useful, because of the reasons discussed above. These bubble calls do not represent good investment advice, and we have no reliable way to test them, even in retrospect. We do have some methods for testing for the existence of bubbles, but they need to be far more systematic. In fairness, Shiller has also done some of those more systematic tests, and found some intriguing results (such as what looks like excess volatility in US equity indices.) Even those tests can be challenged, but they are at least suggestive.

But these recent bitcoin bubble calls are virtually untestable. I don't believe bubbles exist, however I also think the probability that Bitcoin prices will be lower in 2020 than today is much greater than the probability that prices will be higher. But that doesn't mean it's a bad investment, because prices might be vastly higher. And if lower, that provides very little support for bubble theories, because that outcome is likely to occur even if bubbles don't exist.


Comments and Sharing






COMMENTS (9 to date)
Mark writes:

So, essentially, you're saying that even if the price of Bitcoin is more likely to go down than up, it's expected value (I mean the term in the mathematical sense) may still be higher than the current price, and therefore still a reasonable investment. Right?

It seems a big issue with bubble prediction (perhaps your hinting at this?) is the attribution of acyclical variation to booms and busts. There is a simple way to test this with Schiller's Bitcoin predictions: ask Schiller when, specifically, he thinks Bitcoin will fall (the bust). The next time Bitcoin goes down (maybe specify that it go down by at least a certain amount to call it a 'bust'), take the difference between the actual time of price decline and Schiller's predicted time, which we'll call delta. Then, looking at Bitcoin's historical price data, calculate the average time elapsed between significant price declines, and divide by two, and if Schiller's delta is significantly lower than that value, he did better than one would expect by chance.

Then have Dr. Schiller repeat this process over and over again a bunch of times and use all the 'deltas' to compute a nice p-value, if you wish.

David J writes:

Mark, I don't think that's a correct interpretation of Sumner's point. His thought experiment starts out by asking us to assume the EMH is true. That means that the expected return of an investment in Bitcoin at any given time is zero (ie, that it is fairly and accurately priced at all times given all available information). Therefore, it would not be a wise or foolish investment (maybe a marginally foolish assuming you have to pay to move in and out of Bitcoin). The point is that while expected return in this case is 0, the distribution of possible returns is not "normal". There is a long tail on the upside and an overall higher likelihood that the value will decrease in order to balance out the extremely high outlier return possibilities in the tail.

Brian Donohue writes:

Excellent post sir!

Michael Rulle writes:

I will take a circuitous route to agree with Scott on Shiller's view of Bitcoin.

It is very difficult to determine if a particular bubble exists or existed, even in retrospect. Shiller became well known for his prediction that Nasdaq was in a bubble in the late 90s. I also believed that it was obviously a bubble.

But when I first thought it was obvious, the market still increased by 100%. But I still believed it was a bubble and "predicted" that the Nasdaq would decline by 90% (in January 2000---it still went up 35% after that). It turns out it did decline by about that number. In fact, the 17-year total return of the NDX 100 (including dividends) has been virtually zero.

But I think that was a reasonable guess because of the implied future value the then current valuations implied. Absent GDP growth of 7-10% for 20 years those values were absurd. So, my point is that Shiller became famous for predicting the obvious (in my opinion). That is the only time I ever believed something was a bubble (even retrospectively) in my lifetime. And I was really wrong on timing. Shiller is obsessed with finding "bubbles". He sees them everywhere.

As it relates to Bitcoin, he believes something with certainty something we cannot know. Bitcoin is an attempt to create a currency so that it cannot be "manipulated" :-) by governments. We already have virtually eliminated cash, thus decreasing freedom. Bitcoin is an attempt to take back some of that freedom. We have no idea what its value may be. Governments hate it, but am not yet sure they can eliminate it. I do not see how one can have any opinion about the value of bitcoin, as the variables which will ultimately determine it are virtually unknowable. There is no valuation "model".

Cloud Yip writes:

I think his "prediction" is not really that important as they are some casual implications from his irrational exuberance/ narrative economics framework.

As you have commented earlier this year, this framework is far from complete and almost like a ad hoc theory that can "explain" most of the things around the world.

I don't mean to disrespect Prof.Shiller's work, and in fact I think these are right steps toward a better path, but right now, his model is really far from perfect and making predictions that is hard to justify...

Alan Goldhammer writes:

Irony of ironies. China halts trading in bitcoins. Price is down 27% since September 7. Is this the bursting of the bubble???

AbsoluteZero writes:

Someone wrote:
Irony of ironies. China halts trading in bitcoins. Price is down 27% since September 7. Is this the bursting of the bubble???

No. "China" did not "halt trading in bitcoins". One exchange, BTCC, voluntarily chose to halt trading. One reason is the recent announcement that initial coin offerings (ICOs) will not be allowed. This doesn't affect the launching and trading of digital currencies. There's also talk of regulating exchanges. What the linked Bloomberg piece said, that China will ban exchanges and trading of digital currencies, is speculation.

Turning that into "China halts trading in bitcoins" is not just misleading, it's irresponsible.

To put things into perspective, BTC has, depending on the period one examines, DAILY volatility of at least a few percent, and annual volatility of well over 100%. 27% is not the bursting of anything.

Jack PQ writes:

Indeed one needs a method to assess forecast accuracy, something like forecast calibration. How many predictions, for which specific horizons? And then compare with realizations. Shiller is sometimes right, but his case is like the economist who predicted ten of the last 5 recessions....

Lorenzo from Oz writes:

Fine post.

Perhaps folk should stop talking about "bubbles" and talk more about asset price instability and its causes?

At least the latter does not rely on claims about information not yet available.

Comments for this entry have been closed
Return to top