Scott Sumner  

Whereof One Cannot Speak, Thereof One Must Be Silent

PRINT
For Individual Liberty, Size D... For a Free Market in Plasma...

We talk too much. I probably talk way too much. Humans like to explain everything, even things that cannot be explained.

Over at TheMoneyIllusion I did a post trying to rebut the bubble view of NASDAQ, circa 1999-2000. Lots of people bought tech stocks, mostly during periods of time when NASDAQ was around 3500 to 4000. (It briefly spiked to just over 5000.) Now it's over 7200. Yes, that's not a good rate of return for a period of 18 years, but it's not obviously horrible (especially if you include reinvested dividends, and especially for those who didn't buy at the absolute peak, which lasted very briefly.) In the comment section, Matthew Waters asked:

Saying NASDAQ had an OK value in 2000 brings up the same question as the 1987 crash: how could both the 2000 and 2002 NASDAQ prices be efficient? For that matter, how could both 2002 and 2003 NASDAQ prices be efficient? End of 2002: NASDAQ was at $1,300. End of 2003: NASDAQ was at $2,000.

Maybe bona fide news on future cash flows accounted for a 75% drop followed by a 50% increase within three years. I doubt it. For its part, Bitcoin has such swings in WEEKS rather than years.

Is there anything that could falsify the EMH? It should be dramatically reformulated from "prices reflect all available information." It would be far stronger and more predictive to say "it's very difficult to beat the market on a 6 month or 1 year timeline."


This is a very good question, and one that reaches into the field of epistemology. What can we know about the world? Which I would rephrase as "What information is useful"?

To make my views easier to see, let's start with an analogy. Suppose I have been playing a certain roulette wheel in Vegas and believe it is tilted toward the red numbers, with relatively few blacks showing up. So the casino offers a test, 10 spins of the wheel. In this test, the number of reds and blacks is pretty even, but I notice another interesting pattern in the numbers:

15-12-33-17-15-3-24-36-30-9

Notice anything odd? Nine of the ten numbers fall in the right side column of the number grid on a roulette table:

Screen Shot 2018-01-16 at 12.33.06 PM.png
How likely is that!

I claim the roulette wheel is fixed, biased towards numbers divisible by three. In fact, this pattern is no more unlikely than any other pattern. Weird things happen all the time in casinos. Indeed the first time I ever walked into a casino (Surfer's Paradise, Queensland, 1991) I won my first 12 hands of blackjack, before losing the 13th. How likely is that!

I hope you see the problem here. It's not kosher to ex post make up a theory to fit the data. (I'm not accusing Matthew of that---we'll get to his excellent question later.) Yes, much of social science is done exactly this way, but that doesn't make it right. Indeed data mining (aka P-hacking) helps to explain why people don't believe social science research, unless they already found the hypothesis to be plausible before being presented with the regression results.

Matthew is right that the EMH doesn't do a good job of explaining the 1987 stock market crash, or the 2000-02 tech stock crash. It's hard to find fundamentals that would justify such a dramatic shift in prices over a short period of time. (Actually much harder for the 1987 crash than the tech stock declines, which took considerably longer.) So how do I defend the EMH? Two points:

1. The EMH is very useful to me in all sorts of ways. It's also consistent with a lot of research on the wisdom of crowds, and basic economic ideas such as competitive rates of return in competitive markets with free entry. It's got a lot going for it. Because of the EMH, I've invested in index funds, and also engaged in buy and hold of stocks (not day trading). I ignored Shiller's 2011 comments on overvalued stocks. My 401k has done very well as a result. It also helped me during my research on the Great Depression, when I found that market responses to policy shocks were much more perceptive that expert opinion, even the expert opinion of Friedman and Schwartz.

2. The EMH cannot explain certain puzzling facts. (Matthews right about that). And on these points we should just keep our mouths shut.

But people cannot leave well enough alone, they want to explain everything. So they don't keep their mouths shut; they develop alternative anti-EMH theories, such as the bubble theory of asset prices. And this is where they get into trouble.

I have many posts that talk about the way that cognitive illusions bias people toward believing that bubbles exist. One of my themes is that the debate over the existence of bubbles is meaningless, unless it has useful information. And (as we will see) it does not. If not useful, a bubble theory is just a sort of insult directed at the market, calling the market "irrational."

Now bubble theories might be useful. For instance, if bubbles exist and can be spotted in real time, it could provide useful investment advice. Or point to the need for regulation. But if not useful, then they are pretty meaningless.

If you are going to claim that NASDAQ was "obviously" wildly overpriced in 1999 and 2000, you had better be confident of that claim. If 18 years later it no longer looks so obvious, then you can't say, "well then NASDAQ was obviously wildly underpriced in 2002, and so the EMH is still wrong". "It was a negative bubble." The claim back in 2002 was that the whole world was obviously crazy in 2000, and that a mania had taken hold. The view was that now (in 2002) we had come to our senses, and the NASDAQ was back to appropriate levels. So are we now to believe that in 2018 we now know that people were actually rational in 2000, and that a wild mania of depression had obviously gripped the country in 2002, causing tech stocks like Amazon and Apple to be wildly undervalued? If all of this is so obvious, why do we keep having to change the story? When will we reach a point where we can look at the world dispassionately? (I say never.)

What you should say is that moves in the stock market are often larger than we'd expect, based on our knowledge of the fundamentals. We simply don't know why that is the case, and bubble theories don't help. Perhaps the value of stocks to the public is highly sensitive to things that we don't understand very well.

Robert Shiller has one of the best anti-EMH bubble theories. He looks at historical patterns for things like P/E ratios, and then correlates that data with the performance of the stock market over the next few decades. Unfortunately, while his theory looks good on paper, apparently it is not very useful, as he seems to frequently give questionable investment calls. (I.e. calling stocks overvalued in 1996 and 2011.)

But I actually have a lot of sympathy for Shiller. When I looked at the market in 2011, it wasn't obvious to me that it was over or undervalued. And as I look at the market today, at a dramatically higher level, it's still not obvious to me as to whether it is over or undervalued. For some reason, it seems extremely hard (for me) to figure out what stocks should be worth. I wish I could explain why, but I can't. (And don't get me started on bitcoin, which is even more inscrutable).

When looking at investment puzzles, we are faced with two choices:

1. Develop anti-EMH theories of anomalies such as bubbles.
2. Keep silent, and acknowledge our ignorance.

It's more useful to keep silent---saves wear and tear on the vocal cords.

PS. The EMH can be "falsified", i.e. found not useful, when anti-EMH theories are found to be useful. For instance, if mutual funds that are based on bubble theories fairly consistently outperform index funds, that would falsify the EMH.

The EMH cannot be "falsified" by engaging in data mining. Finance profs make the mistake of looking for market patterns inconsistent with the EMH, when they ought to be looking for evidence that others have found market patterns that are inconsistent with the EMH.

In another post I used the following analogy. If I were looking for evidence that lead could be turned into gold at low cost, I would not study theories of alchemy, I'd look for evidence (in global gold output data) that someone else had recently made the breakthrough.


Comments and Sharing






COMMENTS (21 to date)
Mark Bahner writes:
I won my first 12 hands of blackjack, before losing the 13th.

It's almost certain. Everyone knows 13 is an unlucky number. ;-)

Dikran Karagueuzian writes:

Scott:

Do you consider the success of Jim Simons and Cliff Asness to be "evidence that others have found market patterns that are inconsistent with the EMH"?

I'm sympathetic to the EMH generally, and think nonbelievers should be busy taking free money off the table, rather than arguing the point. But it seems that Simons and Asness are doing just that.

Mark writes:

"The EMH cannot be "falsified" by engaging in data mining."
Actually this isn't really true.

Here I can speak from experience. Suppose I want to determine whether some patter of gene expression correlates with cancer. I get a bunch of patient data and I apply some machine learning tool to find expression patterns that correlate with cancer. Buy, like i your analogy, even (especially) my super machine learning tool will find a pattern even if there isn't an underlying one, because there are really a nearly infinite number of possible 'interesting' patterns that could occur in the data by chance. If I try to assert that whatever patter I do find is real, I've engaged in what scientists call a fishing expedition.

So is it impossible to falsify a hypothesis that a given pattern of gene expression (or no pattern at all) correlates with or predicts cancer? No, it is posible. I do it by taking a patient data set that I have not yet analyzed, let's say 500 patients with paired samples of cancer and normal cells, and I split it randomly into two groups of 250 patients 'training set' and a 'test set.' I apply my machine learning tool (or any algorithm supposed to find patterns) to the training set. Then, I take the pattern of expression found to correlate with cancer, and I test the model on the test set. If the pattern is due to an underlying relationship between gene expression and cancer, the imputed pattern will prove to be good at differentiating cancer from normal cells; if the pattern isn't very good at differentiating them, then that is evidence that the pattern I imputed from the training set was just due to chance.

The same method can be applied to economic data. You could split up the training set from the test set by period, country, or industry. Now, you may say there are too many confounding variables, too few data points, or too much noise. But it is theoretically testable; and the problem isn't epistemological in nature; it's a problem of not having good enough data.

And of course, ceteris paribus, you can apply whatever pattern you've imputed retrospectively on new data going forward. It seems this is exactly what Schiller has tried to do. EMH is indeed falsifiable; Schiller tried to falsify it; he failed; so, for now, we reject the anti-EMH hypothesis. Just like how it works in other sciences.

" Finance profs make the mistake of looking for market patterns inconsistent with the EMH, when they ought to be looking for evidence that others have found market patterns that are inconsistent with the EMH."
This is obviously false; in order for others to have found those market patterns, someone else has to have ignored your advice and looked for mistakes rather than assuming someone else has already found them. And the "someone almost certainly is smarter than me" argument fails as well. Lots of people are smarter than me, but only maybe a few hundred work on the exact same topic as me; only a few apply the same types of methods to the problem. Academia is so highly specialized that for a given academic, it isn't unlikely that, at least in some arrow area that might have important implication for a broader hypothesis, there are only a few or a few dozen people that have about the same or superior understanding of that particular phenomenon. Consensus is fallible in part because (per Hayek's knowledge problem) expertise is highly dispersed, and it's likely that 99% of the people whose opinions make up consensus only vaguely (if at all) know what they're talking about when it comes to that particular issue. E.g., I don't care a whole lot about doctors' opinions in general on what causes cancer. I care a little bit about what oncologists in general think about what causes cancer. There are a dozen or so researchers on the cutting edge of research about what causes cancer that, imo, outweighs the consensus of doctors in general or even oncologists in general.

Matthew Waters writes:

Thanks for the response. FWIW, I do worry constantly about ex post data snooping.

I do think a good bit of ex ante value investing logic holds up. The "Intelligent Investor" gives a portfolio weighed between bonds and stocks based on their interest and earnings yields, way before Shiller looked at the P/E correlations.

Yeah, this argument has been hashed many times. I find some agreement with weak EMH compared to, say, technical or momentum trading.

I also recommend regular investors to do passive index ETF's. The contrarian value investing requires a lot of stubbornness. Many people would be likely to, say, sell NASDAQ at $3,500, buy at $5,000, sell at $2,000.

I know you've said "well, if you say invest in index funds, what point do anti-EMH theories have?" For one thing, I'm really interested in markets and economics for sake of knowledge. I'm interested in the seemingly durable outperformance of value vs. growth stocks, even though most people wouldn't like the swings of value stocks. And, well, I do think pushing against the grain of interest rates vs. P/E can make more money.

Kevin L Guo writes:

Scott: I'm a PhD student in finance. Predictable stock prices don't nullify the EMH. You would need the predictor variable to be something that has no connections to a rational risk premia. The EMH interpretation of Shiller's CAPE ratios is that risk-premia vary with respect to time (see cochrane "Dog that didn't bark") because dividends are not predictable. RP are higher in recessions and lower at the end of a long expansion. If someone tried to invest according to Shiller's advice, he would need to empty his bank accounts and buy stocks in a middle of a recession when unemployment risk is at an all time high. His advice is doubly not useful: not only is market timing less effective than indexing, but timing requires you to buy according to psychologically and financially unrealistic rules.

For example time series momentum is considered one of the things EMH can't explain, since it is based totally on past prices with no regard to any fundamentals. Asness has made a living investing in momentum. On the other hand momentum (and maybe low volatility) are the only "anomolies" that have robustly stood the test of time against data-mining. I'm willing to bet that if you invested in a bunch of "anomoly" stocks, you'd be worse off than just a small cap value index over the next decade.

Scott Sumner writes:

Dikran, No, for the same reason that my winning 12 hands in a row in blackjack mean nothing. Tests need to be systematic. Do anti-EMH mutual funds outperform index funds?

Or, are excess returns positively correlated to any significant extent? And I mean overall, not for any single person. My excess returns in blackjack were positively correlated in that Queensland casino.

Mark, I agree that out of sample tests are better than nothing, but they do not completely solve the data mining problem.

I'm not convinced that finance profs have any special expertise in finding good investment opportunities.

You said:

"This is obviously false; in order for others to have found those market patterns, someone else has to have ignored your advice and looked for mistakes rather than assuming someone else has already found them."

I agree that market traders need to ignore my advice for the EMH to be true, but I don't see why that makes my claim false.

Kevin, Good points, but let me say that I personally don't have any problem with going all in on stocks at the bottom of a recession. The risk doesn't bother me at all. If I could time the market, I would do so.

Mark writes:

"Mark, I agree that out of sample tests are better than nothing, but they do not completely solve the data mining problem."
But we're going to keep getting data continuously forever. Each new data point contributes to our confidence in the validity or invalidity of the model originally posited.

The problem still seems practical rather than theoretical: that new factors are continuously being introduced in economies, confounding things and ruining our 'ceteris paribus', and the time scale is very long. But Given many centuries and no central banks, governments, and technological upheavals to alter things mid-experiment, I think the problem would be solvable.

"I agree that market traders need to ignore my advice for the EMH to be true, but I don't see why that makes my claim false."
Perhaps paradoxical would be the better word. Like the proverbial joke about the two economists walking down a street, one says there's a $20 bill on the ground, and the other says, without looking down, 'no there isn't; if there were someone would've already picked it up."

Paradox may not be the right word. What do you call a proposition whose truth depends on people not believing it?

Matthew Waters writes:

AQR's returns don't seem too impressive.

https://www.ifa.com/articles/deeper_look_performance/

The Momentum strategy seems to kill any investor except for non-profits due to the short-term cap gains. Using the after-tax 5-year return of AQR's large-cap momentum fund, an investor would make 78%. VOO's after-tax return is 101%.

Morningstar AMOMX

Morningstar VOO

I don't want to speak too ill of the guy. By my calculation, the fund DID beat the market before expenses and short-term gains.

But I found his paper here had too much hand-waving and appeal to data sets, rather than his own funds. I'm suspicious of these high-frequency data sets, especially with small cap stocks, as your own buying and selling affects the bid and ask prices (in addition to the trading expenses and taxes).

https://www.aqr.com/library/journal-articles/fact-fiction-and-momentum-investing

Russ Abbott writes:

What do you think of work like this: Huber, T. A., & Sornette, D. Can there be a physics of financial markets? Eur. Phys. J. Spec. Top. (2016) 225: 3187

We examine the argument that, given the reflexivity of financial markets there is a fundamental difference between social and physical systems. We show that econophysics provides the methods, concepts, and tools to scientifically account for reflexivity.
Sornette claims to be able to recognize bubbles as they occur.
Todd Ramsey writes:

The EMH explains large rapid changes in prices when one considers that markets price in not only the information available to all participants, but also the collective emotions of all participants.

People are prone to herd instincts and tend to adopt the fear or greed emotions of the herd. Compared to changes in fundamentals, changes in emotions happen quite rapidly.

This theory explains why an investor like Buffett can have continued success, even in an EMH world, by insulating himself from the emotional swings of market participants. In fact, Buffett made a conscious decision to locate in Omaha to help prevent being swept up in the emotion of the crowd.

Taking this logic a step further, one can postulate that fear is a more powerful emotion than greed, based upon the observation that the largest daily declines are greater than the largest daily advances.

Andrew_FL writes:
If you are going to claim that NASDAQ was "obviously" wildly overpriced in 1999 and 2000, you had better be confident of that claim. If 18 years later it no longer looks so obvious, then you can't say, "well then NASDAQ was obviously wildly underpriced in 2002, and so the EMH is still wrong". "It was a negative bubble." The claim back in 2002 was that the whole world was obviously crazy in 2000, and that a mania had taken hold. The view was that now (in 2002) we had come to our senses, and the NASDAQ was back to appropriate levels. So are we now to believe that in 2018 we now know that people were actually rational in 2000, and that a wild mania of depression had obviously gripped the country in 2002, causing tech stocks like Amazon and Apple to be wildly undervalued? If all of this is so obvious, why do we keep having to change the story? When will we reach a point where we can look at the world dispassionately? (I say never.)

This is a very strange line of argument. Why do you appparently believe that in order for there to have been a bubble in the past, stock prices must never rise above bubble levels again, even decades later? What could stock prices in 2018 that reflect the situation in 2018 as it is presently understood, possibly tell us about the situation in 2000? What's the statute of limitations on this argument? Twenty years? Fifty? Five billion?

Alan Goldhammer writes:

Excellent column and well thought out comments. I don't know if the EMH can be falsified or not; it's largely dependent on what type of data set one looks at. There certainly are counter examples that show one can do very well as an investor just as there are examples that show one can lose a lot of money quickly (witness the big run up and now deflation of Bitcoin).

IMO, the NASDAQ composite is a poor choice because of the large number of companies that have little or no earnings record. This was certainly true of the Internet bubble and has also be true in the biotechnology sector since it's beginning in the middle 1970s. I remember hearing presentations by investment analysts trying to justify valuations for biotech companies that had no product, just some ideas on the table. Most of those companies are no more and the amount of investor loss is quite significant.

I'm a Graham and Dodd acolyte and spend a significant amount of effort looking at balance sheets to figure whether something is a good investment or not. It's tough slogging but rewarding when the proper homework has been done.

Steve F writes:

As far as I can tell, "bubbles" are explained by change in information.

Garrett writes:
I'm a Graham and Dodd acolyte and spend a significant amount of effort looking at balance sheets to figure whether something is a good investment or not. It's tough slogging but rewarding when the proper homework has been done.

Have you outperformed after costs/taxes versus VOO (or VT if you invest internationally as well)?

James Pass writes:

Sumner wrote: "Because of the EMH, I've invested in index funds and also engaged in buy and hold of stocks."

Is there any value to holding stocks when there is a large shock to the financial system? (e.g. bubble collapse, national financial crisis, major terrorist attack that affects economy) In other words, are there circumstances in which a hold policy isn't rational?

I agree that patterns often emerge that are mere correlation, not causation. And I like the analogy of lead into gold and looking for evidence in global gold output data that someone had recently made the breakthrough.

Scott Sumner writes:

Andrew, You said:

"Why do you apparently believe that in order for there to have been a bubble in the past, stock prices must never rise above bubble levels again, even decades later?"

I don't believe that.

A technical point that people sometimes don't realize: the value of a company is (to a first approximation) the temporally discounted sum of all its future profits.

This implies that, with high growth rates, you may only need small differences in fundamentals to switch a company from being worth zero to being worth a few billion. You may only need your estimate of the growth rate of revenue to go up by 1 percentage point for the value of the company to shift several billion.

Todd Ramsey writes:

"This implies that, with high growth rates, you may only need small differences in fundamentals to switch a company from being worth zero to being worth a few billion."

If all price swings are explained by fundamentals, to validate your theory you must explain what fundamentals changed between October 16-19 1987 to discount the S&P 500, essentially a proxy for the entire US economy, by 20% in three days.

"If all price swings are explained by fundamentals, to validate your theory you must explain what fundamentals changed between October 16-19 1987 to discount the S&P 500, essentially a proxy for the entire US economy, by 20% in three days."

No, I don't.

I think the markets are smarter than any single individual. Trying to figure out what "it" is thinking is a fool's game (many try and fail). The market is also smarter than all of the individuals that compose it (who are often ignorant, but collectively they produce the best estimate of value that we, humans, can get).

I don't have to be able to explain why the market moved anymore than I would have to come up with Bg5 myself to be able to claim that Alpha Zero is the best chess player we have right now. Anyone who disagrees can prove me wrong by demonstrating an alternative method that consistently beats the market in the same way that it'd be easy to disprove that Alpha Zero is the best player: just beat it in a 24 match game and I'll be convinced.

I just claim that it's not prima facie absurd to imagine that there was indeed new information that shifted the best estimate of the fundamentals enough to move the S&P 500 by 20%. It implies a much smaller move in the fundamentals than the 20% number would seem to suggest. A 20% move in value.

The S&P 500 is good proxy for the whole US economy, but it's a small fraction of it. It's only ~20 trillion USD, just barely more than GDP per year.

James writes:

Scott,

You say "Finance profs make the mistake of looking for market patterns inconsistent with the EMH, when they ought to be looking for evidence that others have found market patterns that are inconsistent with the EMH."

No. Looking at the totality of the evidence on a question is not a mistake. If the EMH is a theory about markets, then it can be falsified by observations of markets.

But the type of evidence you insist on is available for anyone willing to look. Research documenting classical factors like value and momentum dates back to the 1960s at least. Those factors kept working in each decade for half a century out of sample up to the present day, even after being publicized in widely read journals.

Is this absolutely conclusive? No. EMH believers can (and generally do) move the goal posts rather than change their minds. I would suggest that each time you have to move the goal posts or exclude some class of evidence in order to defend some theory, you should scale down your confidence in that theory.

Mr. Econotarian writes:

A lot of people were just DEAD WRONG during the Internet tech stock boom. They had a very wrong idea about how much revenue could be rapidly generated from Internet commerce and media. It turns out that much, much later, a good deal of that revenue eventually turned up, but it was way past the expected pay-off point, and as a result many companies burnt through cash and went out of business.

The crowd was "unwise" in retrospect. But frankly, this was a completely new industry, markets that never before existed were being tapped, and no one could say for sure what was going to happen. A very few companies (like Google) really did clean up over time (I know someone who is a billionaire thanks to a ~$100,000 early investment in Google).

But I certainly was wrong to make the investment I made then! About $50,000 and a few years of my earning life down the tubes, but I learned a lot about business...

Comments for this entry have been closed
Return to top