Scott Sumner  

Warren Buffett, out of sample

Wishful Thinking and Unintende... Larry Summers's Anti-Trump Cas...

In a recent post I discussed a couple of arguments against the Efficient Market Hypothesis, which I used to hear a lot when I first started blogging in early 2009. One had to do with the outperformance of hedge funds, and the other was the so-called "inevitable" housing crash in the US (which never occurred in lots of other "bubble" economies, and is reflating again in America.)

Another argument was that people like Warren Buffett would persistently outperform the market. This was supposed to violate the EMH. It doesn't---the EMH predicts that out of every 50 million stock investors, about 49 will beat the market in 20 consecutive years. Buffet never came close to that, but he had some years with huge outperformance, and thus his overall record is amazing. But did it violate the EMH? Maybe, but I was skeptical. In economics we often do "out of sample tests". That is, when someone published a theory years ago, we return to the data to see how it held up using more recent data.

This article
lists the annual return for Warren Buffett's Berkshire Hathaway, ever since 1965, as well as the annual return on the S&P 500. Over the past 51 years, Buffett averaged a 20.8% rate of return while the S&P 500 only averaged 9.7%. But how about since the beginning of 2009, when I started blogging?

Warren Buffett: 10.8%

S&P 500: 14.8%

It's quite possible I've made an error here, either in my computation, or in my interpretation of the article. A footnote suggests that because of tax factors, the Buffett return would have been lower than the S&P 500, even if he had done equally well in picking stocks. But I also recall that his insurance company gave him some leverage. If any commenter can help me out here I'd appreciate it. My guess is that any mistakes would make the figures more equal, but not affect my qualitative claim that Buffett has not significantly outperformed since early 2009.

PS. Buffett did make one good investment in 2009, the Burlington Northern Railroad, which has benefited hugely from President Obama's decision not to approve the Keystone Pipeline. Buffett and Obama have been friends going all the way back to when he was a Senator from Illinois. I actually don't think there was any corruption here, either in terms of inside information on the decision, or Obama making the decision to benefit Buffett. But I also think this shows the danger of drawing inferences about corruption. Most people know that Buffett lives very modestly and plans to give away most of his wealth and also favors higher taxes on the rich. So he's trusted, and I think justifiably so. But imagine someone with a poor reputation in the same situation---what would people think?

Comments and Sharing

COMMENTS (29 to date)
CMOT writes:

Buffet is not proof or disproof of the EMH because he is outside of the EMH.

EMH assumes that investors can only sell or buy securities but otherwise have no influence over the market or the companies in it.

Not only does he have personal connections to policy makers and regulators, as noted above, he controls many of his investments. So he's as much a manager as an investor.

Even where he holds only a small share of ownership, he has influence - if Warren Buffet makes suggestions to management, they'll listen.

What's both illustrative and troubling is that while Buffet made his fortune in the past by betting on the success of the American middle class, today he seems much more interested in investing in politicians than the middle class.

Charlie writes:

Some research has suggested that Buffett took advantage of factors like value outperforming growth, quality outperforming junk, and low volatility outperforming high volatility. A lot of EMH defenders like Cliff Asness and Fama consider these factors to be consistent with EMH. Once you adjust for these and Buffett's leverage, his outperformance is not so stark.

How Warren Buffett beats the market

John Hall writes:

Keep in mind that a lot of Buffett's outperformance was when his AUM was much smaller. He could buy stocks that no one else was paying attention to. EMH is about information. Large stocks that everyone is paying attention to should have fewer opportunities for stock picking than smaller illiquid companies. I don't consider it a contradiction that he outperformed then and underperformed now.

Njnnja writes:

While your example is illustrative, there are a million details that mean that you can't simply compare returns of an active investor versus an index and say that either proves or disproves EMH. EMH doesn't deny random chance, and out of millions of investors EMH would agree that large numbers of them will be very successful. Further, EMH only talks about how prices incorporate information, and conclusions about portfolio returns are ancillary (because portfolio returns are basically changes in prices). Things like leverage can make portfolio return comparisons better or worse and that would have no implication on the validity of EMH.

But in Buffett's case, there are 2 fundamental problems that make a comparison of Buffett's return to an index not an applicable test of EMH, even if you could adjust the numerical comparison to deal with the kind of issues mentioned above.

First, for a long period of time, Buffett has been able to take controlling, or at least influential, shares of entire companies. "Buy companies, not stocks." And while Buffett is not as active in the management of his owned companies as other investors are, there is no doubt that having Buffett own, say, 10% of your company, you are going to listen to and take advice from him and his people. Once control comes into play, EMH doesn't really apply.

Second, while I agree that there is quite unlikely to be any sort of corruption where Obama stopped Keystone because he wanted his pal Buffett to make a few extra dollars, it is possible that because Buffett has been able to speak to Obama and his advisors directly, Buffett may have known for a long time that the president's opposition to Keystone was much more firm than he let on. Such nonpublic knowledge is not covered by any but the strongest form of EMH, and not the one that is typically discussed when saying "you can't beat the market."

DF writes:

What relevance is your blogging career to constructing an out of sample period? His entire career is out of sample, so why does a recent period of 7 years provide a better estimator of his abilities than the entire period of 51 years?

Also, you should be measuring risk adjusted return, not total return. Otherwise you don't account for differences in leverage.

JK Brown writes:

It's been a long time since I read it, but I remember Buffet himself attributing a good deal of his early market beating success to his being out of the market for most of the 1970s.

BillB writes:

Take a look at the latest Buffett letter to shareholders:

See page 10 and the increase in insurance float, close to a 20% annual increase. Then note that the cost for such float has been negative!

There is Buffett's genius.

Garrett M writes:

To expand on what BillB pointed out, I recommend anyone interested to read the white paper "Buffett's Alpha" that AQR put out a couple years ago. Their conclusion was that Buffett's main value add came from selecting safe, high quality stocks and utilizing the free leverage that his insurance business provided.

Scott Sumner writes:

Am traveling today, will respond tomorrow--thanks for the comments.

Andrew_FL writes:

Yup calling for higher income taxes when A) The reason your tax rate is so low is because most of your taxes aren't the income tax, and B) you take an enormous charitable givings deduction (what do you know, what was one of the very first amendments put on the "Buffet rule"?...take a guess) sure is something deserving of trust.

He said sarcastically.

Corrupt? No. It's not corrupt to engage in perfectly legitimate gaming of government policy to favor you against others.

But it is worth of derision, and not respect.

SD000 writes:

[Comments removed pending confirmation of email address. Email the to request restoring this comment. A valid email address is required to post comments on EconLog and EconTalk.--Econlib Ed.]

Khodge writes:

Much like arbitrage, a market technician exploits weaknesses in the market. It is entirely conceivable that a technician exploited an inefficiency but, when it had been plugged, was not able to identify a similar weakness. Following just one individual would not necessarily disprove that a series of technicians could, separately, exploit inefficiencies.

A writes:

When other people make similar arguments about "bubbles", you ask them for time frames. But what was your model for predicting underperformance in the starting from 2009? Without that prediction forensic, it looks like you are just paying attention because recent information is validating your priors. That is closer to a recency bias than an out of sample discipline.

CW writes:

WRT the "leverage" issue.

Here's how I think of it: for most of the period, Buffett has had about 50 cents in float for each dollar of equity.

Pretend that the equity and the float both earn the S&P rate, and then compare the result to the rate earned by BRK.

The last time I did the calculation, the BRK return was about 19.40% per year, the float adjusted return on the S&P was about 14.85% per year, and the difference was BRK's float-adjusted alpha, about 4.55% per year.

James writes:


Are you sure it is the EMH that you believe? Whenever you write in defense of the EMH, the evidence you cite also supports the much less specific claim that returns are hard to forecast which could be true whether or not markets are also efficient.

Do you have any evidence which supports the specific claim that markets are informationally efficient, and couldn't be just as easily accounted for by the more general claim that forecasting returns is hard?

Joakim Book writes:

As noted above, most (if not all) of Buffets superior returns can be found in his superior funding:

It's not so much his ability as a stock-picker or investor, it's his incredibly low funding costs; if you can borrow at 3%-point lower than everyone else, and invest at average returns, you'll beat the average, of course.

Tim Worstall writes:

Two pieces that look to exactly this:

Philip George writes:

The period 2009 to 2015 is the rising part of a business cycle. It is when the tide goes out that the naked swimmers get exposed. Buffet stays with safe stocks. Over an entire business cycle he always wins.

Michael Savage writes:

Buffett's own point in this context is that it's not about him. It's that his *strategy* consistently outperformed - value investors consistently outperform other investors. I think there is evidence that value and momentum strategies do beat the market (a little bit). These anomalies have made a few people very rich, but neither is so profound as to invalidate the general usefulness of EMH.

There are a couple of subtleties to the leverage points too. First, leveraging up to buy low-risk stocks is better than buying risky, highly-leveraged stocks directly. Wouldn't an efficient market be neutral as to where the leverage lies? (Answer is partly regulatory - limits on fund managers leveraging up). Second, every insurer earns a float, but many screw up total returns by loss-making underwriting that wipes out investment profits. That's a different thing from exploiting market inefficiency, but some credit to Buffett here.

Scott Sumner writes:

Everyone, Thanks for the very helpful comments.

DF, You said:

"What relevance is your blogging career to constructing an out of sample period?"

When I started blogging I received many arguments against the EMH. Now I am trying to revisit each of those arguments, to see if they have had any useful implications. Did Buffett's success have any implication for future performance? Are housing bubbles predictable? Are there talented predictors of asset markets, or do people just get lucky? So far I am seeing no useful implications for anti-EMH theories, which suggests to me that they have little or no value.

Garrett and Bill, I do recall reading that. Are there barriers to other wealthy people creating insurance companies to take advantage of the free leverage? I'm not being skeptical, just wondering why others don't do it.

Andrew, He favors higher taxes on capital--isn't that something that would apply to him?

A, You said:

"But what was your model for predicting underperformance in the starting from 2009?"

I was not predicting underperformance. This exercise is not a scientific study, I am just revising all the arguments I faced in 2009, to see if any hold up. So far I don't see any, but it's possible I missed something.

CW, Thanks, that's helpful.

James. One can never prove there are no inefficiencies, as there are an infinite number of possible inefficiencies. But there are many studies supporting the EMH, which provide some evidence for market efficiency. At this point I'm more interested in the anti-EMH claims. Are there any useful anti-EMH models.

Thanks Tim, see my question to Garrett.

Michael, Value outperformed in the past, but is there any evidence that it will in the future? If the ball on the roulette wheel comes up red 16 times out of 20, is there any evidence that the red black split will be anything other than 50/50 going forward?

Michael Savage writes:

What's surprising to me is that value & momentum have endured so well *after* being identified. Looks like dollar bills on the street (& I say that from perspective of basically agreeing with EMH). It seems to pass the out-of-sample test, notwithstanding possible recent wobble.

Doug T writes:

To expand upon Philip George's comment above, every investment style goes out of favor, sometimes by a lot, for a period of time. Buffett's genius has been to stick with his style even when it underperforms. Because the market mean-reverts: what you lose on the front end, you make up in spades on the back side. Usually.

To that end, the last 5 years have been a lot like the late '90s--a time when Buffett and Munger were considered a dinosaur, out of it--mainly because he didn't "get" technology. Needless to say, the 2000-2002 bear market did wonders for their reputation. So I'm a little suspicious of your "out of sample" selection. It really looks more like recency bias, as "A" wrote.

Buffett has also proven resilient, within his core disciplines, learning as he goes along. He has a great deal of "market humility," for all the personality cult that surrounds him. He often leads off his investment letters by trumpeting his failures. That's something everyone could learn from.

Andrew_FL writes:

@Scott-Warren Buffet knows full well that advocacy for higher taxes "on the rich" means "higher top marginal income tax rates" to the average person, and it means that will be the actual policy. He also knows full well how to make sure he pays as little in taxes as possible whether capital taxes increase or not.

Warren Buffet is not sincere. If he was, he'd be and large Gifts to The United States to his tax payments every year already.

Jamie writes:

It seems his methods can be replicated with modern portfolio theory. Smart, but can be replicated. His lackluster recent performance can be a symptom of value stocks underperforming growth. Wouldn't declining risk premia disproportionately help growth stocks?'s%20Alpha%20-%20Frazzini,%20Kabiller%20and%20Pedersen.pdf

James writes:


Maybe you misunderstood. I'm not asking you to prove that there are no inefficiencies. I'm asking you why you claim markets are efficient when the only evidence you muster could just as easily be explained by a more parsimonious alternative.

In other words, you are committing a conjunction fallacy. Every time you talk about efficient markets, you reference facts that could be explained by more parsimonious hypotheses e.g. "Returns are hard to forecast," or less parsimonious hypotheses e.g. "Returns are hard to forecast because prices are tightly coupled with the fair values of underlying assets." Why are you not reasoning to the most probable explanation for the observed facts?

Since you asked, there are useful models of inefficiency. Here's one for free: Stocks that have not had positive earnings in the last four quarters tend to be overpriced. You can use this by investing in indexes that screen constituents for positive earnings such as the S&P products rather than purely cap weighted indexes.

On that note, it is wierd that you chose the S&P 500 to compare to Buffett given that the S&P indexes are constructed by a committee with an earnings screen. Indexes constructed in that way do not represent a market portfolio.

Toby writes:

Hi Scott,

Have you by any chance read the rebuttal of Warren Buffet entitled "The Superinvestors of Graham-and-Doddsville"? It's included with the book "The Intelligent Investor" I believe.

In addition, the out of sample test seems faulty for other reasons than the one you mentioned. The wealth Buffet has to manage is many times larger these days than it used to be when he "beat the market" so to say. This would bias the results against him as to achieve the same result many more unexploited profit opportunities would have to be sought.

Finally, I don't think Buffet's preferences make the Railroad/Pipeline situation any less shady. There are no doubt plenty of individuals with what one could consider good motives who have broken the rules to achieve their laudable goals.



ChrisA writes:

The problem with being "like Buffet" is not a lack of financial cleverness, but of psychology. It is very very hard to be a sensible long term investor in such a volatile market such as the stock market. Why Buffet is so wealthy has more to do with his reputation so that he is allowed to be more long term than most professional investors.

Cameron writes:

Buffett's secret sauce is to lose less in bad years, rather than make more in good years. So starting at 2009 will not capture what he does that's amazing.

Jose Romeu Robazzi writes:

Buffet is an "alpha" generator investor, he does not seek, as an investment objective, to beat the SP500 or any other index (although that may happen). What he seeks is to deliver a high return/risk profile, capturing positive idiosyncratic risk.

It does not matter is he is big relative to the market, as long as there exists large companies which he believes, based on proprietary research, that will deliver idiosyncratic positive returns.

Comments for this entry have been closed
Return to top