Arnold Kling  

Non-normal Distributions

PRINT
Robert Solow on Model-Building... The Missing Take-Out...

David A. Shaywitz reviews Nassim Taleb's views on randomness.


The problem, insists Mr. Taleb, is that most of the time we are in the land of the power law and don't know it. Our strategies for managing risk, for instance--including Modern Portfolio Theory and the Black-Scholes formula for pricing options--are likely to fail at the worst possible time, Mr. Taleb argues, because they are generally (and mistakenly) based on bell-curve assumptions.

If we accept Mr. Taleb's premise about power-law ascendancy, we are left with a troubling question: How do you function in a world where accurate prediction is rarely possible, where history isn't a reliable guide to the future and where the most important events cannot be anticipated?


I think that sums up where I stand on climate change. The talk of "consensus scenarios" and "90 percent certainty" leaves me cold. I want to know about outlandish scenarios and how to detect their likelihood.

More on Taleb's "Black Swan" thesis from Niall Ferguson.


Comments and Sharing





COMMENTS (9 to date)
mgroves writes:

I'm just a novice at stats, but 90% (as applied to a confidence interval) isn't all that great.

drtaxsacto writes:

I got his book last week. He covers a serious subject but with a great deal of flair and even some humor. I am not sure the reviewer read the book - indeed Taleb covers the question - "How do you function in a world where accurate prediction is rarely possible, where history isn't a reliable guide to the future and where the most important events cannot be anticipated? " But I think Taleb is not as negative as the reviewer suggests. A lot of life is less predictable - Taleb goes through a fairly thorough analysis of the things which we think are predictable and are really not. Those cautions are useful in trying to think about a world that is a bit less predictable. I did not read his earlier book (Fooled by Randomness) but this one is clearly worth reading.

Barkley Rosser writes:

You are right on this one; it is the tails of the distribution that matter the most, and nobody is in agreement about how much value we should put on those tails for climate change.

With respect to financial markets, there are complicated variations on Black-Scholes and all that, which can be used to deal with at least some known forms of fat tails. This is the bread and butter of econophysics, anyone curious to get an overview of which can see my Palgrave entry on "Econophysics," available on my website, listed here.

Matt writes:

Referring to the review of the Stern Report on climate change, and I forget the author, but it was a well done article.

He pointed out that the probability distributions associated with global warming were dominated by extremely rare but very costly possibilities.

When finding the principle components of the economy under global warming, the only choice is to break out these rare scenarios and treat them as separate components.

But, if the economy cannot afford catastrophic insurance, or a general rescue plan for these events then we cannot do our normal spectral curves to estimate long term costs.

Matt writes:

Having now read the article Arnold linked.

The better we predict, the more we create self fulfilling prophecy and the effect is to push uncertainly and noise out further in time where is aggregates up to dangerous proportions.

So, the problem is making predictions in the level of noise, and promulgating them as fact.

John Thacker writes:

The attractiveness of the bell curve also resides in its ubiquity under a great many conditions. The Central Limit Theorem often does apply.

The example of LTCM has essentially nothing to do with heavy tailed distributions versus normal distributions. (After all, even under a normal distribution an event of at least any particular rarity will happen eventually almost surely.) That quote alone gives me grave concern about the author's understanding of probability.

John Thacker writes:

There are *extremely* important differences between actual heavy tailed distributions (such as power laws like the Pareto distribution) and distributions which have the same sort of cumulative distribution function but, importantly, are NOT heavy tailed. Poisson and Gamma distributions can easily be fitted to many of these same types of data to which he (and others) blithely fit heavy tailed distributions.

It makes an enormous difference in the probability of extremely rare events which distribution you choose-- events so rare that they're not likely to be in your sample that you use to fit at all, but in the limiting case the probability is incredibly important.

I remain very unconvinced that people have demonstrated that these events are actually governed by heavy tailed distributions. It is extremely easy to specify how to obtain Poisson and Gamma distributions (and Erlang, and exponential) from very simple thought experiments and things that arise naturally. It is much harder to explain how a heavy-tailed distribution would naturally arise.

"It just looks like it" is patently unconvincing.

frank writes:

Options markets have a volatility 'smile': the implied volatilities of the extremum strike prices are much higher than implied by gaussian distributions. But this has been so for decades, so while true, it's not really news.

I think a lot of criticisms of basic parametric procedures miss the point on what a model does: it simplifies. It's trivially true that the real world is more complex than any theory, but theory is very useful nonetheless.

Fundamentalist writes:

I want to know about outlandish scenarios and how to detect their likelihood.

There is so hope for predicting rare events in using data mining techniques and simply using a binomial distribution rather than a normal one. Banks use such techniques to predict and detect fraud. Marketing people use them to predict purchases, which are always low probability evnets. Also, data mining, which employs machine learning with statistics, doesn't care what the distribution is, so you don't have to assume a normal distribution. They're far from perfect, but a good start towards predicting events in the tails.

Comments for this entry have been closed
Return to top