David R. Henderson  

MIT Technology Review's Huge Logical Fallacy

Another Case Against Education... You don't know what you've got...
What factors, then, determine how individuals become wealthy? Could it be that chance plays a bigger role than anybody expected? And how can these factors, whatever they are, be exploited to make the world a better and fairer place?

Today we get an answer thanks to the work of Alessandro Pluchino at the University of Catania in Italy and a couple of colleagues. These guys have created a computer model of human talent and the way people use it to exploit opportunities in life. The model allows the team to study the role of chance in this process.

The results are something of an eye-opener. Their simulations accurately reproduce the wealth distribution in the real world. But the wealthiest individuals are not the most talented (although they must have a certain level of talent). They are the luckiest. And this has significant implications for the way societies can optimize the returns they get for investments in everything from business to science.

This is from "If you're so smart, why aren't you rich? Turns out it's just chance." March 1, 2018.

And in case you have any doubt about the article's conclusion, here's the dec line:

The most successful people are not the most talented, just the luckiest, a new computer model of wealth creation confirms. Taking that into account can maximize return on many kinds of investment.

The author of the piece is making a huge logical fallacy. The technical term for the fallacy is "affirming the consequent."

A implies B. B. Therefore A.

A computer program with luck programmed in gives us the same wealth distribution that we observe. Therefore luck is driving the results in the world.

HT2 Cyril Morong.

Comments and Sharing

CATEGORIES: Income Distribution

COMMENTS (18 to date)
DeservingPorcupine writes:

I read this article and was shocked by this very sloppiness.

Mark Z writes:

I'm pretty sure it's a trivial task to write a function of x (x being any quantitative variable - some measure of talent or luck, or height, IQ, etc.) that yields output (representing wealth here) that has a similar distribution to the current wealth distribution.

Literally any variable. There are pretty much an infinite number of possible models that could give such a result. I'm tempted to open Rstudio and find some data and try this with, I don't know, number of cars owned in one's life, nose width, or time spent per day on facebook as the determinant of wealth, and try and submit the results to a high impact journal.

Kevin Dick writes:

Within the mistaken logic, there is actually an important insight here. There's an interesting experiment to run:

"The strategy that delivers the best returns, it turns out, is to divide the funding equally among all researchers. And the second- and third-best strategies involve distributing it at random to 10 or 20 percent of scientists."

If the government is giving out research dollars anyway, might as well do a test to see if this supposition is true.

As it turns out, I'm currently running a version of this experiment with startup investing.

There's also literature on the microeconomics of skill in producing outcomes with a large random component. If this skill has diminishing returns to scale, each possessor of that skill will increase his scale until that skill is almost 0. So "good" scientists will take on projects until their projects are only slightly better than average. As will investors increase the size of their funds. This also argues for a broader portfolio.

drobviousso writes:

I was very curious how they modeled "luck," because my own experience with traffic simulations shows that its insidiously easy to make "luck" swamp everything. Here's what they did.

"Figure 1: An example of initial setup for our simulations. N = 1000 individuals (agents), with different degrees of talent (intelligence, skills, etc.), are randomly located in their fixed positions within a square world. During each simulation, which covers several dozens of years, they are exposed to a certain number NE of lucky (green circles) and unlucky (red circles) events, which move across the world following random trajectories (random walks). In this example NE = 500. All simulations presented in this paper were realized within the NetLogo agent-based model environment [45]."

There's an old joke about engineers being out of touch. Its about a fancy college prof saying he knows better how to get cattle to market. A bunch of rancher don't agree and say 'Oh yeah, prove it.' The fancy prof says 'OK, first assume a spherical cow...'

These people have literally assumed a spherical luck orb.

Their overall model is worse. Its insane. It bakes all sorts of contested assumptions into the beginning, and is needlessly complex (see above).

MikeP writes:

I skimmed the original paper.

Let's see if I got it right... They took a normal distribution, ran a random process over it, and come to the conclusion that most of the people at a particular point of the resulting distribution are from the mode of the original distribution, not from the tail.

What a shock.

drobviousso writes:

That was a very negative comment. I'll try to be a little more constructive:

This model assumes that each person is completely without agency. At a minimum, we know that people's risk profiles change as they age and that they seek more risk/reward at a young age and less at an old age. In this simulation, everyone is equally exposed to risk. They should let people dial a risk/reward seeking parameter in line with other features of that person such as age.

This model introduces coupling of risk between people in all cases. In truth, some random events are coupled between people (maybe a viral epidemic) and sometimes not (maybe tripping in traffic). Instead of these luck orbs, they should build social networks, have good and bad events occur to people according to a poisson distribution, and have each event have a chance to propagate along the network.

This model assumes each person is 100% atomic. In reality, we live in social networks that smooth both windfalls and bad breaks. When fate deals someone a bad hand, their friends and family help them out. When someone wins the lotto, their friends and family usually get a cut. That isn't happening here.

They should be validating their model with intermediate checks. But since luck orbs aren't real things, they can't. I didn't see where they use objective information about how big of an impact good/bad luck should be, or how often those things happen. Adjusting for that would improve their validity.

john hare writes:

The computer model charts each individual through a working life of 40 years. During this time, the individuals experience lucky events that they can exploit to increase their wealth if they are talented enough.

I think this is the money paragraph and they didn't realize it. A lucky event, exploited by talent, and I would add high level of effort.

There is a certain amount of luck involved in success just as there is in failure. Many seem to not realize that it is what you do with it that matters. I can look back at many events involving luck or chance in my life, and anytime I care to examine them, I can point to how changing my actions could have vastly changed the outcome.

Mark Z writes:

I think the key problem with this paper is, well, the idea that it has any meaningful implication for the real world we observe.

We observe variation in wealth, yes. The authors contend that a model with variability in talent and luck can lead to a similar pattern of variation in output. But couldn't a model that *only* contains variation in luck - one in which all participants are equally talented - also yield the same outcome? Conversely, you could make a model where variation in talent mirrors the observed variation in wealth, and you would find - what a shock - that a pure talent, no luck model can lead to the same wealth distribution as what we observe.

By tweaking the frequency of good or bad luck events, or by tweaking the distribution of talent, you can get about any distribution you want from the output. Their observations, therefore, seem trivial.

I think MikeP's terse assessment basically sums it up.

Seth writes:

I agree.

I also don't think a few academics know what talent is.

But, I don't need a computer model or study to convince me that luck plays a big role in wealth distribution.

I agree with this:

"this has significant implications for the way societies can optimize the returns they get for investments in everything from business to science."

That's better than where I thought they were heading after this statement, "how can these factors, whatever they are, be exploited to make the world a better and fairer place?"

Maybe the fairness they refer to is opening the doors in areas that have been cordoned off by rent-seekers who have developed a mystique of talent.

Instead of funding research grants, having contests with bigger prizes for research results, for example.

Thaomas writes:

I agree about the computer program not "proving" that real life outcomes are affected by luck. I'd say it is an argument against the crude, "I deserve whatever is mine (and so any taxation to redistribute part of it is illegitimate)." And while few people will admit to actually believing that, many will behave politically as if they did.

Mark Z writes:


One need not believe everything everyone owns they deserve to own in order oppose redistribution.

According to Malcolm Gladwell, many of the top earners are not gambling or taking risks at all. This was also regarded as a refutation of capitalist dogma. Is a debate called for?

robc writes:


I deserve whatever is mine* so any taxation to redistribute part of it is illegitimate.

Even If I got it by luck (although in my case that probably nets to about zero, luck goes both ways).

*with the exception that I favor the single land tax, because while I support absolute property rights, I find no natural law basis for them, so the Georgist SLT appeals** to me.

**Also I dislike rent seeking, and land ownership is that pretty literally.

Glen Smith writes:

From personal experience, luck is a major factor but that is based on personal experience. Are there things I can look back on and say that was a mistake? Yes, but that is sort of falling victim to the black swan. If the article shows anything, it is that financial success is dependent on multiple variables often co-dependent on each other.

n shackel writes:

Observations of what a theory predicts are taken to be confirmations of the theory. Note that the following is therefore the form of such scientific inference:

If my theory is true, we will observe X
We observe X
Therefore my theory is true

Certainly, this is not deductively valid: it is affirming the consequent. Nevertheless, this is how most theory confirmation goes, and no less so in economics than anywhere else. Affirming the consequent is the form of most scientific confirmation. Almost no scientific knowlegde is got from the valid affirming the antecedent:

If we observe X then my theory is true
We observe X
Therefore my theory is true.

The reason this is not used is because the first premiss is almost never true. This is because theories are underdetermined by the observational evidence.

So if you reject their result on the grounds of the fallacy, you must also reject all the science that is founded in the same way: you will have to reject most of physics, chemistry, biology etc etc.

David R Henderson writes:

You make a good point. But you overstate it. Let's say that I argued that when the American League wins the world series in years divisible by 4, that causes the Republican presidential candidate to win the election--and I proceed to show a complex chain of hypothesized causation. And let's say that the data fit perfectly. Would you accept my theory?

MikeP writes:

Observations of what a theory predicts are taken to be confirmations of the theory.

You are describing inductive reasoning, which as you suggest is how science works. But, much like the article...

The most successful people are not the most talented, just the luckiest, a new computer model of wealth creation confirms.

...you seem not to know what 'confirmation' means.

I think you are looking for the word 'evidence'.

If my theory is true, we will observe X
We observe X
Therefore we have evidence that my theory is true

It's okay to make this error in a blog comment. It is unconscionable in a publication from a major scientific institution.

MikeP writes:

By the way, let me try this scientific inference thing:

Let my theory be that, like Mark Z suggests above, a model can be tuned to get any interesting result someone would like to report on.

If my theory is true, we will observe articles reporting on the results of models that can be tuned to get an interesting result.
We observe an article reporting on the results of a model that can be tuned to get an interesting result.
Therefore we have evidence that my theory is true.

Now I would argue that the Bayesian priors make the evidence for my theory stronger than the evidence for the paper's theory.

But just to add to the evidence of my theory...

First, why do lucky events apply with probability proportional to talent and unlucky events apply with probability 1?

My theory would suggest that that choice made the model result match the desired interesting result more than either both probabilities being proportional to talent or both probabilities being constant. I do not know what their theory would suggest and could not find any reason for their choice in the paper. That's evidence for my theory.

Second, note that instead of talent the normal distribution could be height, or head circumference, or average difference from the speed limit one drives, or anything else that is normally distributed after normalizing for obvious differences such as gender.

The paper's model applies to all of them. All of them would yield exactly the same results. Why was talent chosen as the one the paper studied? Here the paper at least makes an argument for this decision. But my theory suggests that selecting talent would make the result more interesting than selecting saturation of the irises or the like, so that's why they selected talent. The fact that the MIT Technology Review reported on it is even more evidence for my theory. I again think there is more evidence for my theory than for the paper's theory.

Is there any evidence for the paper's theory that isn't better explained by my theory? I happen to believe that wealth is positively but not hugely correlated with talent, and that luck plays a large part in wealth accumulation. But it looks to me like this paper provides no confirmation or evidence that this is true since the conclusions of the paper are better explained by alternative theories.


Return to top