Bryan Caplan  

In Defense of Low Correlations

PRINT
Employer-Provided Health Insur... Financial Regulation in Hindsi...
An interesting side discussion from "The Power of Personality" defends the practical importance even small correlations:
Walter Mischel (1968) argued that personality traits had limited utility in predicting behavior because their correlational upper limit appeared to be about .30. Subsequently, this .30 value became derided as the ''personality coefficient.'' ...

Should personality psychologists be apologetic for their modest validity coefficients?  Apparently not, according to Meyer and his colleagues (Meyer et al., 2001), who did psychological science a service by tabling the effect sizes for a wide variety of psychological investigations and placing them side-by-side with comparable effect sizes from medicine and everyday life. These investigators made several important points.  First, the modal effect size on a correlational scale for psychology as a whole is between .10 and .40, including that seen in experimental investigations... Second, the very largest effects for any variables in psychology are in the .50 to .60 range, and these are quite rare (e.g., the effect of increasing age on declining speed of information processing in adults). Third, effect sizes for assessment measures and therapeutic interventions in psychology are similar to those found in medicine. It is sobering to see that the effect sizes for many medical interventions--like consuming aspirin to treat heart disease or using chemotherapy to treat breast cancer--translate into correlations of .02 or .03. [emphasis mine]
Now you might take a Hansonian view of this: Just say, "Medicine is as useless as personality," instead of "Personality is as useful as medicine!"  But should you?
[A] modest correlation between a personality trait and mortality or some other medical outcome, such as Alzheimer's disease, would be quite important... In terms of practicality, the .03 association between taking aspirin and reducing heart attacks provides an excellent example.  In one study, this surprisingly small association resulted in 85 fewer heart attacks among the patients of 10,845 physicians (Rosenthal, 2000). Because of its practical significance,  this type of association should not be ignored because of the small effect size.
For aspirin, this is a pretty convincing argument.  For chemo, not so much.  The difference, of course, is that an aspirin a day is painless, but chemo is horrible.  If this estimate of the medical benefits of chemo for breast cancer is correct, then I really wonder how many women would do it.

The bottom line: For costless and near-costless actions, small beneficial correlations are enough to justify changing your behavior.  If you find a lottery ticket for a free lunch, you might as well hang on to it.  But you shouldn't count on finding such lottery tickets with any frequency.  In the real world, beneficial changes usually have a steep price, and a lot of "free lunches" are just scams.


Comments and Sharing





COMMENTS (4 to date)
Arare Litus writes:

Different people seem to select "small" thresholds vastly differently - I would not say that a .1-.4, or a .5-.6, correlation is "small" - I would say it is significant and important. If something is 10% or higher it seems to be an important factor to consider. On the other hand, to say that a 0.02 is "significant" is questionable, and one would wonder if other factors may explain things - for example placebo effects, gathering support of family and friends in fighting an illness, etc.: perhaps chemo acts as a lightening rod for these other effects (and the actual chemo diminishes them with its toxic and negative side effects). With weak effects like that, then it becomes very important to consider other explanations if the negative consequences are significant themselves (and considering how costly chemo is financially, it seems like it is important on a "good use of
money" front also).

Chemo amplifies the visual signal of cancer - and thus marshals support. But if this marshaled support is what is key, and we do know it is important, then reconsidering chemo seems key.

I am surprised that the correlation is so low! Wow.

Steve Sailer writes:

Say that there is a low but positive correlation between IQ and job performance. There are all sorts of other factors that impact job performance. So, if you are, say, Sonia Sotomayor, that would seem like a good reason to enforce the EEOC's Fourth-Fifth's Rule to prevent disparate impact on Non-Asian Minorities.

On the other hand, aggregated across a lot of people, these low but positive correlations really matter. Those who scored above average on the IQ test turn out on average to do notably better on job performance than those who scored below average. For ethnic groups with substantial differences, the dysfunctionality of "disparate impact" rules are even more clear.

Dr. T writes:

Low but statistically significant correlations in human studies are useful ONLY if knowledge of slight differences between easily distinguishable population groups has benefit. Otherwise, you are wasting your time. This study fits into the wasting your time category. There is little benefit to saying that a group of people who score high on personality component X are 15% more likely to get a divorce than a group of people who score low on component X. Since few people have taken the relevant personality test, who cares? Since few people can change their personality, who cares? And, of those who know their personality score, does telling them they are 15% more likely to get a divorce matter?

I often encounter similar studies in medicine. "There is a 0.2 correlation between apolipoprotein B blood levels and the risk of myocardial infarction within 10 years." (Note: I made up the correlation.) This is followed by justifications for measuring apo B, recommended frequencies of measurement, where to buy test kits, and a list of reference labs. This test is being used by almost no one today, because it has no real utility. The correlation was real, but there are over a dozen other easier to measure biochemicals with similar correlations. That's the pitfall in these studies: when you say A correlates with Z, you may not realize that A', A'', B, E, G, H, and Q are non-independent variables that also correlate with Z.

In the weak personality correlations, perhaps the weakness was due to the testing technique or the chosen method of 'lumping and splitting' particular characteristics into specified personality types. But, more likely, the weak correlations were just that: personality isn't a great predictor of life events, it just predicts how one will behave in certain situations.

Willem writes:

Put it another way: Suppose an insurer has information that predicts 30% of incured damages. Would we use that information to model risks, or not? I dare say that a few .3 correlations build a very nice model.

Those variables usually also explain part of the residual in the other variables - giving you a stronger model.

Comments for this entry have been closed
Return to top