Bryan Caplan  

Russ's Challenge

PRINT
The Coming Minimum Wage Increa... The Myth of the Rational Vo...
From my colleague Russ Roberts:
I continue to ask the question: name an empirical study that uses sophisticated statistical techniques that was so well done, it ended a controversy and created a consensus--a consensus where former opponents of one viewpoint had to concede they were wrong because of the quality of the empirical work.
If you put it that way, I can't meet Russ's challenge.  But his challenge is excessive for two reasons:

1. Ending a controversy and creating a consensus almost always takes dozens, if not hundreds, of empirical studies.  As well it should - virtually every specific study is open to reasonable doubt.

2. What exactly counts as "ending a controversy and creating a consensus"?  Does every active researcher in the world have veto power?  Or is convincing two-thirds of them enough?  What if the "former opponents" just quietly give up rather than admitting error?

In light of these points, suppose we modify Russ's challenge thusly:
Name a body of empirical work that is so well done, it won over two-thirds of active researchers and induced half of the unconvinced to quietly give up or recant?
By this standard, the obvious response for economists is behavioral economics.  I don't know any economist under the age of 40 who denies that there are major exceptions to the standard rational actor model.  Many older economists are unconvinced, but they don't publish much about it.

For political science, I'd point to the literature demonstrating the falsity of the self-interested voter hypothesis.  I'll defer to people in the field, but my strong impression is that a solid consensus of active researchers now admits that objective self-interest tells you very little about voter behavior.  The people who aren't convinced do very little contrary empirical work; they just retreat into mathematical modeling.

For psychology: How about the three laws of behavioral genetics?

I've noticed that people skeptical of the value of empirical work often ask to see the one "bulletproof" study that "proves X."  But their request is confused.  Solutions to homework problems can be bulletproof because they're self-contained.  Since theory papers in econ are usually just solutions to really hard homework problems, theorists, unlike empiricists, have a warehouse full of of bulletproof research.  The problem, unfortunately, is that this bulletproof research answers questions of of little or no practical interest.  It's far better to focus on the real world and marginally advance our knowledge about something that matters.


Comments and Sharing





TRACKBACKS (1 to date)
TrackBack URL: http://econlog.econlib.org/mt/mt-tb.cgi/1954
The author at Fahreunblog in a related article titled Economia e Scienza. Russ Roberts has a question writes:
    Over the years, I have become increasingly skeptical of the power of statistical techniques to measure causation in complex systems like economics... I realized in a way I hadn’t before, that how we feel about the reliability of statistical results lin [Tracked on June 15, 2009 6:21 AM]
COMMENTS (6 to date)
saifedean writes:

I would take exception to the first, because I would modify Russ' challenge to include results about something that actually matters and exists in the real world.

The rational actor model is a construct of the fancy of economists in the 20th century. It was wrong to begin with. It means nothing to the real world and it tells us nothing. It is useful for constructing unrealistic models that are at best interesting thought exercises. Destroying this does not count as a triumph for economics--it's simply attempting to cancel out something that is bad to begin with: taking this model in any way seriously.

I have the same problem with Tyler's examples--they seem to me to be about abstract constructs of economists--not real life questions and problems. The only one that is real is number 5 (more policing reduces crime) and that strikes me as an awfully common-sensical result not really worth bragging about.

dWj writes:

I find it interesting how much of Becker's and Friedman's writing from the fifties and sixties, whenever it refers to expected utility or other optimizing behavior, tips the hat repeatedly to the possibility of exceptions before almost apologetically proceeding with the discussion of the consequences of the "standard economic model".

Pedro writes:

Regarding self-interested voter: I tried to discuss this point in a graduate public choice course, and nobody had a clue what I was talking about.

James Hanley writes:

As a political scientist, I would argue that you're wrong on the status of the concensus about the self-interested voter hypothesis. First, many political scientists who don't believe in the self-interested voter weren't convinced by the empirical research--they never believed in it anyway and no empirical research could ever have won them over.

And your criticism of those who ignore the empirical research and focus on mathematical modeling is a bit inapropos--they did not "retreat" into mathematical modeling, but for the most part have never done anything but mathematical modeling.

That leaves a relative few who have actually been won over by the empirical research itself.

In a nutshell, political science is still far too fragmented a discipline (regretably) to have concensus about much of anything. It doesn't help that far too many either think that only mathematical models are good research (and empirical verification is simply a less elegant redundancy), or think that any kind of positive research is a conservative reinforcement of the status quo.

Juan Carlos writes:

how about Daron Acemoglu's, Simon Johnson's and James Robinson's series of articles testing the hypothesis that institutions are te cause of long run growth?? they use instrumental variables in a variety of historical examples and prove that is in fact the truth.
i think the consensus in the profession is that they (along with others, for example Engerman and Sokoloff) beat the 'geography is the cause' people

Gena Kukartsev writes:

Isn't your modified challenge satisfied by a body of works on global warming? If so, doesn't that disturb you?

I do not have a good alternative to suggest but "consensus" in science is a dangerous thing.

Comments for this entry have been closed
Return to top