Arnold Kling  

Signal and Noise in Economics Writing

PRINT
Is the Beast Starved?... The Dodd-Frank Bill...

Kartik Athreya writes,


Why should anyone accept uncritically that Economics, or any field of human endeavor, for that matter, should be easy to process and contribute to?

Pointer from Greg Mankiw. Read the whole thing.

He is suggesting that bloggers supply more noise than signal on economic topics. I understand his point, but I disagree with it.

It is a fair point that it is tempting when writing for an audience that includes non-professionals to try to oversimplify, to make your views sound more well-grounded than they are, and to make others' views sound sillier than they are. If you read just one economics blogger, you will get that blogger's prejudices and blind spots along with whatever insights might be on offer.

It is possible, however, for the collective efforts of many bloggers to produce more signal and less noise. That would be the case if the competitive market serves as a check on the more unsound ideas. I am not saying that it works that way, but it might.

Athreya takes the view that the academic process of refereed journals is more rigorous and works well. I do not fully share that view. The peer-reviewed journal process may be the better than anything else someone has come up with, but it is a deeply flawed process. It rewards ritual over substance, and trend-following over originality. The process failed badly in the area of macroeconomics over the past thirty years, an era which I believe Paul Krugman is justified in describing as a Dark Age.

Athreya draws an interesting contrast between reactions to the economic crisis and reactions to natural disasters. He points out that the tsunami in East Asia and the earthquake in Haiti combined to kill hundreds of thousands and to impose hardships on many others that are far worse than what has been inflicted by the recession. Yet neither of those disasters was met by a denunciation of seismology for failing to predict them nor an outpouring of ill-informed speculation about what happened. He may be forgetting the "God's revenge" explanation proposed for the Haiti earthquake, but his point is well taken.

My pushback would be that economists have claimed to know more about the process of recessions than seismologists have claimed to know about earthquakes and tsunamis. No seismologist has ever said that we have "conquered" such events the way that economists have in the past claim to have conquered the business cycle.

I agree with Athreya that non-economists should express opinions about macroeconomics only with great humility. Where I disagree is that I think that economists, too, need to show humility. In fact, I have thought about opening my hypothetical book on macroeconomics something along these lines:


If a textbook covers a body of knowledge in order to raise the reader's level of proficiency, then what should one call a book that covers a body of knowledge in order to raise the reader's level of doubt? A doubtbook, perhaps. Above all, what you are about to read (or discard, as the case may be) is a doubtbook.

Herbert Stein, in his memoir Economic Washington Bedtime Stories said that he had learned two things from experience about economic matters.
1. Economists know very little.
2. Non-economists know even less.

Athreya is repeating the second point, and I agree However, I would also place stress on the first point.

[UPDATE: Scott Sumner lets loose with both barrels. Also, follow the other links in David Beckworth's post to find more on the controversy over Dark Age macro. Finally, one branch of heterodox macro, involving complexity theory and agent-based modeling, gets discussed by David Warsh. The bottom line: I don't think any of us should be saying that our view of macro is scientific while someone else's is not.]


Comments and Sharing


CATEGORIES: Economic Methods



COMMENTS (15 to date)
JFox writes:

Yes I agree. Back on the subject of your book, I for one won't hold it against you if you go out on a limb so to speak and make unsubstatiated claims, express opinions and bash economic theories you think are wrong. I am a non-economist, but am an MBA, CPA, etc. i.e., not a total fool, but I suffer my share of ingnorances. Anyway, many economists seem to tiptoe around the subject. I say, knock it off! Look at the contribution of someone who doesn't do that, such as Art Laffer. He obviously has a strong point of view that many disagree with. But so what. Is his contribution to the field diminished? I don't think so. Because economics deals in the subject of human behavior, and thus is less a hard science than something like physics, it is harder to establish clear cause-and-effect. You need to guess sometimes. I say go ahead and do that, and explain why you are guessing the way that you are.

Daniel Klein writes:

My "Who Is the Practitioner of Political Economy?" picks up on another major difference between political economy and sciences like seismology.

swhanson writes:

A difference between seismologists and economists is that we know, due to the significant and contradictory differences in the accepted and proposed theories of economists, that a great number of them are wrong about what they "know", depite their education and experience. But we just dont know which are wrong. So any uneducated blogger can be assured that he is no more wrong than most economists.

guthrie writes:

Your editor might baulk at opening the book on such a down note... You might want to consider re-working the line to something of an obverse iteration of Stein's quote. Something like, 'This book will, with any hope, will create doubt in those who are sure of themselves within this body of knowledge, and give hope to those who doubt their knowledge... that their doubt is healthy and well placed.'

... or somesuch.

Rebecca Burlingame writes:

Years ago as a piano teacher, I noticed how students would develop a passion for learning a certain song, and they would jump ahead two or more "grades" in order to learn it. I believe the same is true for economics today, as people gain passion for economics partly in order to understand why the world is not what they were once told it was. I believe that this newfound passion can greatly change the dialogue from what happened in the past thirty years. And I especially want to thank you, Professor Kling, for believing the same.

topcat writes:

As is often said about generals and war, the economy is too important to be left to economists.

Gary Rogers writes:

A note on blaming economists for the financial crisis: If seismologists had been mucking around with the earth's crust just before the event happened, they would rightfully have been blamed for the tsunami. Economists, on the other hand, prescribe unproven solutions for controlling the economy that distort growth and create all sorts of unintended consequences. Is it any wonder why they would be blamed for the financial crisis?

Just as an example, take economic stimulus. Economic stimulus is no more than a package of incentives that encourage people to spend rather than save, so is it surprising to find after years of stimulation we have overextended homeowners that have spent too much and saved too little and now have trouble paying their mortgage? So the enlightened economists have a cure, more stimulus. Maybe I know less than economists, but I am afraid economists know too much that just is not true.

Tom West writes:

To summarize, I think *any* field would face second guessing by amateurs if

(1) Given the level of disagreement on almost every aspect of the field, many, if not most, of the practitioners *must* be wrong (as swhanson points out).

(2) Economics essentially offers prescriptions on how our lives should be lived (as topcat and Gary Rogers point out). Few are the economists who don't have specific policies they believe should be enacted based on Economic principles rather than personal preference.

Lastly, I would say that most amateurs are NOT trying to contribute to the field of economics. Instead they are trying to offer policy advise in fields upon which the field of economics has some bearing.

It seems that her article is predicated on the assumption that offering policy advice that differs from the policy advice generated by someone's economic models is tantamount to suggesting the model is wrong. I think in most cases it's simply a lack of confidence that the economic model is the complete solution, which is a view shared by a an awful lot of economists as well.

The basic error is assuming that failing to heed the economist's policy advise is expressing any opinion about economics at all (except the opinion that field of economics doesn't hold all the answers).

Pedro writes:

My, I think that is a very uncharitable reading of Athreya. Perhaps he did not stress it sufficiently, but in my reading of his piece, he is calling for more humility among those economists that blog and often offer "simple" solutions to complex problems. I read him as saying "look, this is really complicated stuff, I do it for a living and still can't give easy answers to these problems and more often than not some blogger, economist or no, will have written a pithy 300 word post on why this or that is the solution". I agree with you that the process of peer review is largely about the ritual, but the ritual has a lot of merit precisely because it can be incredibly brutal in enforcing discipline in thought rather than disciplining thought.
Now, you could say that it may be that the methodology in Macro is completely wrong (which I would vehemently disagree with), but the larger point of his essay remains: even in its current state, modern Macro offers mostly ambiguous answers and can cater to a wide, wide variety of ideological bents and professional economists are less likely to be certain about the effects of a specific policy than any randomly chosen economics blogger. Sure enough, if you think Bellman equations have no place in Economics, then you're out of the loop entirely but I think the jury is still out on whether any theoretical contribution that has come strictly out of the blogosphere passes the crudest and least stringent test on internal consistency and generality.
Lastly, and while I think there are many things to be learned from people who offer their thoughts online and a lot of criticism than can and should be taken on board, I think his indictment is especially true of people like Krugman, who will tell all who will listen that the problem is "simple" when, as the massive disagreement on the topic clearly shows, it is not.

MAI writes:

Is the title of Stein's book correct- couldn't find on google

Washington Bedtime Stories: The Politics of Money and Jobs
Herbert Stein

Rahul writes:

A few things,

Comparing economics with cancer research is wrong. I think it is more comparable to auto - knowledge of enthusiast and car designers. Even designers know that enthusiasts, engineers or not, know a lot about cars and should be taken seriously.

In fact economics is so very important to daily life that we should encourage kitchen table conversations about some of the principles. To leave it to economists would be wrong.

I think, economists in particular, and scientist in general are more prone to in-breeding thanks to peer-review. A dose of reality is definitely required.

On a slightly different track, economics is not yet a science. There are "Schools" in economics very reminiscent of arts like schools of architecture(victorian, gothic), styles of artist (e.g. impressionist ). And in art, as someone said, "even my mother-in-law's dog has an opinion".

fundamenalist writes:
My pushback would be that economists have claimed to know more about the process of recessions than seismologists have claimed to know about earthquakes and tsunamis.

Exactly. And as Mises pointed out, the subject is us in economics. We have an advantage in studying economics in that we are studying humanity, something every human knows quite a bit about. Econ took a wrong turn when it decided that economics is not the study of humans but a subdiscipline of physics.

fundamenalist writes:
The peer-reviewed journal process may be the better than anything else someone has come up with, but it is a deeply flawed process.

That's right. The process serves to maintain the status quo.

Greg writes:

"as Mises pointed out, the subject is us in economics. We have an advantage in studying economics in that we are studying humanity, something every human knows quite a bit about. Econ took a wrong turn when it decided that economics is not the study of humans but a subdiscipline of physics."

Excellent comment, but lets not take this truth too far.

Saying that economics is not a subdiscipline of physics doesnt mean there arent some tools that physics uses that economics can use. For example. Laws about conservation of energy or equations about electrical circuits provide insight to people who wish to tinker with old or create new systems which involve energy or electricity. Using these laws one can find out if your energy inputs will give you the energy output you seek.

Applying this to economics and the current situation, we KNOW that all income is a result of spending. No one has ever earned a dollar without someone else spending. Therefore any proposed decreases in spending WILL result in income decreases IN AGGREGATE (distribution matters but......). Austerity packages will result in loss of income which will result in more defaults and job loss and will bring more hardship not less. Where these losses occur does matter as well but I wish someone pushing these packages would at least admit that unemployment and bank loan defaults will get worse as a result. They are pushing these as improvements (and necessary!) and not saying where the bad will be.

I have a comment on the Athreya article at Mark Thoma's blog if you're interested (http://economistsview.typepad.com/economistsview/2010/06/dont-let-fed-economists-tell-you-otherwise.html#comment-6a00d83451b33869e20133f1e8d4f6970b)

But I would like to add something to your lines, "The peer-reviewed journal process may be the better than anything else someone has come up with, but it is a deeply flawed process. It rewards ritual over substance and trend-following over originality." Here is a quote from the great growth economist Paul Romer of Stanford:

In evaluating different models of growth, I have found Lucas's (1988) observation, that people with human capital migrate from places where it is scarce to places where it is abundant, is as powerful a piece of evidence as all the cross-country growth regressions combined. But this kind of fact, like the fact about intra-industry trade or the fact that people make discoveries, does not come with an attached t-statistic. As a result, these kinds of facts tend to be neglected in discussions that focus too narrowly on testing and rejecting models.

Economists often complain that we do not have enough data to differentiate between the available theories, but what constitutes relevant data is itself endogenous. If we set our standards for what constitutes relevant evidence too high and pose our tests too narrowly, we will indeed end up with too little data...

My greatest regret is the shift I made while working on these external effects models...I suspect I made this shift toward capital and away from knowledge partly in an attempt to conform to the norms of what constituted convincing empirical work in macroeconomics. No international agency publishes data series on the local production of knowledge and inward flows of knowledge. If you want to run regressions, investment in physical capital is a variable that you can use, so use it I did. I wish I had stuck to my guns about the importance of evidence like that contained in facts 1 through 5.

– Journal of Economic Perspectives, Volume 8, Number 1, Winter 1994, Page 20.

Comments for this entry have been closed
Return to top