Bryan Caplan  

Goldberger Explains Why My Referee Was Wrong

PRINT
Communications Media and Insti... Letting Students Drop a Questi...

When the typical professor reads a referee's rejection letter, his standard explanation is that "The referee was an idiot." Most of the time this is unfair - at least half the time, it's the author who's the idiot, and the referee is doing the world a favor by keeping him out of print.

Nevertheless, the referee occasionally is an idiot, or at least says idiotic things. Case in point: My last rejection. (Paper and journal withheld to protect the guilty). The referee spent over a page explaining that my results suffered from multicollinearity. Never mind the fact that - despite the high correlation between two of my independent variables - the standard errors were quite small. What my referee needed, in short, was a massive dose of the profession's wisest econometrician, Arthur Goldberger. From a great interview in Econometric Theory:

Goldberger: There is one more expression which you haven't asked me about because you haven't heard of it, and that's micronumerosity.

ET: Micronumerosity?

Goldberger: Take a look at any econometrics text; there's always a chapter on multicollinearity in which the author broods about all the terrible things that happen when you have multicollinearity. It turns out that the terrible thing that happens is you get large standard errors.

ET: Right.

Goldberger: Right. And that's what you get. It's very analogous to trying to estimate a population mean when you have only a small sample. You get a large standard error. Right? Nevertheless, you don't find any chapters in the textbooks about the terrible things that happen when you're trying to estimate a population mean and have a small sample. Why not? Because there's no fancy name for that situation. You can't just say "estimating a population mean when you have a small sample." So I've constructed a fancy name, and now there'll have to be a chapter, and that name is "micronumerosity."

Actually, I've written a parody of the paragraphs that appear in multicollinearity chapters.

The Econometric Theory interview then blockquotes the parody. Come on, give econometric humor a try!
The extreme case, perfect micronumerosity, arises when n=0, in which case the sample estimate of i is not unique. (Technically, there is a violation of the rank condition n > 0: The matrix (0) is singular...) That extreme case is easy enough to recognize, but "near micronumerosity" is more subtle, and yet very serious. It arises when the rank condition n > 0 is barely satisfied. Near micronumerosity is very prevalent in economic research. Later we will discuss tests for the presence of micronumerosity and see what can be done when those tests suggest that micronumerosity is lurking in n. But first, its tragic consequences.

Consequences of micronumerosity:

(1) Precision of estimation is reduced. There are two aspects of this reduction: estimates of it may have large errors, and not only that but V(Y) will be large.

[...]

Testing for micronumerosity:

(1) Tests for the presence of micronumerosity require the judicious use of various fingers. (Some researchers prefer a single finger, others use their toes.) A generally reliable guide may be obtained by counting the number of observations...

The point of all this, of course, is that "multicollinearity" and "micronumerosity" are not statistical flaws; they are features of the data that tend to create justified uncertainty. If you get strong results despite the fact that your variables are highly correlated, or your sample is small, more power to you.

Unfortunately, the guy most in need of a massive dose of Goldberger, my referee, probably doesn't read this blog. But if you want to avoid making his mistakes when you're playing gatekeeper, let Goldberger be your conscience.


Comments and Sharing





COMMENTS (10 to date)
jsalvati writes:

I'm curious. Do you do mostly bayesian stuff?

Steve Miller writes:

The second reader was even worse!

Tim Lundeen writes:

LOL! "The extreme case, perfect micronumerosity, arises when n=0," I just love it.

stanfo writes:

The Goldberger text is the best and simplest textbook. He seems to have a good grasp of econometrics, which is more than I can say for other textbook authors...

Oh yeah, and I love the econometric humor!

One solution for micronumerosity: Assume it converges asymptotically to the mean. Which is what we do. Which is silly.

dave smith writes:

Brian, this also happened to me in my latest paper, although it did not lead to a rejection.

In my revise and resubmit, I explained the multicollinearity was not a problem...my t stat was still 4!

I think it is what people say when they don't know what to say.

Daniel Meng writes:

This happens to me several times when the audience raised their concerns about multicollinearity during my presentations.

eric writes:

Sometimes I see papers with lots of correlated regressors, a kitchen sink regression (eg, ebit/assets, net income/net assets, Earnings per share). Lots of them have 'wrong signs', in that their coefficient is opposite what it would be in a univariate regression. while the coefficients are often highly significant, I don't trust 'wrong sign' regressors, because they are clumsy ways to capture nonlinearity in the data.

eric writes:

Remember Heckman's dismissal of Murry and Hernstein's Bell Curve in the JEL (1995)? He made a big deal out of the fact that one is trying to infer the effect of IQ, when the true model included *both* IQ and environment. Thus, you could never say anything about IQ's importance. This from a Nobel Prize winning econometrician (who now acknowledge's IQ's importance).

I read that and thought, if so, 99% of econometrics is bunk. But this was a very tendentious review.

Hey Caplan.
Have you tried to use Shrinkage regression, as lasso regression or ridge regression?
There is a trade-off bias for variance, I know, but maybe you can increse even more the significance.

I don't know much about it, but it seems to be a good "solution".

I hope you apply your paper to somewhere else!

Glen writes:

Isn't the other problem with multicollinearity that you may be splitting what is essentially one effect between two explanatory variables? E.g., if you try to explain wages using both SAT and ACT scores, you'll split the effect between them, getting two smaller coefficients with lower levels of significance. But that would understate the value of "standardized test scores" in predicting wages.

Comments for this entry have been closed
Return to top