Bryan Caplan  

Test the Predictions - Or Check the Assumptions

PRINT
Schools of Macro... What Is the Male Marriage Prem...
Where do economists draw the line between science and dogma?  In most cases, they say something like this: "A model is scientific if and only if it makes true testable predictions."  Perhaps this is why Arnold was dissatisfied by my brief reply to last week's challenge.  Arnold:

My question is how to reconcile low employment with low unit labor costs. Presumably, low unit labor costs would cause labor demand to be higher rather than lower.

My translation of what they have to say is this:

Bryan: Notwithstanding remarkably low unit labor costs, if unit labor costs were even lower, we could have full employment.

Scott: Notwithstanding remarkably low unit labor costs, if we had higher nominal GDP, we could have full employment.

Each of their positions amounts to a non-falsifiable hypothetical. Not that we should be shocked by non-falsifiable statements. This is macro, of course.

I agree that testable predictions are one sign of science.  But there's another, equally good, sign: Whether the assumptions empirically check out.  In practice, economists frequently examine the realism of their models' assumptions.  Furtiveness aside, I see nothing wrong with this approach. 

Case in point: The main reason I believe that nominal wage rigidity is an important cause of unemployment isn't the powerful predictions the approach implies.  The main reason I believe that nominal wage rigidity is an important cause of unemployment is that the assumptions seem so undeniably true:

Assumption #1: Nominal wage cuts hurt workers' morale - and therefore their productivity.

Assumption #2: Demand for labor slopes downward.

Convince me that either assumption is false, and I'll change my mind.


Comments and Sharing





COMMENTS (21 to date)
Ken B writes:
But there's another, equally good, sign: Whether the assumptions empirically check out.

My bold. Equally good? You sure you want to defend that? Because I can mass produce different wrong predictions from the same set of premises. That ultimately is why we test drugs for example.

You can make predictions about the motion of electrons through gratings. People did in the 1920s. The predictions were wrong when made with classical mechanics, right when made with quantum mechanics.

RPLong writes:

Given the way the second paragraph of this post reads, I expected to read something actually empirical in the subsequent paragraphs.

Ken B writes:
Convince me that either assumption is false, and I'll change my mind.

My complaint is that you might be wrong anyway, if there is something else afoot. "Something else afoot" is why the real test is prediction not plausibility.

Warren writes:

Not related to this post; but here's a Calvin and Hobbes to go with Bryan's forgotten vs failed in education:

http://blog.measurableadvancement.com/?tag=perry-child-development-center

Arthur B. writes:

How is your hypothetical non-falsifiable anyway? If labor cost changed to 0 ceteris paribus and full employment wasn't observed, your theory would be falsified.

M.R. Orlowski writes:

Sounds a little Misesian, no?

Chris Koresko writes:

At the risk of piling on here, Arnold Kling, Ken B, and others have this right.

Consider how a model is built:

* You start with some input assumptions.

* You do the math (rigorous logic, in other words).

* You make a prediction of something observable based on something else observable.

Arnold Kling et al. argue that the only way to tell your model is right is to make a set of non-trivial predictions (i.e., predictions you couldn't have reached without a model like yours) and then show that they are correct. I agree that this can be good evidence that your model has merit. It doesn't prove your model is right, since it's possible that you have hit upon a wrong model that happens to behave mathematically like a correct one. But it does enable you to reject many, and probably most, bad models.

You argue that it's just as good to look at the input assumptions and confirm that they make sense. But that can't be true, for several reasons:

* You might have correct input assumptions and do the math wrong. This is easy to imagine, especially if the math involves writing a computer program to generate your predictions.

* You could get the math right, and then misinterpret the results.

* You could get good results, but your model may fail in the sense that you've correctly predicted what is actually a minor effect that's swamped by a much large effect you didn't think of. This is Ken B's "something else afoot".

* Or your assumptions that "seem so undeniably true" could be wrong. For example, the assumption that a drop in nominal wages reduces productivity could be wrong because it's a signal to workers that the company's in trouble and they need to get cracking if they want to keep their jobs.

Any of those mistakes, and probably a lot more I haven't listed here, have the potential to make your predictions fail to match the data. If you just proceed confidently on nothing but the belief that your input assumptions seem reliable, you can go a long way off track.

mark writes:

Idk. Regarding your two postulates, I think there is only a modest correlation between workers' morale and their productivity. It exists at the margin but it's not that substantial. Let's say at the start of a semester a teacher comes to the conclusion s/he is not going to get tenure, dampening his/her morale considerably. S/he still completes teaching the courses s/he is respnsible for, because s/he knows it will affect future job prospects. The university still collects tuition for the course, the students still get the assigned number of credits, etc. Sure at the margin, s/he might miss a class, the quality of lectures might suffer somewhat - although with teacher review sites, that would be a mistake for future employment - but at the end of the day, all the output gets delivered. There are lots of other examples - machines replacing people dampen employment and morale but not necessarily output, etc.

Bill Nichols writes:

The requirements on assumptions and testable predictions are not logical "or" but rather logical "and" . Either are necessary but insufficient.

PrometheeFeu writes:

I strongly disagree. Your assumptions are a very important part of your theory and they should be looked at critically and empirically, but they are only the beginning of your theory. Following your assumptions comes a whole bunch of logic, math, stories, reasoning etc... All of that could still be wrong.

Furthermore, you could be missing some variable that makes your whole theory wrong.

Without a testable prediction, you don't have a theory. You just have interesting musings which can lead to endless conversation of the "but what if?" type. It can be fun and intellectually stimulating, but it's also not very helpful to furthering our understanding of the world as it actually is.

Ken B writes:

@Chris Koresko: That is PART of my 'something else afoot'. One's model may just not fit reality at all. That is the case with the gratings example I gave: the apparently clear notion of a bullet following a path is just wrong when applied to small enough objects. In a model you can state your assumptions, in science it's not so easy.
(An even better example is Bell's Theorem but that's too complex for a blog comment.)

Caplan's remark shows I think a profound misunderstanding of the difference between mathematics -- manipulating models -- and science. Either that or he just misspoke :> But the error as it stands , "science doesn't need predictions, checking our assumptions is just as good", is a howler.

Pandaemoni writes:

I just had to agree that the validity of one's assumptions are not an "equally good" indicator of what is science. I can easily imagine a situation in which one can find no flaw in one's assumptions *but* nevertheless one's model fails to accurately describe reality.

You generally cannot use pure logic and seemingly valid assumptions to deduce the way the universe is, scientifically speaking. The key to science is not the testing of assumptions which underlay a model (not that they should not be tested), it is comparing results of the model to reality, and looking for examples of phenomena that demonstrate the model to be incorrect.

D. F. Linton writes:

Assumption #1: Nominal wage cuts hurt workers' morale - and therefore their productivity.

Assumption #1.a: Workers are too stupid to realize that inflation has reduced their real wages - and therefore work cheerfully and productively with thoughts of COLA's never entering their pretty heads.

blink writes:

"Check the assumptions" gets your foot in the door, but unless you have a Landsburg-style proof that is all you get. With sticky wages, your assumptions buy you downward pressure on employment but hardly enough to say they are "important" for unemployment. What about the effort margin, for example, among others?

Jonathan Soup writes:

Nah Nah Nah it aint like that bra.

A wage cut could easily STIMULATE worker productivity. It signals that - ruh roh - the next stage might be termination. If they can cut your pay, then maybe they can cut out your position. It scares workers into working harder. This is mainly true in a depressed economy with high unemployment where just having a job - say for medical benefits - matters much more than wages.


ahhh boy. what 'ave we dun?

Chris Koresko writes:

Interestingly, one of the dictionary definitions of dogma is "a settled or established opinion, belief, or principle".

By this definition, aren't "assumptions seem so undeniably true" pretty much dogma?

Tony J writes:

Assumption #1 Do Nominal wage increases lift workers' morale - and therefore their productivity? Again, only for the short-term. The effects of either a cut or lift in nominal wages trigger an emotional response due to the changes in circumstance. When this emotional response stabilizes, people accept/adapt and a new norm is created. Eg Pavlovian conditioning.

Assumption #2 Demand for labor only slopes downward after passing the tipping point in regards to the "Law of Diminishing Returns". Also, labour is mobile and can give the illusion of lowered demand (at Point A) when it's actually being redistributed (usually to developing countries in the pursuit of profits via wages arbitrage ; Point B).

Mark Little writes:

Well, where to start?

I agree with Bryan's assumptions, but I agree with Arnold's conclusions. There are two key points here.

First, what "labor" market are you talking about? The demand curve for what slopes downward? Arnold's argument (I agree) is that in the modern economy labor is extremely heterogeneous and less substitutable than one might imagine. (Thus the recalculation story.) Employees are not a commodity like wheat. You shouldn't think in terms of the classical model of a market for "labor hours" (as still exists, in a very small way, for unskilled day laborers). Aggregate arguments about sticky wages for "labor" in general thus miss the point.

Second, the "market" for employed staff is different from the market for new hires. Bryan's first assumption applies to the former, not the latter. The wages for current workers are a done deal, subject to renegotiation on a longer time frame than the pricing of offers for new hires. For employed staff, long term investments are involved on both sides of the contract--for the employee, an investment in firm-specific human capital, and for the firm an investment in organizational capital.

(The degree this is true varies tremendously, of course, both over firms and over positions. Note that this distinction of new versus done deals applies even to commodities like wheat: the price of the bushel of wheat sold to the miller yesterday is much more "sticky" than the price of the bushel offered at auction today.)

Other commenters have responded sufficiently to Bryan's non sequitur that the truth of a theory's assumptions validates the theory. To me this calls to mind Milton Friedman's infamous The Methodology of Positive Economics essay. His overall thesis is widely regarded as untenable, but his discussion regarding the realism of assumptions is an interesting counterpoint.

Ken B writes:

Overheard at NASA:
Collins: Wait, wait, you mean the guys figuring the trajectory model the capsule as a POINT? A dimensionless POINT?
Armstrong: Whoa, that's just not a realistic assumption.
Wendt: Yes but ...
Aldrin: We aren't going. Don't you guys understand it's realistic believable assumptions that validate a theory? Get yourself some point-size astronauts.

Steve Roth writes:

'The main reason I believe that nominal wage rigidity is an important cause of unemployment ... is that [it seems] so undeniably true."

I don't know how anybody could argue with that.

Jon writes:

Suppose I own a hot dog stand with 2 employees. I sell a thousand hot dogs a day. I'm making a good profit and I don't see any additional demand. I'd expand if I could, pay 2 more people to sell another 1000 hotdogs. But I don't think I can sell them, so I just take my current profit. An unemployed individual says he'll work for me for half of what I pay the other guy. Good. I fire the other guy and make even more profit. But I don't expand.

What I could do is lower my prices and try to horn in on the hot dog business run by my competitor across town. Suppose since I pay such low wages I can do that. I displace his business. He has to lay off his 2 employees and I'm able to hire 2 more at half price to run my new hot dog stand. Total hot dog consumption doesn't change. All that has changed is the employees make half of what they did and my profits are up. Overall unemployment is unchanged. But now since employees make less money that reduces consumption for them just a bit. If anything this drives demand for hot dogs down. If anything I'll be laying off someone soon as demand falls. The reduced wages to employees actually depresses employment.

Comments for this entry have been closed
Return to top