Bryan Caplan  

How Would We Know If a Bail-Out Worked? Take Your Stand

Public Opinion and the Bail-Ou... Paulson as a High-Pressure Tel...

We're still likely to see a bail-out in the near future. So here's a question: If the bail-out happens, how will we know if it prevented disaster? If unemployment stays under 8%, proponents will say that the bail-out prevented a recession. But if unemployment hits 8% or higher, proponents will say that things would have been even worse without the bail-out. Opponents, of course, will flip things: Good conditions mean the bail-out wasn't necessary, bad conditions mean that the bail-out made things worse.

Ex ante, though, basic Bayesian reasoning doesn't allow us to claim that whatever happens confirms our position. If the occurrence of X raises the probability of A, then the occurrence of not-X must reduce the probability of A. See Eliezer's classic essay.

Unfortunately, if you let people see whether X happens before they update, you can't show that they're being bad Bayesians. You only see how they updated, not how they would have updated if the news had been different.

So here's my request: Tell us how you're going to update in advance. Assume the bail-out happens. If unemployment stays below 8%, does that make you more or less confident that the bail-out prevented disaster? If unemployment rises to 8%+, does that make you more or less confident that the bail-out helped prevented disaster? If you're a good Bayesian, you must give opposite answers to these two questions.

I'll bite first. If the bail-out happens, and unemployment stays below 8% for the next two years, I'm going to become less confident that the bail-out prevented disaster. After all, even a near-miss with disaster should look pretty ugly. Alternately, if the bail-out happens, and unemployment hits 8% or higher during the next two years, I'm going to become more confident that the bail-out prevented disaster. I still won't be convinced, but I'll be less skeptical than I am now.

You can easily argue with my judgments. My point is that you can't reasonably get more confident no matter what happens. One good way to make people believe reasonably is to get them to explain their updating procedure before they update. So who wants to go on the record? Anyone who does rises in my eyes; and you know what that means about everyone who doesn't, right? :-)

Comments and Sharing

TRACKBACKS (1 to date)
TrackBack URL:
COMMENTS (19 to date)
kebko writes:

I don't understand your logic.

What if I think that raising tariffs on all imports to protectionist levels is what the economy needs, or maybe a severe contraction of the money supply? Does that mean that you have to concoct a set of predictions based on macro-economic data that admits some circumstances where my proscription was positive?

It seems reasonable to me to be able to take a position that a given proscription will be good or bad, regardless of what some macro-level measurement is.

another bob writes:

i'm expecting libor to go above 4% shortly and unemployment to break 8% soon and stay above that level for the next 12 months. further, i expect a bail-out to pass this week but it won't address the financial plumbing issues correctly or in a timely enough fashion to change the outcome.

if a bail-out passes and libor goes back below 3% within 60 days and unemployment stays below 8% for the next 12 months - i'll eat my hat. it must have been the bail-out changing the outcome.

if a bail-out passes and libor is above 4% and umemployment is above 8% can i say that the bail-out is then responsible for the 12%+ inflation that is reported by month 12?

Curunir writes:

Great post. I'd be interested to know how you'd think about the opposite question I've asked elsewhere; namely, what if the bailout doesn't pass? What scenario of events would cause you to in retrospect raise your belief that the bailout would have been a good idea?

Todd writes:

I agree with Bryan. Though I might go one step further. If unemployment doesn't pass 8% at least briefly in the next two years, I'll assume that the government successfully quashed one of the most vital aspects of a free-market economy, and that we will all be worse off in the long run.

Bryan Caplan writes:
Curunir writes:

Great post. I'd be interested to know how you'd think about the opposite question I've asked elsewhere; namely, what if the bailout doesn't pass? What scenario of events would cause you to in retrospect raise your belief that the bailout would have been a good idea?

That one's easier. If you say that someone will die without treatment and you don't treat them, you can just check whether they die. Similarly, if we don't pass a bail-out and unemployment stays below 8%, I'll be very confident that we were never facing disaster. If unemployment goes to 8%+, I'll be less confident.
Jeff writes:

That's easy. The bailout will be deemed a great success if campaign contributions from the finance industry increase significantly, and if more than 20 by-then-former congressmen find lucrative employment in the industry, preferably by sitting on a board of directors, since that doesn't require any actual, you know, work.

Daublin writes:


Kudos on the general idea. That would be splendid if various talking heads could be encouraged to take up your challenge.

One little question, though: why unemployment? When I think economic meltdown, high unemployment would be but one kind. From what I understand of the plausible results of a meltdown due to the financial system going haywire, though, it would simply mean that risky loans become impossible to get and thus that entrepreneurship would stop. Even in such a scenario, it seems likely that non-risky business models would continue the way they are, and thus that employement would stay high. Everyone would be employed, but they wouldn't be productive.

So, why not GDP instead? Intuitively, if the GDP grows at all, in real terms, then I think we've avoided anything like what I think of as a catastrophe. What scares me is the prospect of GDP decreasing. That's the kind of dire prospect that would cause me to support some sort of desperate effort at the federal level.

Maybe I misunderstand something basic, though. Why did you pick unemployment?

ryan yin writes:

Unemployment has already gone up a bit, and we were already increasing how likely we think a recession is, so even before the current crisis, hitting 8% didn't seem so improbable. So I would think that if we have a bailout and we don't go past 8% for the next 2 years, I'd be more sure that we didn't need a bailout (ex post). But I don't know if that's the cutline. At >10%, sure, I'd probably be more certain the bailout was necessary, but between 8 and 9%, and I don't know if I'd have gained new information one way or the other. (I don't think this violates Bayes' Rule, but if someone can explain why it does, I'll update :-)

I wanted to add that we should be careful about misinterpreting the meaning of being wrong (in an ex post sense) about the bailout. Usually when we buy insurance, we end up not needing it, but this doesn't mean you shouldn't buy collision insurance because it has a negative expected value. It's useful to pin down what people think, of course.

Les writes:

It seems to me that the question depends on what the specific bailout provisions are. In essence the crisis was largely caused by the politicians.

If the bailout means that Fannie and Freddie will be liquidated over a couple of years, and the politicians get out of the way and let the mortgage market alone, then I think economic conditions will gradually improve.

But if the bailout keeps the hand of politicians in the till, then I think it will take very much longer for economic conditions to improve.

liberty writes:

I think its more complex than that.

If the chicken littles are right and the economy is going down right now, and is not just in need of some short term restructuring, we would have expected without the bailout for the unemployment rate to rise above, say 8%, and stay that way for some time. The bailout could, in the short run, prevent that; or it could exacerbate the problem and extend the period of unemployment.

If the pro-bailout folk are right, it could prevent high unemployment it in the long-run too.

On the other hand, if the chicken littles are wrong, then the unemployment rate would not rise above 8% even without the bailout in the long run, but might still rise that high in the short run in order to allow for some necessary restructuring - say 6 months or so?

The bailout, though, could still exacerbate the current problems (which require restructuring) and drive the unemployment rate up if it never would have gone that high. Or it might not.

So, looking at a post-bailout (within 6 months) unemployment rate above 8% could mean:

(a) There was a small crisis and the bailout didn't help (or only helped a little).
(b) There was a massive crisis but the bailout kept it at bay.
(c) There was not a crisis and the bailout created one.
(d) There was a need for restructuring, but the bailout screwed up the proper restructuring without actually reducing the unhappy effects of it (unemployment).

Looking at a post-bailout unemployment rate below 8% could mean:

(a) There was no crisis and the bailout didn't make things worse in the short run (or made them only slightly worse).
(b) There was a crisis and the bailout kept it at bay, in the short run.
(c) The restructuring didn't require high unemployment, but the bailout prevented necessary restructuring anyway.
(d) The restructuring did require high unemployment, which the bailout prevented, but it also prevented necessary restructuring which means it will come back to bite harder next time.

Or something along those lines... so, I am not sure that a single figure like unemployment rate at some time t after the bailout can tell us very much.

Steve Roth writes:


1. More banks fail prior to the bailout, and

2. We have a bailout, and

3. No banks fail after the bailout,

I'll consider the bailout necessary, and a success.

Not a terribly risky position, I'll admit—except that a single bank failure after the bailout would prove me wrong.

Could perhaps be tweaked to specify a certain number of failing banks before/after a bailout.

Dog of Justice writes:


1. More banks fail prior to the bailout, and

2. We have a bailout, and

3. No banks fail after the bailout,

I'll consider the bailout necessary, and a success.

Um. Are you claiming none of the banks still standing have made the sort of "heads I win, tails please bail me out" -EV bets that mean they SHOULD fail, lest the wrong precedent be set and we sow the seeds for an even greater disaster many years down the line?

The question is whether the REST of the economy is mostly protected from the consequences of bankers' irresponsibility. If half our banking sector fails, but the remaining half proves to be enough to extend adequate capital to qualified individuals and businesses, that's a GOOD thing. So Bryan's unemployment metric is far better than yours. (10% might be a more informative cutoff than 8%, though.)

PJens writes:

We will know if the bailout worked by the end of the year.

If the bailout works as intended:

1) The stock market indicies (DOW and S&P) will end the year on an upswing. Maybe not reaching the values of Jan 2, 2008, but if the trend is up, the bailout is working. If the values of the stock market is declining, the bailout has failed.


2) Small business continue to have access to loans and small business start ups rise.

The government bailout is to lenders. Small businesses need lending to start up and operate. As long as that credit is available (and I believe it will be available even without government intervention), the economy will progress.

The primary effect of government intervention is to calm investors, primarily people who have retirement accounts tied to the stock market.

That is why I say the bailout will be a success if the stock market regains investor confidence and those close to retirement keep their money invested in private company stock versus government bonds.

8 writes:

What is the point of this bailout again? The proponents keep saying it is to prevent a Depression or something similar, which sounds like 15%+ unemployment to me. They keep saying this is to unfreeze the credit markets, not fix the economy. I expect 8% unemployment no matter what, at best.

If only 1 major bank fails (FDIC assumes operations or FDIC midwifes a takeover after the bank is insolvent but before the market opens), then it was a success. If less than 5 regional banks fail, it is a success. If more than that fail in the next six months, it is a failure. If it delays the failures by 6 to 12 months or more, then it made things worse.

Adam writes:

The bailout may well affect employment, yet have little effect on economic performance. Certainly, if the bailout is effective as a transfer of debt, fewer financial firms will go out of business and unemployment will be less in the financial centers. On the other hand, the increase in unemployment due to the bailout will be diffuse and in the future, due to the heavier burden of taxation as the debt is paid or due to higher interest rates and crowding out as the Federal govt borrows even more. Its the Bastiat difference between the seen and unseen.

A better distinction between the success or failure of the bailout would be whether the govt makes money on the purchased debt. If the govt makes money, that is evidence that the markets are not pricing efficiently--or in layman's terms, are panicked and making bad decisions. If the govt loses money, that is evidence that the basic irrationality hypothesis is wrong and the bailout is a failure.

I don't support the bailout because I view it as a simple shift of debt to the govt from poorly conceived financial entities. However, if the govt makes money on the debt, say, after 2 years of holding the debt, I'll be more inclined to view the bailout as a success. Of course, there are caveats. Making money on the debt has to be on average, not just on a subset of bonds. Also, we'll need some honest auditors to determine whether there was a gain or loss!

Wishing you the best in these difficult times,


rpl writes:


As long as we are looking at it from a Bayesian perspective, then we have to acknowledge that with only a single data point our posterior is going to look an awful lot like our prior, unless the lone data point was wildly improbable under the prior.

Thus, for example, if you think that a given coin is fair, and you evaluate it after just a single flip, then you are still going to think it is fair after the experiment. Likewise, if you think it is weighted to come up heads 75% of the time, that opinion too will be mostly unchanged by the lone flip. An uninformed prior (e.g. any value of p(heads) equally likely) would remain mostly uninformed after the experiment. Only if you believed it was a double-headed coin and it came up tails would your beliefs be changed significantly.

In the case of the bailout, there is only one scenario that looks to me like the "(putatively) two-headed coin comes up tails" case above, and that is the one where we have no bailout, and the economy is still "ok" (whatever that means) afterward. That would well and truly bust the "bailout or doom" prior. Any other outcome is broadly consistent with pretty much any prior, and so we would expect people to come out thinking pretty much the same as what they thought going in.

Nick Rowe writes:

I understand what "the probability of no disaster conditional on a bailout" means: P(-D/B)

I understand what "the probability of no disaster conditional on no bailout" means: P(-D/-B)

But what does "the probability that a bailout prevented a disaster" mean? Is it P(-D/B)-P(-D/-B)?

Nature tosses two coins (maybe bent ones). The coin tosses are independent. The first coin tells us whether there is a disease which needs fixing; the second coin tells us whether the cure will work. If the first coin is tails there will be no disaster, regardless of the second coin. If both coins are heads, the bailout works and was needed, so there is no disaster. If the first coin was heads, and the second tails, the bailout fails, and there is a disaster.

I don't observe the outcomes of the coin tosses, but do observe whether or not there is a disaster.
HH means -D
HT means D
TH means -D
TT means -D
So, observing -D (low unemployment) means that either HH or TH or TT happened.

If both coins are fair (this roughly matches my priors), then HH, TH, and TT are equally likely, so each has a 1/3 probability. The probability that the first coin was heads P(H./-D) is now 1/3, and the probability that the second coin was heads P(.H/-D) is 2/3. The probability that both were heads P(HH/-D) is 1/3. So my posterior probability (after observing no disaster) that the patient had the disease and the cure worked (that the bailout prevented disaster) is 1/3.

My prior probability (before observing whether or not there's a disaster) that the patient had the disease and the cure would work, (that the bailout would prevent disaster) was P(HH)=1/4.

So observing no disaster would increase my probability that the bailout prevented a disaster from 1/4 to 1/3.

Did I make a mistake anywhere?

Bryan, I really don't see how observing no disaster could lower your probability that the cure prevented a disaster??

The only way we would know if this bailout works is if we have two identical economies in which one gets the bailout, and the other does not.

A close second would be to look at other times when something similar happened and the government did nothing one time, and did something similar to this at another time.

A third option: take the ethical physician's approach -- when in doubt, do nothing. Unfortunately, too many who should doubt don't; the government is full of economics quacks, and the media is interviewing only economists who agree with the quacks. This resembles the beginnings of modern medicine: "Wash your hands? Don't be ridiculous!"

RobbL writes:

Ethical physicians are in doubt much of the time and it does not stop them from trying various treatments...just sayin'

Comments for this entry have been closed
Return to top