Bryan Caplan  

What Counts as Evidence that the Bailout Worked?

Priceless Macro, Part Two... Grist for Arnold's Mill...
Back in September, I asked, "How would we know if the bail-out worked?"  "Wait and see" doesn't cut it, because:
If the bail-out happens, how will we know if it prevented disaster? If unemployment stays under 8%, proponents will say that the bail-out prevented a recession. But if unemployment hits 8% or higher, proponents will say that things would have been even worse without the bail-out. Opponents, of course, will flip things: Good conditions mean the bail-out wasn't necessary, bad conditions mean that the bail-out made things worse...

Unfortunately, if you let people see whether X happens before they update, you can't show that they're being bad Bayesians. You only see how they updated, not how they would have updated if the news had been different.

So here's my request: Tell us how you're going to update in advance.
I accepted my own challenge, saying in advance that good outcomes undermine the case for the bail-out, while bad outcomes reinforce it.  But alas, almost no one accept my challenge, which is why we can now read stuff like this Steven Pearlstein column.  Who benefited from the bail-out?  According to Pearlstein:
Every investor, every household and every business in the United States. You may not like the fact that, as a result of these actions, overpaid bankers were allowed to hang on to their jobs or preserve the value of their stock holdings. And you may be unhappy that the financial system remains in such fragile shape that it is still hard for some people and businesses to get loans they think they deserve. But let me assure you that things would have been a whole lot worse if these actions had not been taken.
It's possible, I admit, that Pearlstein thought all along that "Good outcomes raise the probability that the bail-out is a good idea; bad outcomes reduce this probability."*    But I have to doubt it.  If the banking system had tanked despite the bail-out, would he now have a more skeptical view about bail-outs?  Or just be saying, "It was too little, too late"? 

At risk of Bayesian self-righteousness, I'd like to trumpet, "If you didn't tell us in advance what would count as a vindication of your position, you don't have any business patting yourself on the back and insisting that events proved you right."

* If someone digs up a pre-November Pearlstein statement to this effect, I apologize to him in advance.

Comments and Sharing

COMMENTS (12 to date)
Robin Hanson writes:

Yup, absolutely right.

pushmedia1 writes:

My guess was that good news was bad news for the effectiveness of the bailout. Specifically, I said unemployment rates above 7.8% would make me more sympathetic to bailouts. We're getting close but we're not there yet.

greenish writes:

You could also have committed to the outcome having a certain evidential weight - as in, whether it would be a big piece of evidence, a small piece of evidence, the only evidence you need, or completely useless as evidence. I chose "hardly any", because I had no idea how damaging the bailout would be, so both outcomes fit my views.

Nick Rowe writes:

Bryan: here's my comment from September:

"So observing no disaster would increase my probability that the bailout prevented a disaster from 1/4 to 1/3."

So I have now slightly increased my belief that the bailout prevented disaster.

I also showed how I derived that answer (though I'm not sure if I did it right).

Now, here's my challenge to you: can you show me how you could derive your result, using Bayes' theorem (or whatever), that observing NO disaster would REDUCE your belief that the bailout prevented disaster? Because I just don't get it.

8 writes:

I thought the people pushing the bailout (Paulson, Bush, Bernanke) argued that it was needed to prevent a collapse in the financial system and total freeze of credit markets. They said that would cause unemployment and other bad stuff, but I took that to be a secondary argument. They didn't say it would prevent a recession or actually improve employment.

Did the bailout work? Maybe it did, since the financial system didn't collapse yet. I think the bailout misoverallocates capital to failed financial and industrial firms and will lead to higher unemployment, lower GDP, and a longer depression.

Basically Bush & Co. used the Sledge Hammer approach to problem solving,
In the pilot episode, he deals with a sniper on a roof by blowing up the entire building with a rocket launcher, then turns to the uniformed cops on scene and says "I think I got 'em";

Billy writes:

What do you make of this?

Michael Rulle writes:

We can never know the impact the bail out has. But this is no different then any other social problem we face.

But our best guess, however, should be consistent with absolute conditions. I do not understand Brian's "prior". It seems as if he said that if conditions improved we did not need it, implying I think that it had no negative impact; or that it was lower than the natural positives in the economy. But if things got worse it means we did need it implying it improved conditions from what would have been worse. This seems backward to me and illiogical (what do I know, anyway).

It seems like the following is better. The Paulson crowd "prior" is that the bailout would make things better. Claiming, before or after the fact, that when things got worse it would have been even more bad seems like cheating to me. The purpose was to make things "good" not less bad. If conditions became "good", our best guess is we cannot reject the null that "the bailout worked". Pearlstein's claim even if said before hand demonstrates nothing. What is the Null? What if the Null were "the bailout worked if unemployment were less than 10% by December"? Cannot reject it but it seems empty of content given that liklihood is so low. If I were to have defined "bailout working", it would have been something consistent with an average recession or better. I justify this based on the fact that the Bushies were forecasting doom if we did not do it. But it seems like we have the very doom they were forestalling. Does that mean it caused it?

That could have been the prior. In fact, on my blog as they were putting out a crazy idea a day, I was saying they were making it worse. Once Paulson changed his tune and bought preferred, I said it was not helping and probably hurting.

Of course we should have known immediately this was all nonsense. The solution to doom in September was to buy mortgages, according to Paulson. The action taken was to buy bank preferred stock instead. We need only observe the randomness of the actors themselves to know the bailout did not work. My "prior" is that when the "creators of a bailout act in a random fashion", it will not work. Not working means "things get worse" relative to other economic crisis of similar measurable magnitude (1990-92). They never had a clear belief to begin with, so why should one assume it should work? Might as well believe in voo doo.

Gary writes:

Suppose you have a prior that smoking is harmful to human health. Then you meet a smoker who is 50 years old and has been smoking all his life. If the person dies at 55, would it be reasonable to think that he would have lived even longer if he had not been a smoker? If he dies at 100, would it be reasonable to think the same?

Bob Murphy writes:

It's funny Bryan, because I thought you were going to say that you had to admit the bailout helped, since the economy still is in the toilet and we still have "zombie banks" (or so Tyler tells us every other day at MR). I hope you objectively defined success and failure back in your earlier post too. :)

Let me put it this way: Suppose Paulson & Co. said back when the TARP first failed, "C'mon now guys, let's get with it! By the way, even if you pass this, most analysts will still be calling for an additional $800 billion stimulus package in 3 months." Do you think that would have increased or decreased support for TARP?

Bob Murphy writes:

Another thing: I get where you are coming from, Bryan, but I agree with Nick Rowe; it is a bit weird that you end up saying the evidence of no disaster would make you think the TARP didn't work, when it was sold as a plan to prevent disaster.

Let's switch to standard welfare. Conservative critics of LBJ like to say that the poverty rate was falling faster before Great Society than after it. I.e. relative to the baseline, they are arguing that welfare recipients are poorer after getting LBJ's bailout money.

So you're saying this is backwards? That if poverty had been eliminated by 1980, then *that* would have been a crushing blow to the Great Society, since it obviously wasn't necessary?

El Presidente writes:

Did it work? Did it fail? Does it matter?

Bryan is right to say that we'd have to have some sort of benchmark or goalpost against which to judge it, and that judging predictions with integrity requires that we develop the standard prior to implementation of the program.

Determining whether the actors had integrity and brought about the consequence they intended is less important to me than figuring out what consequences did occur and what they portend for our immediate next steps and our longer term policy landscape.

Before the bailout vote, I put it to a forecaster this way:

"If we are in a liquidity trap of sorts, will dumping money into the vaults of the banks that are declining to lend help the situation?"

His response in front of 500 people:

"That's an impossibly difficult question to answer, but I'll try. It'll tend to make the dip slower and shallower, but there's really no telling what it will actually accomplish."

Very unsatisfying, but probably as good a guess as we could have made at that time. There was just no way to know what would or would not happen because we were, and continue to be, flying blind with regard to the leverage positions of many large market players. If we don't know what their stakes are, we can't estimate their payoffs and can't reasonably evaluate their potential alternatives to see which ones they might select.

This is where I believe good intentions lead libertarians into dark places. Policies have consequences. Being blind as to their impacts on individuals does not mean that they will have no impacts. Some will be negative, and some will bepositive. We should really either avoid action altogether (a very unrealistic approach) or have greater disclosure about who is affected by our policies, and how. That means actors who play in markets that affect the wellbeing of a great many other people should have to make greater disclosure. A right to property and a right to privacy combined lead to a right to anonymous menace. I'm in favor of the first two, but leery of the third. We need to balance them better through regulation.

Since the point of the bailout was to change the behavior of individual actors and thus aggregate outcomes, details matter. We didn't know them then. We don't know them now. Guessing is the best we can do. When we know everybody's guessing, are we really going to concern ourselves with distributing merit badges to those who guessed right?

Jovan I. writes:

The government is positive in their resolution to this ailing economy. This is to assure that there is still hope for the economy to prosper. But the problem would be that the people are suffering too much with the long running recession and this changing inflation and deflation. The Treasury Department released a report of its activities, having allocated $386 million to 23 banks. The federal government has been reacting to the call for more transparency in what is done with our money, the kind of disclosure you get from a lender of payday advance loans, after half of the bailout fund disappeared before its usage was tracked. Banks across the nation and Puerto Rico have been receiving Treasury funds to keep them afloat, like payday advance loans. Read more about payday loans for banks at the Money Blog.

Comments for this entry have been closed
Return to top