Why Experiments are Misleading

Perhaps also the pyschological elements serve as signals whose information content is diluted as more and more firms use them.

"But if that's true, then wouldn't every bank already be maxing out on psychological elements?"

This strikes me as one of those ridiculous sentences that only an economist could come up with. We have this strange instinct that no new information can possibly be both true and useful, because if it were, people would already have exploited it.

But banks would already be maxing out on psychological elements only if what Mullainathan says is true and banks know it's true. The thing is, if they knew it were true already, then they would have had no need to be involved in this experiment.

I agree with conchis whole-heartedly.

Given what we know about psychology.. the assumption that any firm is "maxing out on psychological elements" in almost any industry.. is shaky, at best.

Why do the previous commentors find it hard to believe banks may already be aware of these "psychological effects"? How much money do banks spend developing marketing materials and strategies?

We don't care if banks explicitly know these effects. How do they behave? How do they allocate their resources?

We should also take note that this experiment was not conducted in a US market.

At the risk of sounding a bit un-PC, I think the key phrase is "developing countries" which is where this guys specializes in marketing plans. These people haven't developed marketing burnout yet, unlike consumers in the US who get 3 direct mails and a dozen spams every day offering financial services.

One of the reasons we enjoy looking at vintage ads is that their marketing ploys are so transparent as to be endearing. Grandma bought soap because it FLOATED??? HAHAHA!!!

Not to worry, more than just the pets.com sock puppets will be fodder for the laughter of your grandchildren.

"But if that's true, then wouldn't every bank already be maxing out on psychological elements?"

In every market there will be unsophisticated players. Suckers in other words. So some banks may be ignorant or even a bit "above" these tactics. The savvy player nowadays can search the internet looking for these spreads. For instance, a year ago I was shopping for CDs and seeing some vast differences on rates.

"This strikes me as one of those ridiculous sentences that only an economist could come up with. We have this strange instinct that no new information can possibly be both true and useful, because if it were, people would already have exploited it."

Yes, efficient markets is the butt of the old joke about the two University of Chicago professors who see the ten dollar bill laying on the ground. But somewhere in one of my texts an author points out that it is the action of all the various players in the market, some wise, some not so wise, that creates the efficient market.

I don't doubt that some banks are aware of, and consciously exploit psychological elements. But Bryan's argument requires that "every bank" be doing so, and we have a fairly clear counterexample to that: the bank in question was neither aware of, nor (presumably) investing in this sort of thing.

Unless, of course by "bank" we were supposed to read "US bank".

Even then, the point remains that we don't know whether US banks are already aware of these effects (or, more implausibly, are acting "as if" they were). Whether they are or not is fundamentally an empirical question. But too often economists just assume, as I think Bryan is guilty of doing here, that people/firms must be aware of stuff - because that's what our models say. That strikes me as both a very strange, and quite dangerous way of reasoning.

Bryan echoes the economist in the old joke who won't exert the energy to pick up a dollar in the road, because it can't really be there, because if it were somebody would have picked it up already. In other words, if somebody does some properly conceived and conducted research that conflicts with a prior theory of yours, maybe you should think about the theory rather than simply reiterate it. Here, the problem with the theory seems obvious: Bryan accepts for the sake of argument that the market is made up of irrational humans but not that the banks themselves are no more than communities of the same. There is little about the history of banks and banking anywhere which would lead you to assume that today's banks are temples of rationality - even if theory tells you that they should be, and even if the very worst do indeed go out of business.

I'm not suggesting that more study isn't warranted. In fact I think it would be very interesting, and it is an empiricle question. But in doing such a study in a competitive market my hypothesis would be that the interest rate begins to matter much more. Theory doesn't prevent me from being curious about this, it tells me to be a litte cautious in excepting this one study's implications on all markets. I think that was Caplan's point.

Bryan, your comment doesn't explain your title.

You suggest that all possible practical knowledge is already known and applied in practice. If that were true, then experiments would be pointless rather than misleading.