Arnold Kling  

Survey Availability Bias

PRINT
What Would a Hedonistic Econom... Lecture on Mechanism Design...

What do the following beliefs have in common?

A. Belief that Kerry won in 2004, and that a vote-counting conspiracy took place.

B. Belief that men have more sex partners than women.

C. Belief that epidemiology is equivalent to a clinical trial.

D. Belief that happiness does not rise with income beyond a certain point.

In each case, people are believing survey results rather than other indicators of phenomena.

In the case of the election, people believe the exit polls instead of the election results, even though that means you have to believe that there was a widespread conspiracy to miscount votes that took place across numerous different precincts, election supervisors, and voting mechanisms.

People believe surveys which say that men have many more heterosexual sex partners, even though this is not mathematically possible.

Newspapers eagerly report epidemiological study results, even though time and time again such results fail to hold up in clinical trials.

And some economists take seriously the notion that people are not happier at higher income levels, even as they point out that people have a choice of whether to earn higher income or take more leisure.

I call this "survey availability bias," because the availability of a survey result makes us want to comment, analyze, and interpret the survey, even though the most plausible interpretation is that the survey itself is flawed.

In my view, survey availability bias creates large negative externalities. Surveys add noise rather than signal to our society. If there were less survey availability bias, people would not be so prone to report, pass along, analyze, and comment on surveys. Survey-takers would be less well rewarded, and I believe we would all be better off.

We need a large Pigovian tax on surveys.


Comments and Sharing





COMMENTS (9 to date)
liberty writes:

"People believe surveys which say that men have many more heterosexual sex partners, even though this is not mathematically possible."

The common answer to this is the following: more men are supposed to have many partners, with the remaining men having very few. Fewer women are supposed to have *very* many partners with the remaining majority having few. So "most men" have more partners than "most women" supporting a general rule that men have more sexual partners than women.

This accounts for the virgin/whore dichotomy, or rather, the virgin/whore dichotomy accounts for the tendency toward this bias. You can come up with mathematically feasible scenarios to prove your bias. The much simpler and more realistic scenario, of course, is that people lie, and a societal bias makes people more likely to lie.

Chuck writes:

"And some economists take seriously the notion that people are not happier at higher income levels, even as they point out that people have a choice of whether to earn higher income or take more leisure."

You seem to premise all your views on the assumption of rationality and wisdom in human choices.

dearieme writes:

"And some economists take seriously the notion that people are not happier at higher income levels"; no doubt that is why Economists are famous for accepting the quasi-Nobel Prize in Economics but rejecting the sum of money that comes with it.

General Specific writes:

"Surveys add noise rather than signal to our society."

Humans gossip--produce noise. It's an old ancient habit. Our moral behaviors are, at least partially, controlled by it.

Surveys are merely a modern manifestation, specialization, or formalization of this gossiping habit.

As one who performs due dilligence and analysis for many of my personal decisions, I'm amazed at the misinformation and confusion that are used to motivate and ground decisions.

Labeling this "survey availability bias" isn't going to change anything. So you might as well call it what it is: gossip.

I have a question: do economists, or only certain classes of economists, have a overly-strong tendency towards labeling phenomenon as bias? Is that a kind of bias? What would it be called? Bias-bias?

caveat bettor writes:

Fewer surveys. More prediction markets!

Heather writes:

I think your argument for happiness versus income relationships is flawed. The assumption is that happiness is the only motivator for earning more income. From the studies I have seen on happiness, it generally relates primarily to stress levels, which are not necessarily economic in nature, although they can be. This means that the survey statement is probably accurate for this fuzzy measurement.

Kimmitt writes:

I didn't know utility was cardinal.

Barkley Rosser writes:

Regarding Kerry in 2004, the suspicion has had to do with the fact that the official in charge at the state level was very pro-Bush and some of the precincts where vote patterns changed in Bush's favor were in places where officials were pro-Bush.

Regarding men vs. women, the first commenter has it. It is a matter of how it is stated. If one says, "the average man has more sexual partners than the average woman," this is correct due to the virgin/whore dichotomy. However, of course, mathematically, the expected number of sex partners a member of each sex will have must be identical.

Regarding the happiness-income relation, it is in fact positive within a society at a point in time. Relative income effects are clearly there, and the rich enjoy being richer than others, while the poor do not like being poorer than others. The data gets much messier when one starts to look across societies or in a society over time, the latter providing the basis for the original Easterlin Paradox (incomes rose enormously in Japan, but people did not get happier; people were reported to be happiest in the US in 1956, and so forth), although there has been this recent, much touted, if hardly definitive study by Pew.

So, some of your examples are a bit weak (although I suspect that Bush really did win Ohio in 2004, even though there is strong evidence that Gore really did win Florida in 2000).

Barkley Rosser writes:

Oh, on that sex partner thing, that assumes heterosexual relations. The evidence is pretty strong that male gays have many more partners on average than lesbians. Hence, looking at all men and women, men do have more sex partners than women on average overall.

Comments for this entry have been closed
Return to top