Bryan Caplan  

Sampling Bias on a Plane

PRINT
Questions About Blogging... Capitalism, Democracy, and Pol...

Walking the halls of Comic-Con and GenCon, I repeatedly heard voices gush: "Snakes on a Plane is going to make a TON of money. Everyone I know is going to see it!!!"

The numbers are in: SoaP's domestic gross for opening weekend was $15M - less than a bunch of movies this year that I've barely heard of.

How is this possible? The answer, of course, is sampling bias. Your friends are not a random sample of Americans. In fact, they could easily be a highly unusual sample of Americans (or maybe not Americans at all). Well, to be honest, if you're reading this blog, it's virtually a sure thing that your friends are a highly unusual sample.

When you generalize from a random sample to a population, your inferences are highly likely to be correct. If 100% of people randomly sampled know who the current U.S. president is, you can reasonably bet that almost everyone knows who the current U.S. president is. But if 100% of the people you know say they plan to see Snakes on a Plane, you need to do some hard thinking. How close to a random sample are my friends? How does their education/age/gender/IQ/geekiness/etc. compare to the average American's?

If you realize that your friends are far from average in a bunch of ways, you've then got to ponder whether these abnormal characteristics predict abnormal behavior. And needless to say, attending Comic-Con and GenCon is abnormal behavior.

So it's no surprise that SoaP fell so far short of expectations. But I have to confess that, though I heavily adjusted for sampling bias when I guessed its opening gross, my adjustment was way too small. I reasoned: "If you believe the geeks, it will make $100M this weekend. That's way too high - $50 M. OK, make that $45M with the R-rating." Perhaps before I guessed I should have taken a second look at my pictures from GenCon!

My favorite example of sampling bias used to be people who believed that the 1972 Nixon election was fixed because "I don't know anyone who voted for him." Thanks to Snakes on a Plane, I can now update my lectures.

P.S. I will still be seeing SoaP later this week. Now is no time to distance myself from Nerd Pride.


Comments and Sharing





COMMENTS (8 to date)
Barbar writes:

My favorite example of sampling bias used to be people who believed that the 1972 Nixon election was fixed because "I don't know anyone who voted for him."

Interesting example. Who were these people, exactly? You're probably thinking about the "Nixon quote" mentioned here:

http://en.wikipedia.org/wiki/Pauline_Kael

But it's a good example, because lefties are out of touch with mainstream America.

Jessica writes:

This post - among others! - would seem to suggest a correlation between being a libertarian economist and being a die-hard sci-fi/fantasy geek. Does your experience support this anecdotal observation, and if so, can you elaborate on the possible causation?

RWP writes:

Maybe it is the simplistically stupid name of the film. Why would I go see a film that tells me what is going to happen? Its like Air Force One with snakes. Where is Stephen Seagal when you need him? Glad it is a flop.

Eric Crampton writes:

Snakes on a Plane opens here in NZ only on the 24th. And none of my RuneQuest group's interested in going. Sigh. Will have to go with econgeeks instead.

Jessica: underlying genetic fitness is what's driving things. Those who are the most fit are likely to be better on all margins. Good genes predict being a libertarian economist; good genes predict being a sci-fi/fantasy geek. And so the two variables are observed to be correlated because they're both driven by the underlying variable. At least that's my hypothesis. I'm sure that the empirics will back me up if the study is done properly.

Eric Hanneken writes:

By all appearances, Snakes on a Plane is just another dumb action movie. I don't understand why some people--even self-described "geeks"--were excited about it. Yes, the title was amusing when I first heard about it. Is that it? Is that the reason for the T-shirts, YouTube parodies, blogs, etc.? Are we supposed to spend money to see the film just to extend the joke?

Robert Cote writes:

Why stop at Nixon? Dewey Defeats Truman. The liberal urban media didn't know anyone voting for Truman either. Too old? Then how about an example that is more insidious: Newsweek putting candidate Bill Clinton on the cover after coming in SECOND in New Hampshire.

As to econgeek/scifigeek associations I prefer to think that both preferentially explore alternative possibilities. Williness to transcend the simple, obvious and incorrect answer is an overlap of interests not a correlation.

Nacim writes:

I reluctantly joined my friends in seeing the movie Saturday night. I was honestly expecting nothing besides a few extensions of the joke. I was so wrong. Snakes on a Plane was definately one of the most entertaining movies I have ever seen. It's an awesome mix of humor and just plain over-the-top silliness. It was actually a good movie. Highly recommended!

Bob Hawkins writes:

The day after the 1984 presidential election, the Washington Post ran a story with this lede: "Northwest DC is full of people who voted for Mondale, and everyone they know voted for Mondale. They woke up this morning to discover that there are enough people they don't know, to elect Reagan."

Comments for this entry have been closed
Return to top