Arnold Kling  

Standards of Proof

PRINT
Arnold's Standards of Science:... Questions for Arnold...

Bryan writes,


isn't Arnold's standard of scientific evidence strangely strict? Few results in economics rest on an "experiment or a naturally-occurring event where the results are extremely unlikely to occur unless X is true."

I thought I was simply stating in layman's terms the definition of statistical significance. That is the most commonly used approach in empirical work in economics. When we say, "I found that X has a significant effect on Y," we are saying that the probability of observing the sample results would be low if X had no effect on Y.

As an aside, these formal statistical tests often are not strict enough, in my opinion, because of specification searches (see Leamer's book of that title).

But take an example that uses no econometric(k)s. Gregory Clark wants to argue that in the 1800's British workers were much more productive than Indian workers. He asks that if British and Indian workers were close substitutes, why would you ship cotton from India to Britain, turn it into cloth, and ship the cloth back to India? Instead, if they were close substitutes, you would observe English companies building cloth factories in India in order to serve the Indian market more cheaply. In other words, shipping cotton to England and cloth back to India is something that is unlikely to occur if workers are close substitutes. Instead, it provides evidence, meeting my "strangely strict" standards, that British workers were more productive.

That is an example of a persuasive scientific argument.

As to global warming, I do not want to bet on global temperature going forward, because that is not the nature of my skepticism. I have no strong priors about the distribution of possible global temperatures going forward.

My skepticism is about the effect of man-made CO2 on global warming. If Bryan or anyone else can devise a bet that focuses on that parameter, then that would allow me to put money on my skepticism.


Comments and Sharing





COMMENTS (18 to date)
TGGP writes:

Don't they have conditional bets for that sort of thing?

Eric Crampton writes:

Arnold, here's the ticket.

Take a piece of paper. Split it in four. Write on them as follows:

A. Pays $100 if average global temperatures for the decade 2050-2059 are at least one degree centigrade above average global temperatures for the decade 2000-2009 AND global carbon emissions have risen by 20% or more.

B. Pays $100 if average global temperatures for the decade 2050-2059 are at least one degree centigrade above average global temperatures for the decade 2000-2009 AND global carbon emissions have NOT risen by 20% or more.

C. Pays $100 if average global temperatures for the decade 2050-2059 are NOT at least one degree centigrade above average global temperatures for the decade 2000-2009 AND global carbon emissions have risen by 20% or more.

D. Pays $100 if average global temperatures for the decade 2050-2059 are NOT at least one degree centigrade above average global temperatures for the decade 2000-2009 AND global carbon emissions have NOT risen by 20% or more.

Calibrate the values above as you see fit in terms of time span or temperature/carbon rise. For each of the above, read "$100" as meaning "ownership of a 50 year government bond bought in 2008 for $100".

The set of contracts spans the space and so is worth $100. Then, sell the contract bundle where warming is contingent on carbon: A & D. Somebody who puts better than even odds on carbon affecting warming ought pay better than fair odds for the A&D bundle.

Dr. T` writes:

There are two problems with Eric Crampton's betting scheme. 1. We cannot measure accurately man-made carbon dioxide emissions. 2. When global temperatures rise from any cause, water temperatures rise, and dissolved carbon dioxide outgasses into the air. (Some climatologists believe that the historical correlations between atmospheric CO2 concentrations and global temperatures are effect and cause, not cause and effect.)

Eric Crampton writes:

I don't worry too much about critique #1 as it cuts evenly across all state spaces.

#2 would completely invalidate it though. Back to the drawing board.

Fazal Majid writes:

Econ 101 - Ricardo's theory of comparative advantage shows why you cannot conclude that British textile workers were more productive than Indian ones simply from the fact there is an export flow. Indian mill workers (or weather) might just be too good at growing cotton to waste on weaving cloth. That theory is as old as the cotton trade in question.

Thus you would need additional evidence to support your conclusion.

In addition, statistical significance tests show correlation but not causation. Any use otherwise is junk science at best. A fallacy that is all too common is the soft pseudo-sciences, economics included.

John writes:

Here is the bottom line problem with the THEORY that human activity related CO2 is causing global warming... It does not explain previous warming and cooling cycles of the earth where human activity (CO2) did not contribute to the phenomena.

It is just as reasonable, in fact more reasonable, to conclude that whatever lead to the medieval warming period, the holocene maximum warming 6000 years ago, etc. is causing the warming that is now taking place. That "factor" is obviously not human activity.

Thus, scientific consensus means nothing since we cannot exclude that the cause(s) of past warming periods as the current cause of global warming.

Arnold is thus right to be skeptical "scientific concensus".

caveatbettor writes:

There's always the Global Warming Exchange at http://globalwarming.inklingmarkets.com/market/show/6699

Mason writes:

As I see the global argument there are 3 main points.

1 – CO2 is a greenhouse gas, meaning that pure CO2 traps more heat than our current atmospheric mix.

2 – Human activity is releasing a significant amount of CO2 into the atmosphere and this is causing the temperature to rise,

3. Generally left unspoken - rising temperatures are bad.

Point one is hard science and few question it. Point two is much more questionable, and point 3 is more questionable still.

We do produce lots of CO2, but there is already a lot in the air, how great an effect can we have? This goes along with the natural fluctuations argument.

Nature has a built in system for storing CO2, it takes along time to kick in, but why worry about it when it will take care of itself?

Before we can say global warming is bad shouldn’t we have an ideal temperature in mind? Assuming that current/historical temperatures are best is a little lazy/ naive.

Fazal – nice work.

Mason writes:

All of that said, I think we need to bring oxygen into the economy, some people produce it, (farmers, forest dwellers) some people use it (car drivers). We pay for other goods that we consume and produce, just because oxygen is easy to steal doesn’t mean we should.

If this sounds ridiculous think about how ridicules owning land sounded to Native Americans. Bringing goods into an economy doesn’t create problems it solves them.

Measuring oxygen consumption is easy to do; it is used in fixed proportions to fuels burned (only burning fossil fuel has a net effect on oxygen consumption/CO2 production). I agree that measuring oxygen production is a little trickery, but even a rough guess is better than our current situation.

As far as the bet goes, Arnold - you’re a smart guy if you wanted to put your money where your mouth is you’d find a way.

Daublin writes:

Mason, maddeningly, your point #1 is not really believed by scientists, even though it is frequently presented in pop climate change. You wrote:


1 – CO2 is a greenhouse gas, meaning that pure CO2 traps more heat than our current atmospheric mix.

Trapping heat--the greenhouse effect--can only heat things so much, an in fact most of that warming has already happened due to the current CO2 concentrations. CO2 can go up 10 times greater than now and the greenhouse effect will not go up very much.

Thus, scientists who believe CO2 will cause runaway warming are thinking of something other than the greenhouse effect. What that is, I am not clear on. They always just say, look at our complicated models. Can anyone clear the record for me or Arnold? Is there a simple explanation of how CO2 is supposed to cause massive warming?

Of course, among most non-scientists, it's all about the greenhouse effect.

Mason writes:

Daublin - maddeningly, you admit what you are denying. You wrote:

"CO2 can go up 10 times greater than now and the greenhouse effect will not go up very much."

I don't see how this is different from what I said. (I said it would go up, you said it would not go up very much, which means that it will go up some). Plus I don’t think anyone is saying there will be "massive warming", worst case predictions are for an increase of a few degrees.

Ben Kalafut writes:

Since there's a long time lag between CO2 emissions and associated warming, conditional bets aren't as needed as one may think they are, if you really want to bet on the veracity of the AGW theory per se.

I'd bet $100 w/Arnold, however, that, barring human remediation, if atmospheric CO2 is above 400 ppm, the average surface ocean pH will be below 8.05 in 2040.

Jeff writes:

When we say, "I found that X has a significant effect on Y," we are saying that the probability of observing the sample results would be low if X had no effect on Y.

That's not really correct. What's usually done is something like this: (i) You propose a model that could have generated the observed Y, a model which does not include X. The null hypothesis is that this model is correct. (ii) A second model is proposed that does include X. (iii) Both models are fitted to the data and a test statistic that measures the difference in fit between the two models is generated (e.g., a likelihood ratio or Wald test). If the observed value of the test statistic is unlikely under the null hypothesis, you might say that X has a significant effect on Y.

Of course, that statement is not really true. All you've really done is found evidence that your first model is wrong. It may be that the second model is less wrong than the first, but that's a long way from saying it's correct.

Barkley Rosser writes:

Willingness to bet on arguments and forecasts is partly a function a) how much money one has, b) how risk loving one is, and c) how much a posturing macho asshole one is.

So, pretty much irrelevant to any serious discussion of scientific or intellectual issues.

Barkley Rosser writes:

Just for the record: I do not consider either Bryan or Arnold to be a******s. But, the point remains in general about the use of "willingness to bet" as some criterion regarding the truth of an argument.

(And if one wants to say, "ah, the point is making or losing money on the bet," well, given all the uncertainties surrounding the global climate forecasts, this becomes more a matter of luck rather than knowledge or brains.)

Ben Kalafut writes:

Some of us (even poor grad students like myself) would rather spend the money being part of the solution than betting, but without a meaningful cap-and-trade system, that can't be done.

Morgan writes:

The actual bet, of course, isn't $100. Its the lives of your children, grandchildren, all your relatives and all humans. Now the percentage chance of total catastrophe does not have to be very high before the expected loss from inaction is very great indeed.

Mike writes:

"we are saying that the probability of observing the sample results would be low if X had no effect on Y."

- Hmm, no we are not. Regression results tells you about conditional correlations. In economics we usually have models that states that x-->y, hence when economists finds an effect they say that x causes y (which a paper in e.g. epidemiology never could get away with). But, we still only have correlations from the empirical work.

Further, for Arnold's comment to be true, it is of course also important to know the magnitude of the effect. As Arnold surely knows, there are plentiful of more or less worthless papers in economics reporting statistical significant results, which have a very low "economic" significance.

Comments for this entry have been closed
Return to top