Bryan Caplan  

Scoring Your Ukraine Predictions

PRINT
Krugman's Priceless Economics... Illegal Means Illegal...
I just reviewed all 50 responses to my original Ukraine prediction challenge.  Seven comments did not make unconditional predictions, and two were obvious jokes.  Here's how I scored the remaining 41 sets of predictions:

1. My last post named six major facts about what happened during the last year:


2. Responses got one point for correctly predicting each of these facts.

3. Response lost one point for predicting anything contradictory to each of these facts.

4. Responses were neither rewarded nor penalized for failing to address these six points, or for making other predictions.  My apologies to everyone who made correct predictions I didn't score; many made good points, but I wanted to keep scoring simple and consistent.

5. I tried to grade the responses blind, but only via deliberate inattention rather than a serious blinding protocol.

The resulting histogram has a mean of -.29 and a standard deviation of 1.08, with a maximum of +2 and a minimum of -4.
ukraine2.jpg

The winner of the Ukraine Prediction Challenge was Hansj√∂rg Walther, with a score of +2 out of a possible +6.  His original forecast:

My prediction is very similar to David Friedman's:

- Russia will secure control of the Crimea (or already has).
- There are protests from Western governments, but nothing serious.
- Then a deal is brokered to let people in the Crimea vote on staying within Ukraine or joining Russia, which is easy to sell as self-determination.
- And the probable outcome would be that the Crimea accedes to Russia.

Putin will not try to gobble up Eastern Ukraine or even Ukraine as a whole. That would be a precedent where Western governments would get nervous (although even in this case my guess is that it would be just talk, but a higher risk something goes wrong).

And then there is no clear-cut line where to cut up Ukraine. For the Crimea there are rather recent borders you can refer to. With many people in Ukraine who are favorable to Russia, Putin has much more leverage than otherwise. And he would have a chance to influence Ukrainian politics again. With a split, he would create a second Poland that is headed for the EU and hostile to Russia.

I gave Walther one point for calling the formal annexation of Crimea, and another point for predicting no action by Western governments serious enough to lead to NATO fatalities, with zero lost points for contradicting any of the the other four major facts. 

I suspect that Tetlock will not be surprised by the results.  If he'd like to comment, you'll be the first to know.




COMMENTS (3 to date)
austrartsua writes:

Million monkeys on typewriters...

Finch writes:

This would have been a fairer test if your original post had asked "How many people would be killed?" "Will Russia formally annex Crimea?" "Will Russia declare war on Ukraine?" etc., matching your new question set.

As it is, you're marking people down for not correctly guessing the questions you would subsequently ask.

It seems to me that Walther was perfectly correct, as were most other commentors who claimed Russia would get Crimea with little real consequence. I commented on this uniformity at the time.

If this was supposed to demonstrate the unreliability of geopolitical predictions, it seems to do the opposite. Which frankly I find a little surprising.

brendan writes:

How many points to I get for predicting a spring offensive by the Ukrainian army? Or for noting the high possibility of draft riots in Ukraine of the war continues? Do I lose points for thinking mariupol would be captured already?

Comments for this entry have been closed
Return to top