Garett Jones  

How to Trust Government Data (or not)

PRINT
Travels of a T-Shirt... Better Than Plowing...
I recommend Russ's recent EconTalk with Morten Jerven.  Jerven's forthcoming book, Poor Numbers, shows that GDP estimates in sub-Saharan Africa are fraught with error. Jerven, who visited many statistics offices in the region firsthand, concludes that poorly staffed offices and ever-shifting measurement standards give us reason to doubt the quality of these data.  The whole hour is filled with insights, do listen.  

Jerven makes it clear that low quality data aren't just a problem for sub-Saharan Africa--the problem is endemic across less developed countries.  Low productivity in the private sector typically predicts low quality in the public sector--and that predicts low quality government statistics.  

Independently, Russ offers another potential problem: The temptation of governments to report bad news to make sure the foreign aid keeps pouring in.  It would be valuable to test this empirically but until somebody comes up with a solid test, let's just stick to theory: governments must face some temptation to report news that differs from reality.  Some governments will be tempted to report news that's worse than the truth, some that's better than the truth.  

That temptation to tell excessively good news surely exists in the rich, democratic countries, at least most of the time.  But the civil services in rich countries are immune to crude manipulations of GDP statistics, aren't they?  Apparently not. Consider this study of GDP data in the G-7 club of rich economies:

...in the Japanese data, the last GDP growth rate announced before national elections is systematically about one percentage point (at an annual rate) more optimistic than in other quarters. This result is robust for Japan, but the election effect is not found in any other country. 

Is there a way out?  Is there some independent way for mere observers like us to find out what's really happening? Yes, there is. TC:

Travel is the starting point of learning social science. 

And I'd add "...and of learning GDP per capita."  Not the only way--the financial sector has a strong incentive to find out which regions of the world have dollar bills on the sidewalk, so keep an eye on their reports--but checking out a few different countries or asking questions of people who have visited these places is worth quite a lot.  

Journalism, anthropology, talking to friends, seeing places yourself (especially outside tourist-friendly cities): There are quite a few ways around the bad data problem.  True for rich countries as well as poor. 

Does the place seem as rich as the statistics say?  If your eyes disagree with the official data, maybe it's the data that are wrong. 

Comments and Sharing





COMMENTS (12 to date)
James Reade writes:

Don't wish to be too critical, but doesn't your entire argument rely on Japan here? The quote you give says the effect only stands up in Japan and not in the other six G7 nations.

I'm sure the incentive is there, and I'm sure it manifests itself in other ways, but as an empirical backdrop for the point you want to make, it seems pretty weak to me?

Hadur writes:

If one of your students attempted to debunk data using an anecdote from spring break, you would surely fail him.

Bob Knaus writes:

Indeed. As a native of Miami, I have given this advice many times. Do you want to make money speculating on Latin American currencies? Then visit South Beach, observe which nationality is spending money like drunken sailors, and short that nation's currency. The upper classes in these countries always know about trouble before it makes the news, and so they party while they can.

Tracy W writes:

I'm doubtful of the walk-around hypothesis for data collecting.
For a time, I had a job forecasting tax revenue, and as part of that each round we'd go around the country talking to tax accountants. In the same city we'd often get very differing views of how their clients were doing and thus their clients' likely tax revenue, even where the client base was reasonably similar.

(I know, this is anecdotal data arguing that anecdotal data isn't that good).

Although on the point of reasons to manipulate data, we had a counterpart visit us from an African country, I forget which one, who said that he was under pressure to produce low tax revenue forecasts, as his forecast was a target for the tax department.

Grieve Chelwa writes:

"Does the place seem as rich as the statistics say? If your eyes disagree with the official data, maybe it's the data that are wrong."

My gut feeling always disagrees with the official statistics and conventional story every time I visit Botswana and wonder what Acemolgu, Johnson and Robinson are seeing that I am not.

David Friedman writes:

A few years ago, I visited New Zealand. I happened to have looked at the economic data a little while earlier and it showed, as I recall, per capita income at purchasing power exchange rate not much more than half that for the U.S. But my impression from first hand observation was that the standard of living was not strikingly different from that in the U.S.

I don't know if that reflected an error in the data or problems with my observations--I might, for instance, have been associating mostly with relatively well off people. But I did find the size of the discrepency puzzling.

david writes:

Per capita income varies dramatically with whether you're in Manhattan or somewhere out in the plains.

Rod McFadden writes:

I believe that Dr Russ Roberts would agree that 89.254937 percent of social scientists would agree.

[broken url fixed. You meant blogspot.com, not blogger.com, right? Please check that your links work before posting.--Econlib Ed.]

Mark V Anderson writes:

I agree that "walking around" will give someone only a very gross feel for the country, and as David says, different regions may be radically different.

I vacationed in New Zealand in 2008, and I did get the feel that it was poorer than the U.S., but I would not have guessed only half as wealthy. I made my assessment based on the fact of so few SUVs there compared to the USA, even though NZ is the kind of country suitable for such vehicles, being not very dense and somewhat mountainous. But that's a major hypothesis based on little data.

I have also noticed that even amongst those native to the USA, there is quite a difference on how wealthy we are. I keep hearing how the average person has not gained in income since the '70's. I dispute largely based on the experiences I have had in my own life. I lived middle class in the '70's and am middle class now, and yet it is now so much more common for most to take long trips in the USA and overseas, obviously computers provide us with much better communication ability, and everyone just buys so much more "stuff" than they used to. And yet I can't get many others to agree with me, even if they have had similar experiences as me.

But Garett expects us to understand the economics of a country just by visiting it for a short time? I agree that a visit will provide one data point, but I don't think one can make broad conclusions based on it.

Shayne Cook writes:

There is a marvelous story about a businessman who decides he wants to hire an accountant - and there are three applicants. He wants to hire a "good" accountant of course, so he devises a question he will pose to all three: "What is 2 plus 2?" The first two applicants, after some musings and ciphering, both respond, "2 plus 2 is 4!"
The third applicant, after being asked, "What is 2 plus 2?", responds immediately with, "Anything you want it to be", and is hired on the spot.

Such, I suspect is the case with GDP figures. And not even so much that, "What is our GDP?" being "Whatever you want it to be", as much as GDP being merely an exercise in accountancy.

On several occasions in the past here, Arnold Kling bemoaned the over-emphasis many macro-economist types placed on GDP and like metrics. He even coined a term related to this phenomena, arguing that too many macro-economists viewed the national economy as nothing more than a "GDP factory". Russ Roberts has also expressed his skepticism about the value of attempting to quantify and metricize all aspects of an economy. That is one of the reasons I enjoy reading Arnold Kling and enjoy reading and listening to Russ Roberts.

My point here is that, even if you had absolutely reliable, unequivocal, unchallengable GDP data - from anywhere - all you would really have is sound accountancy. And I, as well as Kling and Roberts, seem to think economics is somewhat more than that.
I'm fascinated by economics. Accountancy, not so much.

Daublin writes:

Not to defend official statistics, but it can be grossly misleading to take a few days' visit.

One case in point is that I have a Greek friend who visits home, comes back, and says everything is fine. Everyone's on the beach all the time. Another Greek friend stays in closer touch with relatives, and says it's awful, nobody is working, people are getting sick and not able to get doctor time. The latter point of view only comes out, apparently, if you have some sort of detailed source of long-term information.

Visiting Switzerland, I personally had the same kind of pattern. A month or so in, I was saying standards of living were about the same. They dress the same. They're healthy. The water is safe. The streets are reasonably clean.

However, as time went on, I noticed quite a number of ways that life is harder than what Americans are used to. Domiciles are small. Median folks don't have air conditioning. They don't have dish washers, nor clothes washers. They only eat meat at restaurants and at special occasions.

Only wealthier Swiss people drive. Median Swiss people ride public trans, turning 5-minute commutes into 30-minute commutes. I'm told they are taking the trans because they prefer it, but that is not how they vote with their feet once they get a little money.

Someone from the other side writes:

[Comment removed for crude language. Email the webmaster@econlib.org to discuss editing your comment. A valid email address is required to post comments on EconLog and EconTalk.--Econlib Ed.]

Comments for this entry have been closed
Return to top