Studies Show

I laud your point but I cannot help but feel that you are preaching to the choir here.

"But not only is the Truth Out There; it's a lot closer than you think."

I think this implies that the evidence the public sees from the internet should lead them to be more trusting of experts. It is not clear to me that this is so. For example, there is clip after clip of Bernanke making claims about the solidness of the economy, the absence of a housing bubble etc.

When the nations arguably most important economist leaves a lengthy and easy to find trail of plainly incorrect predictions, why would the average person conclude that economists know what they are talking about.

Then faced with this "Bernanke" evidence. The average person is supposed to dive into the details of economic studies and divine which ones are correct. Much easier just to watch the contrasting Peter Schiff youtube clips and make your judgements that way. Course, this decision making process is susceptible to the broken watch.

[Comment removed pending confirmation of email address. Email the webmaster@econlib.org to request restoring this comment. A valid email address is required to post comments on EconLog.--Econlib Ed.]

Good point Sir. The is a columnist (and full time medic) for the UK's Guardian who makes a regular feature of investigating the various stories of reporters claiming to be backed up by studies.

http://www.badscience.net/

The superficial subject matter is science and medicine however.

A particular egregious example may be found here:

http://www.badscience.net/2009/07/asking-for-it/

While I completely agree with the sentiment, I cringe at the notion the guilty can only be non-academics. Academics are not immune from faulty logic or from cherry-picking "facts" to make a point to their choosing.

studies show that much of the pathology begins within the research itself -- the biases which determine what will be published, and what spin is put on data by researchers, etc.

I'm surprised you aren't familiar with this literature.

You could have easily included a second Simpson's sequence:

TROY MCCLURE
Just ask this Scientician!

QUICK SHOT OF MAN IN LAB COAT
'Uh...'

TROY MCCLURE
He'll tell you about the 'Food Chain'!

This broken chain of custody is even worse within academia itself. Popularizers and the man in the street have ideological biases, but it's those who have real skin in the academic game who are the most fearful of research that contradicts their worldviews.

Just about every academic field has its set of urban legends. Instead of "studies show," it's "As is well known, Hergerdorfer (1965) demonstrated..." Hey, if they said "as is well known," I won't bother looking up that citation!

There's a balance we have to strike between credulity and skepticism of so-called experts. Maybe some academic fields have better honesty policies, but in general I'd be more skeptical of academic citations that don't include the raw data than I would be of popular citations that don't include the data.

The popularizer may be trying to keep things simple presentation-wise for a mass audience. But if an academic article doesn't spell out the magnitude of the beta coefficients in "Hergerdorfer's (1965) regression demonstrating that...", that's even more suspect. Ditto when an academic article has regression coefficients, etc., *but no plot of the data*.

Yet few academics bat an eyelash when they read a data-free citation in a journal.

And this isn't even mentioning the problem of "As is well known" worldviews that are based on a single study! Let alone the trend toward burying the raw data deeper and deeper into a cyber-labyrinth where few people will bother twisting through. Articles in high-impact journals now read more like ScienceDaily news summaries -- no tables, plots, etc., and even the text uses adjectives like "large effect" rather than pinning it down with numbers.

I worry...

The internet also makes it easier to find unscientific or biased studies. It's important for people to learn how to distinguish between reliable and unreliable sources. If a website or article looks professional enough some people will be convinced that it is accurate. Online tools should help filter out untrustworthy information. For example, Google Scholar shows the source of an article and lists all of the articles that cite it. This helps one determine if the article is accepted by scientists. Additionally, the ease with which multiple studies can be found using the internet lets you compare a larger number of studies than you could before. This lets you draw a more balanced conclusion. -S.N. WCU 1257

LOL. LOL.

Just so you know, I WILL be stealing the "Institute of Shut your fat face" line for future use. Let Matt Groening know.

I will admit that I can be very gullible at times and convincing me that something is true based on some quick evidence is not too tough. However, I have become more aware of this weakness of mine and have learned to doubt others until I have some reliable substantiation. Ever since I took a statistics class my junior year in high school I am more suspicious of claims in commercials, advertisements, studies, etc. that we as an audience tend to accept as facts without proof. I know that I am easily convinced by commercials stating that their product will make you lose weight fast or claim other enticing effects. Even statistics are thrown around haphazardly and in most cases are very misleading and biased. For example, when Al Gore came out with An Inconvenient Truth I bought into the global warming theory whole-heartedly. I believed that because he had so much research and statistics behind it that it must be true. Not until I took an environmental class my senior year did I find information that showed how some of his graphs and research was not completely accurate and was skewed to make it fit his statements. I am not saying that global warming is still not an issue, but I am more cautious about believing extreme claims like Gore’s. In general I think that we all need to be more careful about believing everything we hear. And in relation to the government and the economy, we as a people should not be swayed by everything the government says, we need to form our own opinions and find credible information about things like the economic stimulus, the war in the Middle East, universal healthcare, etc.

Studies, studies, studies. It seems nowadays that there is constantly a new study that explains a commonplace trend that "you" have abided by your entire life without even realizing it. There are studies for everything. Studies for why you brush your teeth with your right hand and not with your left and studies that determine just how successful your kids may or may not be. I grow weary of the studies that surround me, studies that beckon me in to whisper in my ear that "you are not normal...obey". Trends, they are great to know, general trends are beneficial to the human action for they guide what is expected of humane, not human, behavior.

I agree with the commenters who object to scholar.google.org being a reliable brand. All it takes to start a journal is a dozen or two people who say that a topic is important. Once it gets bootstrapped, its importance becomes self-fulfilling, because it's the people already involved who decide what papers matter and decide which people are doing the best at it. Outsiders are, by definition, not experts in the area.

However, at the same time, the journal format does sometimes work. The important thing is to check the source. Merely being on scholar.google.com is not that reliable, any more than being in a printed newspaper is all that reliable. However, you can check the people involved, and you can check the journal involved, and you can follow reverse citations to see what other people are saying about the paper.

Thus, Bryan's key point stands: the world has extraordinarily good access to the scholarly world's writings. Please, though, let's not endorse journals with too broad of a brush. There's an awful lot of complete rubish that is only published because there are an awful lot of poor researchers available to review it.