Arnold Kling  

Bell Curve in Medical Care

Social Security and Indexing... Worrying About Demand...

Atul Gawande writes,

It used to be assumed that differences among hospitals or doctors in a particular specialty were generally insignificant. If you plotted a graph showing the results of all the centers treating cystic fibrosis--or any other disease, for that matter--people expected that the curve would look something like a shark fin, with most places clustered around the very best outcomes. But the evidence has begun to indicate otherwise. What you tend to find is a bell curve: a handful of teams with disturbingly poor outcomes for their patients, a handful with remarkably good results, and a great undistinguished middle.

There would be nothing alarming about this if there were continuous improvement, with the weaker health care providers either shaping up or going out of business. Such a Darwinian process can be taken for granted in most markets, but not in health care.

Thanks to Virginia Postrel for the pointer. She also points to some posts by James Frederick Dwight that challenge Gatawande's analysis. Dwight writes,

So what to do we have to do to compare different doctors? A multi-variate analysis. Here's the bad news - if you don't know what a multi-variate analysis is, you probably can't do one.

There's never anything inherently wrong with information, so I would never be opposed to compiling information. The problem is, raw information can be hijacked by the ignorant.

Gatawande is arguing that the medical profession suffers from a dearth of useful data on outcomes. (I would say the same thing even more strongly about the education profession.) In this regard, Dwight is not being constructive by complaining about the need for multivariate analysis. First, establish the principle that data should be gathered, disclosed, and analyzed. Then complain about methodological impurity.

For Discussion. One thought I have had recently is that it is the goal of much regulation to protect the mediocre from superior competition. To what extent does regulation in medical care fit that model?

Comments and Sharing

TRACKBACKS (2 to date)
TrackBack URL:
The author at The Glittering Eye in a related article titled Sunday quick glances (UPDATED) writes:
    A quick look at the blogosphere this morning [Tracked on January 2, 2005 2:42 PM]
The author at blogs for industry in a related article titled Unintended consequences and public scoring of CF centers writes:
    The new issue of the New Yorker has includes this in the mail: ... It should be emphasized, however, that while the wide spread in the quality of care that different hospitals offer is a real phenomenon, it is only one of the causes that affect outc... [Tracked on January 4, 2005 4:15 AM]
COMMENTS (8 to date)
Ronnie Horesh writes:
...the medical profession suffers from a dearth of useful data on outcomes. (I would say the same thing even more strongly about the education profession.)
First, establish the principle that data should be gathered, disclosed, and analyzed.

All activities [including data-gathering] should be subordinated to explicit, meaningful health outcomes. These should be broad. They could include reduced infant mortality; greater welfare-adjusted longevity etc. Government could set these targets and help finance their achievement, but should not prejudge how best to reach them. The private sector would gather data and regulate, only insofar as necessary to achieve the targeted outcomes efficiently. Mutatis mutandis for education. My site explains how this could be done in a way that allows market forces to allocate resources.

Jon writes:

Generally the empirical evidence opposes Arnold's claim. It is that the goal of much regulation is to protect people from their own lack of knowledge.

What regulation concerning medical care protects the mediocre from the good?

That medical care outcomes should follow a bell curve, instead of a shark fin is not surprising. I suspect this is the case in the quality/value for many goods and services. If there were no differences amoung center qualities, there could certainly be a bell curve just due to random variations. People leave the poorer centers, some centers innovate and hire better people and become better, and hopefully the bell curve peak moves towards better outcomes.

Dave Schuler writes:

One thought that occurred to me was that the fee for services model (or the incentivized salary approach of HMO's) in fact shields physicians from precisely this information.

Lawrance George Lux writes:

Regulation does not protect the mediocre; inability to formally restrain mediocre practice protects them. Government, the AMA, and State licensing Boards protect the mediocre. A ruthless application of Law could vastly improve health care: All Doctors and Hospitals to be examined every decade, with 10% of both Doctors and Hospitals with the poorest records to be decertifed from the practice of medicine. lgl

Patri Friedman writes:

How does this Bell Curve demonstrate that there is any difference in quality between doctors or hospitals? Isn't a bell curve exactly the distribution you would expect from a bunch of independent samples? (hospital = sample, single patient event = one sampling).

I saw this story earlier and was baffled as to why the conclusion drawn was that centers are different. If I flip a coin 10 times, and do that 100 times, and I get a bell-shaped distribution of number of heads, do I conclude that the different sets had different head-generating ability?

Jay writes:

Kudos to Patri!

Indeed this is the case. This assumes that observations of different hospitals is independent though.

I wouldn't say the protection of mediocrity is the goal of regulation as much as an unintended consequence. Except maybe in the cable TV business.

Erik Sargent writes:

I've seen a lot of discussion about private rankings of physicians, based on customer polls or something like this. But this survey, and the comment about multivariate analysis demonstrates the diffence in public knowledge is too great for this to work.

dsquared writes:

I must say that it also mystifies me why Gatawande expected to find anything other than a normal distribution of this data, and I notice that he doesn't actually provide a citation for anyone having ever said that they thought medical outcomes would have the weird right-skewed distribution he describes. In fact, thinking about it, every medical statistics textbook I've ever seen (all two of them) operates off an implicit assumption of normality. I'm calling "shenanigans" here.

(I also disagree with Arnold's assertion that "Such a Darwinian process can be taken for granted in most markets". I can't think of a single market in which I couldn't name at least one utterly mediocre or substandard player. It's also not the case that it is typically the lowest-quality providers which go under in markets; it's often high-quality providers who are driven out because of poor cost control, marketing or financing)

Comments for this entry have been closed
Return to top