Arnold Kling  

Thoughts on Comparative Effectiveness Research

PRINT
Funniest Macro Line of the Mon... Good Answer!...

Tyler Cowen writes,


This commission, if it sticks to its statistical mandate, will be able to recommend many more possible cuts than any vote-maximizing administration will be likely to make. Some other principle will be used to determine cuts. Many defenders of the Obama administration are overestimating how scientific this process will be.

Read the whole thing, as well as Megan McArdle's post, to which he links.

My additional thoughts:

1. I have proposed that the government charter a commission to do research on cost-effectiveness of treatments. I did so in this essay and in my book.

2. I assumed a need for government involvement because of standard public-good arguments. Pure statistical research on the outcomes from different medical protocols is a good from which everyone can benefit, whether or not they pay for it.

3. There are strong incentives for interest groups to try to distort the results of a commission. But those incentives exist whether or not the government is involved. When there are private evaluation systems, people try to distort them also--AAA-rated mortgage securities come to mind.

4. I think that people vastly oversimplify the problem of selecting medical protocols. I cringe every time Peter Orszag speaks in binary terms (for example here where he calls for "more research into what works and what doesn't.")

The research will not divide protocols into "what works and what doesn't." Instead, it will provide probabalistic data. It is not going to tell you that Barack Obama's grandmother's hip operation will be futile. It is going to give you some estimate of the probability of its futility.

Orszag makes it sound as if research will separate treatments into those that work 100 percent of the time and those that work 0 percent of the time. All of the really interesting cases fall somewhere in between.

5. The problem becomes even more complex when you do marginal cost-benefit analysis. For example, suppose that you can prevent 80 percent of colon cancers with a very expensive protocol, and you can prevent 50 percent of colon cancers with a much cheaper protocol. On average, compared with doing nothing, the cost per life saved for the expensive protocol might be, say, $800,000, and you might be ok with that. But when you take into account the cheaper protocol, the cost per marginal life saved of the more expensive protocol might be $2 million, and that's starting to look steep.

6. Which brings up the question of whether the commission should (a) just oversee the studies and publish the results, (b) make recommendations based on the research, or (c) issue regulations that go beyond recommendations. I favor (b), because I think that people could use some expert guidance on issues like decision-making under uncertainty and marginal cost-effectiveness.

It may very well emerge that insurance only pays for treatments that fit with government recommendations, which gives those recommendations a lot of clout. Only if you are paying your own nickel can you get non-recommended treatment. I am ok with that, in part because I think that paying your own nickel should be the norm rather than the exception.

7. Which leads me to the important point. Doing research on medical effectiveness gives people the means to restrain their use of medical services, but it does not give them the motive. For a treatment with high costs and low expected benefits, the research is going to say, "the probability that this achieves the desired outcome is x." If x is greater than zero, and you're spending someone else's money, then you are motivated to get the treatment. So to stop people from getting treatment you either have to change the health insurance system to have higher co-pays and deductibles (my preferred approach) or you need to suppress treatments bureaucratically (the solution to which health care reformers are headed, although they will not admit it).

8. Medical decisions are going to be difficult to make. I believe that statistical information provided by a central source can help. However, the doctor who sees you will have valuable local information that the bureaucrat lacks. And you have information about your risk preferences that the bureaucrat lacks. That is why I prefer to see individuals make decisions, taking expense into account, to having decisions made by remote bureaucrats.


Comments and Sharing





COMMENTS (12 to date)
John Thacker writes:

And to forestall a possible complaint about marginal cost analysis, it's certainly possible with advanced diseases that it's impossible to do cheaper protocol Y then follow it up with more expensive protocol X if Y doesn't work, as the disease will progress too far by the time you can tell that Y doesn't work.

Particularly when the more expensive but more effective treatment is invented first, and initially covered, I can't see government stopping coverage when the cheaper but somewhat less effective protocol is discovered.

Jim Glass writes:

I cringe every time Peter Orszag speaks in binary terms (for example here where he calls for "more research into what works and what doesn't.")

Right. The big problem isn't that people pay for things that don't work at all -- it's that they have a legitimate motive to pay a lot for things that work a little.

Your child has cancer, there's a treatment that increases his chance of survival by maybe 5%, it costs $X million. If it's on the government's tab you demand they pay for it -- that's why you pay taxes!

If it's on the insurance company's tab and they refuse the treatment as "not cost effective" you are in the newspapers, on cable TV and in court complaining out loud about it to everyone who can hear -- the cheap bastards are putting profits before a child's life!

If it's on your own tab and you are a middle-class person, you and your child may be out of luck, "cost containment enforced" -- but you'll be out leading the march for nationalized health care forevermore, to put all such things on the government's dime, because it is absolutely immoral that your child should die just because you weren't born a multi-millioniare.

But whatever, there was a 95% chance that your child was doomed in any case.

Unsympathetic writes:

"Medical decisions are going to be difficult to make"

Always have been, always will be. Doctors aren't going to make them - in fact, doctors will ALWAYS order emergency life-saving measures REGARDLESS of cost. That's how they're trained, that's what they are expected to do to maintain their license.

Arnold - When you get the AMA on board with your policies, you go right ahead and promote them. Until then, ANY cost-cutting idea won't mean anything.

The listed proposals only result in stiffing a hospital which ALREADY provided a service and is due compensation - or removing licenses from doctors who did their job as they were told to do it. Neither of these are positive outcomes.

Any proposed solution must address both the raw dollars AND the medical license issues -- or it's a non-starter.

Mattyoung writes:

Ditto for every other aspect of Obama's government programs. The more we try the more apparent was Hayek's maxim that government cannot centrally plan.

Dan Weber writes:

I have proposed that the government charter a commission to do research on cost-effectiveness of treatments.

As I learned from this blog, the new VHA already does this. They computerized records and they measure everything they do, so they have a very good idea of the procedures that do and don't work.

Unfortunately, most plans to increase government involvement don't seem to be about replicating or supplementing the VHA.

another bob writes:

A small point. You object to Orszag's 'binary' talk. 'Life saved' is another binary term. A treatment may forestall a death, but, for how many days?

To use your dollar cost analogy; does the additional expenditure produce additional days at $100/day or $10,000/day or $1M/day? Now where's the cut-off?

And what about the quality of life during these days? Is the patient concious during these days? Alert and responsive? Ambulatory? Economically productive?

Anthony writes:

"the probability that this achieves the desired outcome is x." If x is greater than zero, and you're spending someone else's money, then you are motivated to get the treatment. So to stop people from getting treatment you either have to change the health insurance system to have higher co-pays and deductibles (my preferred approach) or you need to suppress treatments bureaucratically (the solution to which health care reformers are headed, although they will not admit it).

So how about the insurance company (or government) decide that it will pay x% of the cost, perhaps rounded into 10% increments, with updated research adjusting the percentages every few years?

Art writes:

Comparative Effectiveness Research?? Where is it? Why for 50 years has the AMA refused to do any serious investigation into the efficacy of Vitamin C as a treatment for example, as for a cure for anything. I did a cursory count and there are collectively almost 90 million Vitamin C related links on the 3 major search engines. Why have so many got so much to say about it, and it doesn't even peak the curiousity of the AMA even a bit, simple a one word answer, MONEY!! If alternative medicines are proven effective what happens to the Billions in Drug Company revenues? That is the real reason why the Medical establishment isn't seiously interested in looking at Anything natural, or alterantive!! What natural cheap & effective treatment have they ever endorsed??

Dan Weber writes:

Pubmed has over 42000 entries matching the search phrase Vitamin C. So much for the conspiracy theories.

David S writes:

"the probability of its futility." Is that an oxymoron? Isn't something futile only if the probability it will fail is 100%, i.e. it falls into the doesn't work camp?

Dr. T writes:

One of the biggest problems with comparative medical efficacy studies is weighing treatments when the outcomes aren't perfect. All too often, the values of patients are not considered.

Invented example: 55-year-old with metastatic, terminal pancreatic cancer. There are three options:

1. Treat pain only: death in six months. Not able to work. Cost A.

2. Surgery to remove most cancer, chemotherapy to slow down the rest, plus pain therapy: death in fifteen months. Able to work for 10 months. Cost 10 x A.

3. Chemotherapy and pain therapy: death in ten months. Able to work for six months. Cost 4 x A.

None of the pain therapies is totally effective, so the patient will be in chronic mild to moderate pain while still having side effects (difficulty concentrating and sleepiness) from the pain drugs. The chemotherapies cause nausea, vomiting, and generalized malaise. Each patient has a different desire for extending his life regardless of its quality. Each patient has a different desire for continuing work.

It takes a very nuanced system to assess treatment efficacies, assess patient preferences, assign weighting factors, and determine the best option. I cannot imagine a centralized system doing a good job at this task.

Lord writes:

The main result should be what a treatment is worth depending on alternatives. Then whether it can be provided for that, covered up to that, or only up to the next most effective, or even up to a multiple of the most effective to provide for alternative options can be up to the individual or plan.

Comments for this entry have been closed
Return to top