Take a test for a disease that has a false positive rate of 5%, and a disease prevalence of 1 in 1000--lupus, say. If you test positive in a random assay, what are the odds that you actually have the disease?
Most people--even, apparently, a shocking number of doctors--would say that the odds are 95%. But this is all wrong. If you test 1,000 people for lupus, 1 of them will correctly test positive for lupus--and 50 of them will falsely test positive. The chances are only 1 in 51, less than 2%, that you actually have the disease.
These are in fact the actual numbers for anti-nuclear antibody tests and systemic lupus, at least as relayed to me by my immunologist after I got a borderline positive result on a screen. These suggest that no one should ever do a random ANA; the information it gives is garbage, particularly since they don't treat lupus until you manifest symptoms. Yet lots of doctors, including mine, do.
I have written about this issue many times, but it is worth writing about again. One of the first things that doctors forget when they leave medical school is elementary probability theory. Their statistical ignorance has real effects on health care delivery. It is one of the reasons why I believe we need a medical guidelines commission, and I am willing to suspend "lose the we" and consider a commission chartered by government (although it does not have to be).