(Medical) Diagnostic Testing
The situation:
-
Patient presents with symptoms and therefore is suspected of having some
disease. In truth, the patient either has the disease or does
not have the disease.
-
Physician performs a diagnostic test to assist in making a diagnosis.
-
The test result is either correct or incorrect.
The situation can be summarized in a two-way table as:
|
|
Test Result |
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
| Diseased (+) |
Correct
|
False Negative
|
| Healthy (-) |
False Positive
|
Correct
|
That is, a test result is considered false positive if a healthy
person incorrectly receives a positive (diseased) test result. And, a test
result is considered false negative if a diseased person incorrectly
receives a negative (healthy) test result. When developing new diagnostic
tests, researchers aim to minimize the chance (probability) of false positive
and false negative results. Or, equivalently, researchers aim to maximize
the accuracy of the tests by maximizing the chance of correct results.
Accuracy of Diagnostic Tests
Specificity and Sensitivity
The accuracy of a diagnostic test is quantified by two different conditional
probabilities, namely the sensitivity and the specificity of the test.
The sensitivity of the test is defined as the probability that a
person who truly has the disease correctly receives a positive test result.
The specificity of the test is defined as the probability that a
person who is truly healthy correctly receives a negative test result.
Since sensitivity and specificity are probabilities, their values will
always fall between 0 and 1. The closer the sensitivity is to 1, the more
accurate the test is in identifying diseased individuals, and the closer
the specificity is to 1, the more accurate the test is in identifying healthy
individuals. While perfect diagnostic tests (with a sensitivity of 1 and
a specificity of 1) are the ideal in theory, they are not realized in practice.
The definitions of sensitivity and specificity in conditional probability
notation are:
Sensitivity = P(Test +|True +) = P(Test + and True +)/P(True +)
Specificity = P(Test -|True -) = P(Test - and True -)/P(True -)
Example:
A medical researcher is interested in assessing the accuracy of the
enzyme-linking immunoassay (EIA) technique for screening blood donors for
HIV antibodies. He randomly selects 100,000 people from the population
of all blood donors and summarizes the results as follows:
|
|
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
30
|
1
|
31
|
| Healthy (-) |
1,000
|
98,969
|
99,969
|
| Total |
1,030
|
98,970
|
100,000
|
The numbers within the table reflect the number of people with the
indicated characteristic. So, 31 people truly have the disease, 1,030 people
received a positive test result, and 30 people truly have the disease and
received a positive test result.
So:
Sensitivity
= P(Test +|True + )
= P(Test + and True +)/P(True +)
= (30/100,000)/(31/100,000)
= 30/31
= 0.968
Specificity
= P(Test -|True -)
= P(Test - and True -)/P(True -)
= (98,969/100,000)/(99,969/100,000)
= 98,969/99,969
= 0.99
That is, in a population of diseased individuals, a researcher will
correctly identify 96.8% of them as diseased. And, in a population of healthy
individuals, a researcher will correctly identify 99% of them as healthy.
Not bad, eh?
Note that we could also equivalently describe the accuracy of a diagnostic
test in terms of the probabilities of incorrect results. The false negative
rate of the test is defined as the probability that a person who truly
has the disease incorrectly receives a negative test result. The false
positive rate of the test is defined as the probability that a person
who is truly healthy incorrectly receives a positive test result. Again,
since the false negative rate and false positive rate are probabilities,
their values will always fall between 0 and 1. The closer the false negative
rate is to 0, the more accurate the test is in identifying diseased individuals,
and the closer the false positive rate is to 0, the more accurate the test
is in identifying healthy individuals. The definitions of the false negative
rate and the false positive rate in conditional probability notation are:
False Negative Rate = P(Test -|True +) = P(Test - and True
+)/P(True +)
False Positive Rate = P(Test +|True -) = P(Test + and True -)/P(True
-)
Note that one can alternatively obtain the false negative and false positive
rates by taking note of their relationship to sensitivity and specificity:
False Negative Rate = P(Test -|True +) = 1 - P(Test +|True
+) = 1 - Sensitivity
False Positive Rate = P(Test +|True -) = 1 - P(Test -|True -) = 1 -
Specificity
Example (continued):
False Negative Rate
= P(Test -|True +)
= P(Test - and True +)/P(True +)
= (1/100,000)/(31/100,000)
= 1/31
= 0.032
Or, alternatively, False Negative Rate
= 1 - Sensitivity
= 1 - 0.968 = 0.032
False Positive Rate
= P(Test +|True -)
= P(Test + and True -)/P(True -)
= (1,000/100,000)/(99,969/100,000)
= 1,000/99,969
= 0.01
Or, alternatively, False Positive Rate
= 1 - Specificity = 1 - 0.99 = 0.01
Positive and Negative Predictive Values
Sensitivity and specificity pose a problem in describing the accuracy of
diagnostic tests in the field. Sensitivity assumes that one has first
identified a group of diseased individuals, and then seeks to see
what percentage the diagnostic test can correctly identify as diseased.
Likewise, specificity assumes that one has first identified a group
of healthy individuals, and then seeks to see what percentage the
diagnostic test can correctly identify as healthy. But now in practice,
the "opposite" happens. A person with symptoms presents himself to a doctor,
the doctor administers a diagnostic test, the test result comes back from
the lab, and then the doctor asks: "Given the test result is positive,
what is the probability that my patient really has the disease?" Or, "given
the test result is negative, what is the probability that my patient really
is free of the disease?" The answers to these two questions are,
respectively, the positive predictive value and the negative predictive
value. The positive predictive value of the test is defined as the
probability that a person who receives a positive test result truly has
the disease. The negative predictive value of the test is defined
as the probability that a person who receives a negative test result is
truly healthy. Again, since positive predictive value and negative predictive
value are probabilities, their values will always fall between 0 and 1.
The closer the positive predictive value is to 1, the more accurate the
test is in identifying diseased individuals, and the closer the negative
predictive value is to 1, the more accurate the test is in identifying
healthy individuals. The definitions of the positive predictive value and
the negative predictive value in conditional probability notation are:
Positive Predictive Value = P(True +|Test +) = P(Test + and
True +)/P(Test +)
Negative Predictive Value = P(True -|Test -) = P(Test - and True -)/P(Test
-)
Example (continued):
Positive Predictive Value
= P(True +|Test +)
= P(Test + and True +)/P(Test +)
= (30/100,000)/(1,030/100,000)
= 30/1,030
= 0.029 !!!!
Negative Predictive Value
= P(True -|Test -)
= P(Test - and True -)/P(Test -)
= (98,969/100,000)/(98,970/100,000)
= 98,969/98,970
= 0.9999....
What happened to the accuracy of our test that previously seemed so accurate?
The positive predictive value tells us that for a group of patients who
have received positive test results, only 2.9% of them will truly have
the disease! That means that our test will send about 97% of the people
who receive a positive test result into a needless panic.
This apparent inaccuracy cannot be helped, because its existence is
directly due to the fact that the disease is seemingly quite rare. In our
original sample, only 31 out of 100,000 people truly have the disease.
That is, our sample mostly contains healthy people, and therefore most
people who receive a positive test must therefore be healthy, and hence
the low positive predictive value of the test.
This phenomenon occurs in diagnostic tests that are used to screen a
mostly healthy population for some disease. Screening tests are still useful
in identifying diseased individuals. In practice, it is understood that
the positive predictive values of screening tests are low, and therefore
a patient who receives a positive test result will be tested again, perhaps
with a more invasive and likely more accurate diagnostic test.
In populations in which the disease is more prevalent, the difference
between the positive predictive value and the sensitivity are not as great.
Example:
Consider the following results of a diagnostic test in a population
in which a greater proportion of the people are diseased. This kind of
a summary table might result in a population at high-risk of HIV.
|
|
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
380
|
20
|
400
|
| Healthy (-) |
30
|
570
|
600
|
| Total |
410
|
590
|
1,000
|
Sensitivity
= P(Test +|True + )
= 380/400
= 0.95
Positive Predictive Value
= P(True +|Test +)
= 380/410
= 0.93
Accurate Calculation of Predictive Values
To determine the positive and negative predictive values above, we merely
read the conditional probabilities off of the table. This method can only
be used if the data arose from having taken a random sample from the larger
population, and hence the proportion of people in the sample with the disease
is representative of the proportion of diseased people in the larger population.
Due to prohibitive costs, this situation rarely happens in practice.
Instead, when assessing the accuracy of a newly designed diagnostic
tool, researchers will identify a group of patients who are diseased and
a group of healthy controls, the numbers of which are not representative
of the larger population.
The following two tables illustrate how the number of diseased patients
and the number of healthy controls directly affects the positive predictive
value. In the first table, we assume hypothetically that a physician identifies
500 diseased patients and 500 healthy controls. In the second table, we
assume hypothetically that a physician identifies 800 diseased patients
and 200 healthy controls. In both cases, we assume the sensitivity is 0.60
and the specificity is 0.80.
| Table 1 |
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
300
|
200
|
500
|
| Healthy (-) |
100
|
400
|
500
|
| Total |
400
|
600
|
1,000
|
| Table 2 |
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
120
|
80
|
200
|
| Healthy (-) |
160
|
640
|
800
|
| Total |
280
|
720
|
1,000
|
Note that in Table 1 the sensitivity is 300/500 or 0.60, and the specificity
is 400/500 or 0.80. Likewise in Table 2 the sensitivity is 120/200 or 0.60,
and the specificity is 640/800 or 0.80. Now, the proportion of diseased
individuals in Table 1 is 500/1000 or 0.50, and the proportion of diseased
individuals in Table 2 is 200/1000 or 0.20. In both tables, we assume
that these numbers are merely a function of the types of patients available
to the physician.
Now, let's look at the positive predictive values. In table 1, the positive
predictive value is 300/400 or 0.75, while in table 2, the positive predictive
value is 120/280 or 0.43. Thus, the proportion of diseased people in the
sample greatly affects the positive predictive value, and hence it is critical
that this proportion represent the true proportion in the population.
You can always find the positive predictive value and negative predictive
value accurately as long as you know:
-
the true proportion of diseased people in the population
-
the sensitivity of the test
-
the specificity of the test
Once you know these three things, you can create a two-way table on a hypothetical
population of 100,000 people, and then can read the predictive values directly
off the table.
Example:
Pap smears as a screening test for atypical cells in the cervix
Rate of atypia in normal population is 1/1000 or 0.001
Sensitivity = 0.70
Specificity = 0.90
What is the probability that a woman will have atypical cells in her
cervix given that she had a positive pap smear?
| Step 1 |
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
|
|
100 |
| Healthy (-) |
|
|
99,900 |
| Total |
|
|
100,000 |
| Step 2 |
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
70
|
30
|
100
|
| Healthy (-) |
|
|
99,900 |
| Total |
|
|
100,000
|
| Step 3 |
Test Result
|
|
| True Disease Status |
Diseased (+)
|
Healthy (-)
|
Total
|
| Diseased (+) |
70
|
30
|
100
|
| Healthy (-) |
9,990
|
89,910
|
99,900
|
| Total |
10,060
|
89,940
|
100,000
|
Then, positive predictive value is 70/10,060 or 0.00696 !!
Practice Exercise
1. Find the positive and negative predictive values for a diagnostic
test knowing that 10% of the population has the disease, the sensitivity
of the diagnostic test is 0.96, and the specificity of the test is 0.98.
[Solution: Using a population of 100,000, PPV = 9600/11400 = 0.84 and
NPV = 0.995.]