1
Diagnostic tests
Subodh S Gupta
MGIMS, Sewagram
Standard 2 X 2 tableStandard 2 X 2 table(For Diagnostic Tests)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic testtest
Positive Positive (T+)(T+) aa bb a+ba+b
Negative Negative (T-)(T-) cc dd c+dc+d
TotalTotal a+ca+c b+db+d NN
Gold Gold StandardStandard
Standard 2 X 2 tableStandard 2 X 2 table(For Diagnostic Tests)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-)
Diagnostic Diagnostic testtest
Positive Positive (T+)(T+) TPTP FPFP
Negative Negative (T-)(T-) FNFN TNTN
Gold Gold StandardStandard
Gold standard
In any study of diagnosis, the method being evaluated has to be compared to something
The best available test that is used as comparison is called the GOLD STANDARD
Need to remember that all gold standards are not always gold; New test may be better than the gold standard
Test parameters
Sensitivity = Pr(T+|D+) = a/(a+c)
--Sensitivity is PID (Positive In Disease)Specificity = Pr(T-|D-) = d/(b+d)
--Specificity is NIH (Negative In Health)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) aa bb a+ba+b
Negative Negative (T-)(T-) cc dd c+dc+d
TotalTotal a+ca+c b+db+d NN
Gold StandardGold Standard
Test parameters
False Positive Rate (FP rate) = Pr(T+|D-) = b/(b+d) False Negative Rate (FN rate) = Pr(T-|D+) = c/(a+c) Diagnostic Accuracy = (a+d)/n
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) aa bb a+ba+b
Negative Negative (T-)(T-) cc dd c+dc+d
TotalTotal a+ca+c b+db+d NN
Gold StandardGold Standard
Test parameters
Positive Predictive Value (PPV) = Pr(D+|T+) = a/(a+b)
Negative Predictive Value (NPV) = Pr(D-|T-) = d/(c+d)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) aa bb a+ba+b
Negative Negative (T-)(T-) cc dd c+dc+d
TotalTotal a+ca+c b+db+d NN
Gold StandardGold Standard
Sensitivity = 90/(90+10), Specificity = 95/(95+5)
FP rate = 5/ (95+5); FN Rate = 10/ (90+10)
Diagnostic Accuracy = (90+95) / (90+10+5+95)
PPV = 90/(90+5); NPV = 95/(95+10)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 55 9595
Negative Negative (T-)(T-) 1010 9595 105105
TotalTotal 100100 100100 200200
Test parameters: Example
Gold StandardGold Standard
Sensitivity 90%
Specificity 95%
False Negative Rate 10%
False Positive Rate 5%PPV 94.7%
NPV 90.5%
Diagnostic Accuracy 92.5%
PPV & NPV with Prevalence
Healthy population vs sick population
Healthy Sick
Predictive Values in hospital-based data
Predictive Values in population-based data
Prevalence = 50%PPV = 94.7%
NPV = 90.5%Diagnostic Accuracy = 92.5%
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 55 9595
Negative Negative (T-)(T-) 1010 9595 105105
TotalTotal 100100 100100 200200
Test Parameters: Example
Gold StandardGold Standard
Prevalence = 5%PPV = 48.6%
NPV = 99.4%Diagnostic Accuracy = 94.8%
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 9595 185185
Negative Negative (T-)(T-) 1010 18051805 18151815
TotalTotal 100100 19001900 20002000
Test Parameters: Example
Gold StandardGold Standard
Prevalence = 0.5%PPV = 8.3%
NPV = 99.9%Diagnostic Accuracy = 95%
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 995995 10851085
Negative Negative (T-)(T-) 1010 1890518905 1891518915
TotalTotal 100100 1990019900 2000020000
Test Parameters: Example
Gold StandardGold Standard
Prevalence = 0.05%PPV = 0.9%
NPV = 100%Diagnostic Accuracy = 95%
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 99959995 1008510085
Negative Negative (T-)(T-) 1010 189905189905 189915189915
TotalTotal 100100 199900199900 200000200000
Test Parameters: Example
Gold StandardGold Standard
Prevalence 50% 5% 0.5% 0.05%
Sensitivity 90% 90% 90% 90%
Specificity 95% 95% 95% 95%
PPV 94.7% 48.6% 8.3% 0.9%
NPV 90.5% 99.4% 99.9% 100%
Diagnostic Accuracy
92.5% 94.8% 95% 95%
PPV & NPV with Prevalence
Trade-offs between Sensitivity and Specificity
BE-Workshop-DT-July2007 21
When we use Diagnostic test clinically, we do not know who actually has and does not have the target disorder, if we did, we would not need the Diagnostic Test.
Our Clinical Concern is not a vertical one of Sensitivity and Specificity, but a horizontal one of the meaning of Positive and Negative Test Results.
Sensitivity and Specificity solve the wrong problem!!!
BE-Workshop-DT-July2007 22
When a clinician uses a test, which question is important ?
If I obtain a positive test result, what is the probability that this person actually has the disease?
If I obtain a negative test result, what is the probability that the person does not have the disease?
Test parameters
Sensitivity = Pr(T+|D+) = a/(a+c)Specificity = Pr(T-|D-) = d/(b+d)PPV = Pr(D+|T+) = a/(a+b)NPV = Pr(D-|T-) = d/(c+d)
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) aa bb a+ba+b
Negative Negative (T-)(T-) cc dd c+dc+d
TotalTotal a+ca+c b+db+d NN
Gold StandardGold Standard
24
Likelihood Ratios
Likelihood Ratio is a ratio of two probabilities Likelihood ratios state how many time more
(or less) likely a particular test results are observed in patients with disease than in those without disease.
LR+ tells how much the odds of the disease increase when a test is positive.
LR- tells how much the odds of the disease decrease when a test is negative
25
The likelihood ratio for a positive result (LR+)
tells how much the odds of the disease
increase when a test is positive.
The likelihood ratio for a negative result (LR-)
tells you how much the odds of the disease
decrease when a test is negative
26
The LR for a positive test is defined as:
LR (+) = Prob (T+|D) / Prob(T+|ND)
LR (+) = [TP/(TP+FN)] [FP/(FP+TN)]
LR (+) = (Sensitivity) / (1-Specificity)
Likelihood Ratios
27
The LR for a negative test is defined as:
LR (-) = Prob (T-|D) / Prob(T-|ND)
LR (-) = [FN/(TP+FN)] [TP/(FP+TN)]
LR (-) = (1-Sensitivity) / (Specificity)
Likelihood Ratios
28
What is a good ‘Likelihood Ratios’?
A LR (+) more than 10 or a LR (-) less than
0.1 provides convincing diagnostic evidence.
A LR (+) more than 5 or a LR (-) less than
0.2 is considered to give strong diagnostic
evidence.
Likelihood Ratio for a positive test = (90/100) / (5/100)
= 90/ 5 = 18
Likelihood Ratio for a negative test = (10/100) / (95/100)
= 10/ 95 = 0.11
Disease StatusDisease Status
Present Present (D+)(D+)
Absent Absent (D-)(D-) TotalTotal
Diagnostic Diagnostic TestTest
Positive Positive (T+)(T+) 9090 55 9595
Negative Negative (T-)(T-) 1010 9595 105105
TotalTotal 100100 100100 200200
Likelihood Ratio: Example
Gold StandardGold Standard
Exercise
In a hypothetical example of a diagnostic test, serum levels of a biochemical marker of a particular disease were compared with the known diagnosis of the disease. 100 international units of the marker or greater was taken as an arbitrary positive test result:
Example
Disease Disease StatusStatus
PresePresent nt
AbsenAbsentt
TotalTotal
MarkerMarker
>=10>=1000431431 3030 461461
<100<100 2929 116116 145145
TotalTotal 460460 146146 606606
Exercise
Initial creatine phosphokinase (CK) levels were related to the subsequent diagnosis of acute myocardial infarction (MI) in a group of patients with suspected MI. Four ranges of CK result were chosen for the study:
Exercise
Disease StatusDisease Status
PresenPresent t
AbsentAbsent TotalTotal
CPKCPK>=280>=280 9797 11 9898
80-27980-279 118118 1515 133133
40-7940-79 1313 2626 3939
1-391-39 22 8888 100100
TotalTotal 230230 130130 360360
35
Odds and Probability
Probability of Disease = (# with disease) /
(# with & # without disease) = a/ (a+b)
Odds of a disease = (# with disease) /
(# without disease) = a/ b
Probability = Odds/ (Odds+1);
Odds = Probability / (1-Probability)
Disease StatusDisease Status
PresentPresent Absent Absent TotalTotal
aa bb a+ba+b
36
Use of Likelihood Ratio
Employment of following three step procedure:
1. Identify and convert the pre-test probability to pre-test odds.
2. Determine the post-test odds using the formula,
Post-test Odds = Pre-test Odds * Likelihood Ratio
3. Convert the post-test odds into post-test probability.
37
Likelihood Ratio: Example
A 52 yr woman presents after detecting 1.5 cm
breast lump on self-exam. On clinical exam,
the lump is not freely movable. If the pre-test
probability is 20% and the LR for non-movable
breast lump is 4, calculate the probability that
this woman has breast cancer.
38
Likelihood Ratio: Solution
First step
Pre-test probability = 0.2
Pre-test odds = Pre-test prob / (1-pre-test prob)
Pre-test odds = 0.2/(1-0.2) = 0.2/0.8 = 0.25
Second step
Post-test odds Pre-test odds * LR
Post-test odds = 0.25*4 = 1
Third step
Post-test probability = Post-test odds / (1 + Post-test odds)
Post-test probability = 1/(1+1) = ½ = 0.5
40
Receiver Operating Characteristic (ROC)
Finding a best test
Finding a best cut-off
Finding a best combination
probably negative
Equivocal
Probably positive
Definitive positive
1 - specificity
a
b
d
c
1 - specificity
sen
sitiv
ity
a
b
d
c
ROC curve constructed from multiple test thresholds
Diseased
Notdiseased
Multiple thresholds evaluated in test
b c da
Receiver Operating Characteristic (ROC)
ROC Curve allows comparison of different tests for the same condition without (before) specifying a cut-off point.
The test with the largest AUC (Area under the curve) is the best.
Features of good diagnosis study
Comparative (compares new test against old test).
Should be a “gold standard”Should include both positive and
negative resultsUsually will involve “blinding” for both
patient, tester and investigator.
BE-Workshop-DT-July2007 65
USERS GUIDES TO THE MEDICAL LITERATURE
How to use an Article about a Diagnostic Test?
Are the results of the study valid?
What are the results and will they help me in
caring for my patients?
66
1. Was there an independent, ‘blind’ comparison with a ‘gold’ standard’ of diagnosis?
2. Was the setting for the study as well as the filter through which the study patients passed, adequately described?
3. Did the patient sample include an appropriate spectrum of disease?
4. Have they done analysis of the pertinent subgroups
5. Where the tactics for carrying out the test described in sufficient detail to permit their exact replication?
Methodological Questions for Appraising Journal Articles about Diagnostic Tests
67
6. Was the reproducibility of the test result (precision) and its interpretation (observer variation) determined?
7. Was the term ‘ normal’ defined sensibly?
8. Was precision of the test statistics given?
9. Was the indeterminate test results presented?
10. If the test is advocated as a part of a cluster or sequence of tests, was its contribution to the overall validity of the cluster or sequence determined?
11. Was the ‘ utility’ of the test determined?
Thank you