Date post: | 03-Jun-2018 |
Category: |
Documents |
Upload: | hanny-fadhila |
View: | 233 times |
Download: | 0 times |
of 39
8/12/2019 How to Appraise Diagnosis
1/39
Appraising Diagnostic Study
Muhammad Ali
Medical School
University of Sumatera Utara
8/12/2019 How to Appraise Diagnosis
2/39
8/12/2019 How to Appraise Diagnosis
3/39
8/12/2019 How to Appraise Diagnosis
4/39
What are tests used for?
Log of reasons by several docs:
Diagnosismost common
but alsoMonitoringhas it changed?
Prognosisrisk/stage within Dx
Treatment planning
8/12/2019 How to Appraise Diagnosis
5/39
Basic principles (1)
Ideal diagnostic testsright answers:
(+) results in everyone with the disease and
( - ) results in everyone else
Usual clinical practice:
The test be studied in the same way it would be used in
the clinical setting
Observational study, and consists of:
Predictor variable (test result)
Outcome variable (presence / absence of the disease)
8/12/2019 How to Appraise Diagnosis
6/39
Basic principles (2)
Sensitivity, specificity
Prevalence, prior probability, predictive values
Likelihood ratios
Dichotomous scale, cutoff points (continuous scale)
Positive (true and false), negative (true and false)
ROC (receiver operator characteristic) curve
8/12/2019 How to Appraise Diagnosis
7/39
EBM Issues:
Is this evidence about the accuracy of adiagnostic test valid?
Does this valid evidence demonstrate an
importantability to accurately distinguishpatients who do and dont have a specificdisorder?
Can I applythis valid,importantdiagnostic
test to a specific patient?
8/12/2019 How to Appraise Diagnosis
8/39
Critical appraisal
Use worksheet
Use supporting software
CAT Maker
Save in CAT Banks
http://localhost/var/www/apps/conversion/tmp/scratch_5/CATmaker/CATMAKER.EXEhttp://localhost/var/www/apps/conversion/tmp/scratch_5/CATmaker/CATMAKER.EXE8/12/2019 How to Appraise Diagnosis
9/39
Is this evidence about a diagnostic test
valid?
Was there an independent, blind comparison with areference (gold) standardof diagnosis?
Was the diagnostic test evaluated in an appropriatespectrum of patients (like those in whom we would useit in practice)?
Was the reference standard applied regardless of thediagnostic test result?
Was the test (or cluster of tests) validated in a second,independent group of patients?
8/12/2019 How to Appraise Diagnosis
10/39
Sensitivity
Specificity
Likelihood ratios
Does this valid evidence demonstrate an important
ability to accurately distinguish patients who doand dont have a specific disorder?
8/12/2019 How to Appraise Diagnosis
11/39
SnNoutDiagnostic test with a very high sensitivity ,
a negative result effectively rules out the
diagnosis
SpPin
Diagnostic test with a very high specificity ,a positive result effectively rules in the
diagnosis
8/12/2019 How to Appraise Diagnosis
12/39
Questions to answer in applying a validdiagnostic test to an individual patient
Is the diagnostic test available, affordable, accurate,
and precise in our setting?
Can we generate a clinically sensible estimate of our
patients pre-test probability?
From personal experience, prevalence statistics,
practice databases, or primary studies
Are the study patients similar to our own? Is it unlikely that the disease possibilities or
probabilities have changed since this evidence was
gathered?
8/12/2019 How to Appraise Diagnosis
13/39
Does early diagnosis really lead to improvedsurvival, or quality of life, or both?
Are the early diagnosed patients willing partnersin the treatment strategy?
Is the time and energy it will take us to confirmthe diagnosis and provide (lifelong) care wellspent?
Do the frequency and severity of the targetdisorder warrant this degree of effort andexpenditure?
Guides for deciding whether a screeningor earlydiagnostic maneuver does more good than harm:
8/12/2019 How to Appraise Diagnosis
14/39
Diagnostic tests
Is not about finding absolute truth, butabout limiting uncertainty
establishes both the necessity and thelogical base for introducing
probabilities, pragmatic test-treatment
thresholds ..Start thinking about
what youre going to do with the results of thediagnostic test, and
whether doing the test will help your patients
8/12/2019 How to Appraise Diagnosis
15/39
TRADITIONAL 2x2 TABLE
DISEASE
Yes No Total
3 7Yesa b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
8/12/2019 How to Appraise Diagnosis
16/39
SENSITIVITY
DISEASE
Yes No Total
3 7Yesa b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
SENSITIVITY
The proportion of people with the diagnosis (n=4)who are correctly identified (n=3)
Sensitivity = a/(a+c) = 3/4 = 75%
FALSENEGATIVES
8/12/2019 How to Appraise Diagnosis
17/39
SPECIFICITY
DISEASE
Yes No Total
3 7Yesa b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
SPECIFICITY
The proportion of people without the diagnosis (n=96)who are correctly identified (n=89)
Specificity = d/(b+d) = 89/96 = 93%
FALSEPOSITIVES
8/12/2019 How to Appraise Diagnosis
18/39
PREDICTIVE VALUES (POS or NEG)
DISEASE
Yes No Total
3 7Yesa b
10a+b
c dNo 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
Positife or Negative predictive valuesare the proportion of disease / nondisease people to all who havepositive/negative tests.
PPV = 3/10 = 30%
NPV = 89/90=89%
8/12/2019 How to Appraise Diagnosis
19/39
PRE-TEST ODDS
DISEASE
Yes No Total
3 7Yes
a b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
In the sample as a whole, the odds of having the diseaseare 4 to 96 or 4% (the PRE-TEST ODDS)
8/12/2019 How to Appraise Diagnosis
20/39
POST-TEST ODDS
DISEASE
Yes No Total
3 7Yes
a b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
In those who score positive on the test, the odds of havingthe disease are 3 to 7 or 43% (the POST-TEST ODDS)
In the sample as a whole, the odds of having the diseaseare 4 to 96 or 4% (the PRE-TEST ODDS)
8/12/2019 How to Appraise Diagnosis
21/39
POST-TEST ODDS
DISEASE
Yes No Total
3 7Yes
a b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
In those who score positive on the test, the odds of having the disease are3 to 7 or 43% (the POST-TEST ODDS)
In the sample as a whole, the odds of having the disease are 4 to 96 or4% (the PRE-TEST ODDS)
In those who score negative on the test, the odds of having the diseaseare 1 to 89 or approximately 1%
8/12/2019 How to Appraise Diagnosis
22/39
DIAGNOSTIC ODDS RATIO
DISEASE
Yes No Total
3 7Yesa b
10a+b
c d
No 1 89 90 c+d
4 96 100
TEST
Totala+c b+d a+b+c+d
The Diagnostic Odds Ratio is the ratio ofodds of having the diagnosis given apositive test to those of having thediagnosis given a negative test
2.38011.0
429.0
891
73
DOR
Potentially useful as an overallsummary measure, but only inconjunction with othermeasures (LR, sensitivity,specificity)
8/12/2019 How to Appraise Diagnosis
23/39
BAYES THEOREM
POST-TEST ODDS =LIKELIHOOD RATIO x PRE-TEST ODDS
8/12/2019 How to Appraise Diagnosis
24/39
Natural frequencies provide a more graphic, easy to
understand way to portray probabilities for both
physicians and patients
Interpreting
Sensitivity &
Specificity
Using natural
frequencies to
calculate Bayes
theorem
8/12/2019 How to Appraise Diagnosis
25/39
In small groups what is the likelihood that this
patient has the disease?
A disease with a prevalence of 3% mustbe diagnosed
There is a test for the disease
It has a sensitivity of 50% and a
specificity of 90%
8/12/2019 How to Appraise Diagnosis
26/39
For a prevalence of 3% - in every 100 patients
97
3 will have the disease
Will not have the disease
8/12/2019 How to Appraise Diagnosis
27/39
97How many will nothave the disease?
For a test with a sensitivity of 50%
For every 100
patients with the
disease 1-2 will test
positive
8/12/2019 How to Appraise Diagnosis
28/39
87
For a test with a specificity of 90%
False positive rate
= 1-Specificity(10%)
For those who dont have the
disease 10 patients will go on
to test positive
8/12/2019 How to Appraise Diagnosis
29/39
87
Given a positive test what are my chances of disease doc?
11 people have a positive test Of
which 1 have will have the disease
13% chance of having the
disease given a positive test
8/12/2019 How to Appraise Diagnosis
30/39
87
Given a negative test what are my chances of not having the
disease doc?
88 people
will have a negative test of
which 1 will have the
disease
False reassurance rate of
1.7%
8/12/2019 How to Appraise Diagnosis
31/39
Describe the result to your patient
For every 100 people who have a
positive test 13 will have the
disease
8/12/2019 How to Appraise Diagnosis
32/39
What happens if I have an negative test
Negative test pretty much
rules out the disease but a
small number 1-2 people
in every 100 will still have
the disease
8/12/2019 How to Appraise Diagnosis
33/39
A disease with a prevalence of 30%must be diagnosed
There is a test for this disease
It has a sensitivity of 50% and aspecificity of 90%
Try it again
what is the likelihood that this patient has the disease?
8/12/2019 How to Appraise Diagnosis
34/39
Prevalence of 30%
Sensitivity of 50%
Specificity of 90%
30
70
15
7
100
22 positive
tests in totalof which 15
have the
disease
About 70%
Disease +ve
Disease -ve
8/12/2019 How to Appraise Diagnosis
35/39
Likelihood Ratio
Sensitivity/1-specificity
50% / 10%
=5
LR a
b
8/12/2019 How to Appraise Diagnosis
36/39
Likelihood Ratio
LR a
b
30
70
100
5 150
70Prevalence of 30%
Sensitivity of 50%
Specificity of 90%
8/12/2019 How to Appraise Diagnosis
37/39
Now consider the FOB screening tests
You find out that your father has undertaken the test
and has a positive resultHe ask you whether he has
cancer?
Prevalence of disease is 0.3% must be
diagnosed.
Sensitivity of 50%
False positive rate 3%.
8/12/2019 How to Appraise Diagnosis
38/39
APPLYING A DIAGNOSTIC TEST IN
DIFFERENT SETTINGS
The Positive Predictive Value of a test will vary(according to the prevalence of the condition in thechosen setting)
Sensitivity and Specificity are usually consideredproperties of the testrather than the setting, andare therefore usually considered to remainconstant
However, sensitivity and specificity are likely to beinfluenced by complexity of differential diagnosesand a multitude of other factors
8/12/2019 How to Appraise Diagnosis
39/39
THANK YOU