Evidence based medicine

Diagnostic tests

Ross Lawrenson

Diagnostic tests

• When looking at a paper about a diagnostic test we ask ourselves three questions.

Diagnostic tests

• Is this test useful?

Diagnostic tests

• Is this test useful?

• Is it reliable?

Diagnostic tests

• Is this test useful?

• Is it reliable?

• Is it valid?

Is this test useful?

• The test should have been researched in a study population relevant to the individual or population in whom it is to be used.

Reliability

• Reliability refers to the repeatability or reproducibility of a test.

• It can be assessed by repeating the test using the same or different observers.

Validity

• Relates to whether the test measures what it purports to measure. Is the result true?

Validity

• For example if you measure blood pressure in an obese patient and use a cuff that is too small you are likely to get a falsely high reading. The reading maybe reliable (you get the same blood pressure if you do it again) but it lacks validity.

Sensitivity and specificity

Disease Healthy TotalTest + a b a+bTest - c d c+dTotal a+c b+d

Sensitivity and specificity

Sensitivity and specificity

Disease Healthy Total

Test +ve a b a+b

Test -ve c d c+d

Total a+c b+d a+b+c+d

Sensitivity

• The probability that the test will be positive if the disease is present

• = a/a+c

Sensitivity

• The probability that the test will be positive if the disease is present

• = a/a+c

• A sensitive test is likely to also record a number of false positive tests

Sensitivity

If the cut off point of this test is set low then it will be sensitive (all patients with disease will testpositive) but there will also be a number of false positives

DiseasedHealthy

Specificity

• Theprobability that the test will be negative if the disease is truly absent.

• d/b+d

Specificity

• Theprobability that the test will be negative if the disease is truly absent.

• d/b+d

• In this situation there is a high likelihood of false negatives.

High specificity, low sensitivity

Normal

Abnormal

Sensitivity and specificity

There is usually a trade off between sensitivity and specificity. The more sensitive a test the fewer false negative tests. This is important for a rare and serious diseases such as phenylketonuria. Similarly the more specific a test the fewer false positives that are likely to occur which can be important in common diseases such as diabetes.

Positive predictive value

The probability of truly having the disease when a screening test is positive.

a/a+b

Negative predictive value

The probability of being disease free when the screening test is negative.

d/c+d

Likelihood ratios

For a positive test result = (a/a+c)/(b/b+d)

For a negative test result = (c/a+c)/(d/b+d)

Accuracy of the test

(a+d)/(a+b+c+d)

Example

5000 women underwent a test for blood glucose at 24 weeks following a glucose load. 243 women were found to have a blood glucose greater than 6.8 mmol/L and were referred for an OGTT. 186 were found to have gestational diabetes. Four women who initially had tested negative were diagnosed as having diabetes later in their pregnancy.

Example

Prevalence

Sensitivity

Specificity

Positive predictive value

Negative predictive value

Likelihood ratio + test

Likelihood ratio - test

Accuracy

Diabetes No diabetes Total

Positive 186 57 243

Negative 4 4753 4757

Total 190 4810 5000

Example

Prevalence 190/5000

Sensitivity 186/190

Specificity 4753/4810

Positive predictive value 186/243

Negative predictive value 4753/4757

Likelihood ratio + test (186/190)/(57/4810)

Likelihood ratio - test (4/190)/(4753/4810)

Accuracy 186+4753/5000

Example

Prevalence 3.8%

Sensitivity 97.9%

Specificity 98.8%

Positive predictive value 76.5%

Negative predictive value 99.9%

Likelihood ratio + test 82.6

Likelihood ratio - test .02

Accuracy 98.8%

Gold standard

Gold standard in diabetes is the OGTT. Other tests may have a gold standard that is too expensive or invasive for routine use e.g. fluoroscein angiography for diabetic retinopathy.

Gold standard

.

.

Gold standard• The gold standard is the test or battery of tests that will

most accurately diagnose a particular disease or condition.

• Thus traditionally the OGTT has been seen as the gold standard when testing for diabetes. Other diagnostic tests may have a gold standard that is too expensive or invasive for routine use e.g. fluoroscein angiography for diabetic retinopathy.

• Sometimes the gold standard is a battery of tests or symptoms e.g. the Jones criteria for rheumatic fever

Receiver operator curves

By plotting the sensitivity and specificity of a test for different cut off points a ROC can be produced which helps illustrate the optimum cut off point to use.

Receiver operator curves

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.00

0.2

0.4

0.6

0.8

1

False positive rate

True positive rate

>280

>80>40

ROC for creatinine kinase for diagnosing MI

Receiver operator curves

Different curves may be found for different populations or for different prevalence.

Observer variation

Intra observer variation

Inter observer variation

Percent agreement.

KAPPA

Estimating observer variation

Percent agreement

Abnormal Suspect Normal

Abnormal A B C

Suspect D E F

Normal G H I

Percent agreement = (A+E+I) / Total X100

Pathologist's diagnosis of melanoma

37 cases of melanoma submitted by a panel of melanoma experts of cases they considered definite cases.Reviewed by two pathologistsOne considered 21 cases malignant and 16 benign, the other considered 10 malignant, one indeterminate and 26 benign

Percent agreement

Melanoma Indeterminate Benign

Melanoma 10 1 10

Indeterminate 0 0 0

Benign 0 0 16

Percent agreement = (10+0+16)/37 X100 = 70 %

KAPPA

Second Exam

Normal Retinopathy Total

First Normal 46 10 56

Exam Retinopathy 12 32 44

Total 58 42 100

Observed agreement = 46 + 32/100 = 78%

KAPPA

Second Exam

Normal Retinopathy Total

First Normal 58%x56 42%x56 56

Exam Retinopathy 58%x44 42%x44 44

Total 58 42 100

KAPPA

Second Exam

Normal Retinopathy Total

First Normal 32.5 23.5

Exam Retinopathy 25.5 18.5

Total

Agreement expected by chance=32.5+18.5/100=51%

KAPPA = % obseved agreement - % expected by chance

Estimating observer variation

100% - (percent agreement expected by chance)

KAPPA = 78 - 51/49 = 0.55

Kappa can be between 0 and 1

Usually a score above 0.4 indicates a reasonable level of agreement and above 0.6 is good.

Estimating observer variation

KAPPA

Organ Agreement Kappa

Liver necrosis 47% 0.2

Rectal cancer grading

50-69% 0.1-0.5

Hodgkins classification

56% 0.4

Breast cancer classification

73% 0.4

Bandolier 37