HSS4303B Intro to Epidemiology
Feb 4, 2010 – Screening Tests
Student Obesity Conference
• www.studentobesitymeeting.ca • June 9-12 at uOttawa• Abstract deadline is Feb 12• Registration fee is $95 (includes meals, etc)• 200 students + 25 mentors
CSEB Student Conference
• May 27-28, 2010• Kingston
• Details will be posted on www.cseb.ca
Your Abstracts
• Marks are now posted• 3 people did not submit • Min = 5.7/10 Max=9.5/10 Mean = 7.6/10• No one failed (except the above 3)
Your abstracts – Erin’s comments• The students who got >85% were clear about their research
topics and provided intros and conclusions. • Students who got between 70 and 85% followed the
instructions, but there is some variance in marks due to style/grammar, and the quality of their references.
• Students who received a grade of <70% did not follow the instructions – ie. all of their references were web-based (PHAC, StatsCan,
etc.)– they went way over the word count (some over 400 or 500
words)– and/or there was nothing at all related to epidemiology in their
abstract.
Erin’s Office Hours
• Erin can be available during reading week. Does anyone intend to come by?
• She will not be available March 4• Always available by appointment
Tuberculosis
• What is it?• We apply tuberculin skin test (also called PPD
– purified protein derivative) test• Positive response is an “induration”
– a hard, raised area with clearly defined margins at and around the injection site
What type of curve is this?
Bimodal curve
• ________________ identifies two types of traits in a population
• _________________ separates individuals ho had not prior experience with tuberculosis from those who had prior experience
• Bimodal distribution allows to separate people on the basis of the trait, characteristic or disease
• However, for many of the conditions and diseases people fall under uni-modal distribution
What’s it called when there’s only one hump?
Distribution of systolic blood pressure for men (unimodal distribution)
Unimodal curve• _________________ has a single peak with normal
distribution or tailed distribution• Since it does not categorize people an arbitrary cutoff has to
be used to separate people as hypertensive or normotensive• Cutoff is usually based on statistical evidence, however,
biological, genetic and other information also need to be considered
• Which men are at a higher risk of stroke, myocardial infraction
• Unimodal or bimodal there will still be people in the grey zone and there is uncertainty about these cases
So….
• We are concerned about TB and High BP in the population, and we have screening tests for both
• But you can see that the challenges are different for both types of screening tests
What is a Screening Test?
• A test given to persons who do not show clinical signs of a disease to nonetheless test for that disease
•Validity•Reliability•Sensitivity•Specificity
Examples of screening tests?
• PSA• CT scans• Pregnancy tests• DRE• Phenylketonuria (PKU) Test
Validity
• ability to distinguish between those who have the disease and those who do not have the disease
• i.e., is it detecting what it says it’s detecting?
e.g. PPD Test
• The PPD test purports to test for TB infection• However, it is possible to get a reaction from
the BCG TB vaccine (which is not available in North America)
• With respect to distinguishing between actual TB exposure and vaccine exposure, the PPD test has poor validity
There are many types of validity
• Internal vs External– Refers to the validity of a study– Not relevant for screening tests– We’ll revisit this dude
There are many types of validity
• Construct validity– The extent to which the measurement
corresponds to theoretical concepts– "Are we actually measuring (are these means a
valid form for measuring) what (the construct) we think we are measuring?"
– IQ test
Validity
• Content validity– Also known as “logical validity”– The extent to which the test incorporates all that
is known about the disease– Eg. If test purports to measure “functional health”
then it should include measurements of social happiness, etc, and not just biological markers
Validity
• Criterion validity– The extent to which the test correlates with an
external criterion of the thing you’re studying• Concurrent validity
– The measurement and the criterion refer to the same point in time
– If visually looking at a wound is your test for injuries in a battle, how do you know if the would was inflicted during the battle?
• Predictive validity– The measurement can predict the criterion– SAT scores are a good predictor of freshman marks
Reliability
• Can you repeat the test and get the same result?– Let’s say you’re measuring nose length to
determine cancer risk --will you get different results everytime you measure the same nose?
– Blood pressure has poor reliability because it changes every few minutes
Reliability of Screening TestsReliability of Screening Tests
RELIABILITY:
The extent to which the screening test will produce the same or very similar results each time it is administered.
--- A test must be reliable before it can be valid.
--- However, an invalid test can demonstrate high reliability.
Reliability of Screening TestsReliability of Screening Tests
Sources of variability that can affect the reproducibility of results of a screening test:
1. Biological variation (e.g. blood pressure)
2. Reliability of the instrument itself
3. Intra-observer variability (differences in repeated measurement by the same screener)
4. Inter-observer variability (inconsistency in the way different screeners apply or
interpret test results)
Measures of Validity
• Sensitivity• Specificity
Measure of Validity
• Sensitivity– The probability of correctly diagnosing a case
(case= person with the disease)– i.e. the proportion of truly diseased people who
are identified as diseased by the test
• Specificity– The probability of correctly rejecting a case– i.e., “true negative rate”
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
How do you compute prevalence from these data?
All cases / total pop =(a+c) / (a+b+c+d)
Example: Assume a population of 1,000 people, of whom 100 have a disease and 900 do not have the disease
Screening Test to Identify the 100 People with the Disease
True Characteristics in the Population
Results of Screening Disease No Disease Total
Positive 180
Negative 820
Total 100 900 1,000
Can you fill in the blanks?
Example: Assume a population of 1,000 people, of whom 100 have a disease and 900 do not have the disease
Screening Test to Identify the 100 People with the Disease
True Characteristics in the Population
Results of Screening Disease No Disease Total
Positive 80 100 180
Negative 20 800 820
Total 100 900 1,000
So….
• What’s a false positive?
• What’s a false negative?
So….
• What’s a false positive?– Test says positive but in reality it’s a negative
• What’s a false negative?– Test says it’s negative but in reality it’s a positibe
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
Which are the false positives?Which are the false negatives?
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
Which are the false positives?Which are the false negatives?
False positives and false negatives
• False positives– Burden on health care for follow tests– Anxiety and worry for the people– Psychosocial aspects of the label
• False negatives– Missed being diagnosed and provided with the
timely treatment has compromised prognosis– Shock and disbelief upon diagnosis in advanced
stage of the disease
So….
• We define two more concepts:– Positive Predictive Value (PV+ or PPV)– Negative Predictive Value (PV- or NPV)
These are measures of “performance yield”
PV+
• Also called “precision rate”• Also called “post-test probability of disease”• the proportion of patients with positive test
results who are correctly diagnosed
• Sounds like sensitivity, right?
PV-
• the proportion of patients with negative test results who are correctly diagnosed
Performance YieldPerformance Yield
• People with positive screening test results will also test positive on the diagnostic test:
Predictive Value Positive (PV+)
• People with negative screening test results are actually free of disease
Predictive Value Negative (PV-)
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
PV+ = a/(a+b)
Sensitivity/SpecificityScreening test results
Truly diseases (cases)
Truly non-diseases
Totals
Positive (thinks it’s a case)
a b a+b
Negative (thinks it’s not a case)
c d c+d
totals a+c b +d a+b+c+d
Sensitivity = a/(a+c)Specificity = d/(b+d)
PV+ = a/(a+b)
PV- = d/(c+d)
Relationship between Sens/Spec and PV-/PV+
Performance YieldPerformance Yield
400
98905100
995
True Disease Status
+ -
Results ofScreeningTest
+
-
Compute sensitivity, specificity, PV+ and PV-
Performance YieldPerformance Yield
400
98905100
995
True Disease Status
+ -
Results ofScreeningTest
+
-
Sensitivity: a / (a + c) = 400 / (400 + 100) = 80%
Specificity: d / (b + d) = 98905 / (995 + 98905) = 99%
PV+: a / (a + b) = 400 / (400 + 995) = 29%
PV-: d / (c + d) = 98905 / (100 + 98905) = 99%
Performance YieldPerformance Yield
400
98905100
995
True Disease Status
+ -
Results ofScreeningTest
+
-
PV+: a / (a + b) = 400 / (400 + 995) = 29%
Among persons who screen positive, 29% are foundto have the disease.
Performance YieldPerformance Yield
400
98905100
995
True Disease Status
+ -
Results ofScreeningTest
+
-
PV-: d / (c + d) = 98905 / (100 + 98905) = 99.9%
Among persons who screen negative, 99.9% are foundto be disease free.
Performance YieldPerformance Yield
Factors that influence PV+ and PV-
1. The more specific the test, the higher the PV+
2. The higher the prevalence of preclinicaldisease in the screened population, thehigher the PV+
3. The more sensitive the test, the higherthe PV-
Performance YieldPerformance Yield
Prevalence (%) Sensitivity Specificity PV+
0.1 90% 95% 1.8%
1.0 90% 95% 15.4%
5.0 90% 95% 48.6%
50.0 90% 95% 94.7%
Relationship between prevalence and positive predictive value of a test
Performance YieldPerformance Yield
Thus, the PV+ is maximized when used in “high risk” populations since the prevalence of pre-clinical disease is higher than in the general population….
screening a total population for a relatively infrequent disease can be very wasteful of resources and may yield few previously undetected cases.
Homework• Geenberg p. 105, question 1-13:
– 13786 Japanese patients underwent CT scans to detect first signs of cancer, then had pathology tests 2 years later to confirm whether or not they actually had cancer
CT result Cancer present
Cancer absent
Positive 56 532
negative 4 13194
Compute:
•Prevalence of cancer•Sensitivity & specificity•% of false positives•% of false negatives•PV+ and PV-