Quality Assurance and Control

Objectives

• To define and discuss quality control

• To discuss the key features of the design of epidemiologic studies

• To discuss data control instruments

• To discuss training of staff issues

QualityAssurance

• Steps in Quality Assurance– Specify study hypothesis– Specify general design to test study hypothesis

(study protocol)– Choose and prepare specific instruments (develop

operation manuals)– Train staff (certify staff)– Using trained staff, pretest and pilot-study data

collection– If necessary, modify 2 and 3

Key features of study design (1989 Kahn and Sempos)

• Formulation of the main hypothesis• A priori specification of potential confounding variables• Definitions of the characteristics of the study population• Definition of the design strategy for internal validity• Definitions of the design strategy for reliability and validity• Specifications of the study power• Standardization of procedures• Activities during data collection• Data analysis• Reporting of data

Some quantitative measures of validity and reliability

• Validity– Sensitivity– Specificity– Predictive value positive– Predictive value negative

• Reliability– Youden’s J statistic– Kappa scores

Example of temporal drift in measurement

Phantom measurements

Influence of Prevalence on Predictive Values

0

20

40

60

80

100

120

1 2 3 4 5 6

Prevalence

Pre

dic

tive

Va

lue

(%

)

PPV NPV

Predictive Values at Different Prevalence Rates with Sensitivty .90 and Specificity .90

G.S. + G.S. -N.T. + 9 9N.T. - 1 81

Total 10 90

G.S. + G.S. -N.T. + 22 7N.T. - 3 68

Total 25 75

G.S. + G.S. -N.T. + 45 5N.T. - 5 45

Total 50 50

Prev 10%PPV .50NPV .99

Prev 25%PPV .76NPV .96

Prev 50%PPV .90NPV .90

Spectrum of severity

Predictive Values

Culture + Culture -

PCR+ 80 10

PCR - 20 90

100 100

Kappa Statistic

• po = observed probability of concordance between the two surveys

• pe = expected probability of concordance between the two surveys

• The standard error of the Kappa statistic is calculated by:

• To test the hypothesis Ho:=0 vs. H1:0, use the test statistic:

Percent agreement

Figure 1. Association of average faculty performance rating (from 1, bottom 20%, to 5, top 20%) and absolute rank on the National Resident Matching Program (NRMP) list (r = 0.19; P =.11).

Table 1. Discrepancy Between the DIS and SCAN for the Lifetime Occurrence of Depressive Disorder in the

Baltimore ECA Follow-up*

Table 1-Comparison of WHO and ADA diagnostic categories for undiagnosed diabetes From:   Lee: Diabetes Care, Volume 23(2).February 2000.181-186