Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | ulysses-soto |
View: | 30 times |
Download: | 0 times |
Measures of Reliabilityand
Multivariate Analysis
Objectives
• To discuss measures of reliability
• To discuss multivariate analysis
Kappa Statistic
• po = observed probability of concordance between the two surveys
• pe = expected probability of concordance between the two surveys
• The standard error of the Kappa statistic is calculated by:
• To test the hypothesis Ho:=0 vs. H1:0, use the test statistic:
Percent agreement
Figure 1. Association of average faculty performance rating (from 1, bottom 20%, to 5, top 20%) and absolute rank on the National Resident Matching Program (NRMP) list (r = 0.19; P =.11).
Table 1. Discrepancy Between the DIS and SCAN for the Lifetime Occurrence of Depressive Disorder in the
Baltimore ECA Follow-up*
Table 1-Comparison of WHO and ADA diagnostic categories for undiagnosed diabetes From: Lee: Diabetes Care, Volume 23(2).February 2000.181-186
Kappa score calculation
PlaqueNormal Expected chance agreementPlaque 140 52 192 40.70 Normal 69 725 794 625.70
209 777 986
Pe= 40.7 626 0.676Po= 140 725 0.877
k = 0.877 - .676 = 0.62----------------1.0 - .676
Kappa and prevalence
• You have two populations of 1000 each being compared by interview A and interviewer B
• One has a prevalence of 5 and the other has a prevalence of 40
• Sensitivities and specificities for observer A and B are .90 and .90 and .80 and .95 respectively
• Create a table for each population
Prevalence and Kappa
• Kappa tends toward 0 as the prevalence approaches either 0 or 1
• High values of kappa can be obtained when the marginals of the contingency table are unbalanced. (tends to be higher when the positivity prevalence is different between A and B
Kappa scores for table 8-18