Date post: | 16-Jan-2017 |
Category: |
Documents |
Upload: | kenisha-s-russell-jonsson |
View: | 171 times |
Download: | 3 times |
An assessment of the self-reported version of the Swedish Strengths and Difficulties Questionnaire among children and adolescents 12-16
years old
Kenisha S. Russell Jonsson
Irina Vartanova
SDQ studies • Study type 1: Examination of the psychometric properties (alpha
coefficients) Internal consistency
Retest reliability ??
• Study type 2: Factor stucture (factor analysis & SEM)Controversy of the five versus three structure versus bifactor
• Study type 3: Validity (ROC Analysis, mean comparison) specificity & sensitivity
Convergent validity ??
Data • Community Sample
Survey of children and young peoples mental health (Grodan) conducted in 2009, collected data from students in grade 6 and 9 (roughly between age 11-17). In total there were 172,000 respondents.
• Service Contact SampleDuring 1 mars – 30 september 2014 data a collected from 2 648 children and
young people from 27 municipalities in Sweden who visited a healthcarecenter.
Psychometric properties (1)Internal consistency reliability (Cronbachs Alpha) of the total difficulty scores and subscores
SDQ scale Community Service Contact
Widenfeltet al. (2003)
Goodman (2001)
Koskelainen et al. (2000)
Total difficulties 0.63 0.56 0.70 0.80 0.71
Emotional symptoms
0.69 0.67 0.63 0.66 0.69
Conduct problems 0.55 0.55 0.47 0.60 0.57
Hyperactivity-inattention
0.66 0.71 0.66 0.67 0.66
Peer Problems 0.54 0.59 0.39 0.41 0.63
Prosocial 0.68 0.67 0.60 0.66 0.69
Factor structure (1) Community sampleSpecified 5 factor analyses. Oblimin rotation (community sample)
Prosocial Emotion Hyper Conduct Peer
Somatic -0.004 0.582 0.019 0.105 -0.040
Worries 0.044 0.733 -0.014 -0.041 0.040
Unhappy 0.044 0.730 -0.081 0.130 0.154
Clingy -0.059 0.548 0.116 -0.181 0.094
Afraid 0.050 0.493 0.063 -0.100 0.103
Tantrum -0.070 0.408 0.107 0.335 -0.031
Robeys -0.406 0.070 0.032 0.187 -0.205
Fights -0.068 0.011 0.057 0.788 -0.006
Lies -0.022 0.024 0.167 0.489 0.180
Steals -0.062 0.034 0.038 0.544 0.079
Restles -0.006 -0.003 0.757 0.024 -0.044
Fidgety 0.034 -0.047 0.877 0.008 0.037
Distrac -0.137 0.359 0.362 0.076 -0.047
Reflect -0.456 0.137 0.094 0.150 -0.157
Rattends -0.451 0.315 0.212 0.026 -0.110
Loner 0.027 0.198 -0.003 -0.026 0.530
Friend -0.228 0.103 -0.072 0.079 0.638
Popular -0.383 0.117 0.046 -0.152 0.475
Bullied 0.112 0.203 0.032 0.280 0.558
Oldbest 0.250 -0.030 0.069 0.029 0.470
Consid 0.553 0.129 -0.001 -0.244 -0.121
Shares 0.461 0.046 0.031 0.088 -0.165
Caring 0.615 0.258 -0.002 -0.067 -0.217
Kind 0.498 -0.001 0.048 -0.258 -0.002
Helpout 0.779 -0.036 -0.0002 0.003 0.029
Factor structure (2) Service contact sampleSpecified 5 factor analyses. Oblimin rotation (community sample)
Prosocial Emotion Hyper Conduct Peer
Somatic 0.104 0.506 0.058 0.099 0.010
Worries 0.026 0.759 -0.069 -0.108 -0.002
Unhappy 0.108 0.680 -0.017 0.060 0.142
Clingy -0.110 0.553 0.039 -0.131 0.077
Afraid 0.023 0.459 -0.100 0.008 0.034
Tantrum -0.036 0.366 0.094 0.504 0.015
Robeys -0.254 -0.005 0.111 0.413 -0.091
Fights -0.038 -0.026 0.017 0.739 0.024
Lies 0.052 -0.181 0.111 0.558 0.301
Steals -0.191 0.046 0.089 0.366 0.030
Restles -0.013 -0.031 0.914 -0.032 0.045
Fidgety 0.006 -0.012 0.885 0.0002 -0.028
Distrac -0.046 0.396 0.384 0.203 -0.064
Reflect -0.124 -0.015 0.147 0.494 -0.078
Rattends -0.137 0.337 0.294 0.231 -0.036
Loner -0.191 0.247 -0.108 -0.151 0.435
Friend -0.089 0.061 -0.034 -0.089 0.751
Popular -0.198 0.073 0.044 0.095 0.587
Bullied 0.140 -0.022 0.098 0.129 0.661
Oldbest 0.148 0.080 -0.046 0.124 0.426
Consid 0.491 0.052 0.094 -0.479 0.020
Shares 0.561 0.034 -0.013 0.046 -0.144
Caring 0.727 0.104 0.007 -0.023 -0.134
Kind 0.435 -0.025 0.057 -0.275 -0.017
Helpout 0.765 -0.038 -0.090 -0.001 0.066
Validity (1) Descriptive stats
Validity (2) Descriptive stats
Validity (3) ROC Analysis Receiver operating curves
In a ROC curve the true positive rate (Sensitivity) is plotted as a function of the false positive rate (100-Specificity) for different cut-off points of a parameter.
The area under the ROC curve (AUC)
a measure of how well a parameter can distinguish between two diagnostic groups (community/service contact)a method for reducing the entire ROC curve to a single
quantitative index of diagnostic accuracy
Validity (4) Caseness
True positive: caseswith condition
classified as positive
False positive: caseswithout condition
classified as positive False negative: cases with conditionclassified as negative
True negative: caseswithout condition
classified as negative
Validity (5) Sensitivity–Specificity Report
Emotional Problems: Detailed report of sensitivity and specificity
Cutpoint Sensitivity Specificity
Correctly
Classified LR+ LR-
( >= 0 ) 100.00% 0.00% 0.53% 1.0000
( >= 1 ) 96.75% 16.52% 16.94% 1.1590 0.1965
( >= 2 ) 91.64% 35.90% 36.19% 1.4295 0.2330
( >= 3 ) 81.65% 54.18% 54.33% 1.7821 0.3387
( >= 4 ) 71.79% 68.67% 68.68% 2.2909 0.4109
( >= 5 ) 55.93% 79.78% 79.65% 2.7661 0.5524
( >= 6 ) 41.95% 87.91% 87.67% 3.4693 0.6604
( >= 7 ) 27.97% 93.21% 92.86% 4.1158 0.7729
( >= 8 ) 17.10% 96.42% 96.00% 4.7774 0.8597
( >= 9 ) 7.37% 98.42% 97.94% 4.6694 0.9412
( >= 10 ) 3.00% 99.38% 98.87% 4.8338 0.9761
( > 10 ) 0.00% 100.00% 99.47% 1.0000
Cutpoint:indicate the rating used to classifysubjects with/without a condition
Probability of correctlyclassifying those with a condition
Probability of correctlyclassifying those without a condition
The ratio of theprobability of anegative test amongtruly positive subjectsto the probability of anegative test amongtruly negative subjects
The ratio of theprobability of a positivetest among truly positivesubjects to theprobability of a positivetest among truly negativesubjects
Validity (6) AUC
OBS AUC SD LLCI ULCI
Total difficulties 151803 0.7076 0.0089 0.69017 0.72508
Emotional 151803 0.7541 0.0083 0.73778 0.77049
Conduct 151803 0.5761 0.0097 0.55715 0.59510
Hyperactivity-inattention 151803 0.5768 0.0104 0.55634 0.59728
Peer 151803 0.6104 0.0102 0.59051 0.63038
Prosocial 151803 0.5595 0.0098 0.54025 0.57879
Validity (7) ROC-Emotion
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.7541
service contact versus community sample
Emotional Problems
Validity (8) ROC-Conduct
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.5761
service contact versus community sample
Conduct Diffiulties
Validity (9) ROC -Hyper
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.5768
service contact versus community sample
Hyperactivity-inattention
Validity (10) ROC- Peer
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.6104
service contact versus community sample
Peer Problems
Validity (11) ROC-Prosocial
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.5595
service contact versus community sample
Prosocial Behaviour
Validity (12) ROC-Total difficulties
0.0
00
.25
0.5
00
.75
1.0
0
Se
nsitiv
ity
0.00 0.25 0.50 0.75 1.001 - Specificity
Area under ROC curve = 0.7076
service contact versus community sample
Total Difficulties
Dilemma• AUC low for some of the subscores -> community sample is too high
or the service contact sample is too low or vice versa.
How to solve this???
• Compare results with other countries (specifically nordic sample)
• Further analyses, restricting/more emphasis on service samplereason for the visit
number of visit
who contacted the service center (parent/ child/teacher/other adult)
reason for contact
FACTOR STRUCTURES- THE SWEDISH CONTRIBUTION
Explorative (EFA) vs Confirmative (CFA) Factor Analysis• In EFA, the factor structure is inferred from the obtained correlation
matrix.
• In CFA,the obtained correlation matrix is compared with a specified theoretical model.
• The result of comparison is goodness of fit of the specified model. Thus, we can compare different factor structures for better understanding of the analyzed questionnaire.
EFA vs CFACorrelation matrix
Factor structure
Correlation matrix
Theoretical model
compared
Model fit
Bifactor models – the latest suggestion of model fit improvement
Kobor et al., 2013 Casi et al., 2015
Alternative models fitModel chisq df RMSEA CFI TLI
Original 5-factor model 198,004 265 0.068 0.896 0.882
5-factor model with acquiescence style 125,592 259 0.051 0.942 0.932
Alternative 3-factor model 269,164 272 0.080 0.853 0.838
3-factor model with acquiescence style 232,670 268 0.073 0.880 0.865
Bifactor model (Kobor et al., 2013) 96,664 240 0.044 0.961 0.952
Bifactor model (Casi et al., 2015) 164,093 252 0.063 0.914 0.898
Different model fit measures
Best fit model
Model currentlytesting
Reference
Caci, H., Morin, A. J., & Tran, A. (2015). Investigation of a bifactor model of the Strengths and Difficulties Questionnaire. European child & adolescent psychiatry, 24, pp 1291-1301.
Choi,B.C.K. 1998. Slopes of a receiver operating characteritic curve and the likelihood ratio for a diagnostic test.American Journal of Epidemiology 148:1127-1132.
Di Riso, D., Salcuni, S., Chessa, D., Raudino, A., Lis, A., & Altoè, G. (2010). The Strengths and Difficulties Questionnaire (SDQ). Early evidence of its reliability and validity in a community sample of Italian children. Personality and Individual Differences, 49(6), 570-575.
Essau, C. A., Olaya, B., Anastassiou‐Hadjicharalambous, X., Pauli, G., Gilvarry, C., Bray, D., ... & Ollendick, T. H. (2012). Psychometric properties ofthe Strength and Difficulties Questionnaire from five European countries. International journal of methods in psychiatric research, 21(3), 232-245.
Goodman, R. (1997). The Strengths and Difficulties Questionnaire: a research note. Journal of child psychology and psychiatry, 38(5), 581-586.
Goodman, R., Meltzer, H., & Bailey, V. (1998). The Strengths and Difficulties Questionnaire: A pilot study on the validity of the self-report version. European child & adolescent psychiatry, 7(3), 125-130.
Goodman, A., Lamping, D. L., & Ploubidis, G. B. (2010). When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British parents, teachers and children. Journal of abnormal child psychology,38(8), 1179-1191.
Hanley,J.A and B.J. McNeil.1982.The meaning and the use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:9-36.
Kóbor, A., Takács, Á., & Urbán, R. (2013). The bifactor model of the Strengths and Difficulties Questionnaire. European Journal of PsychologicalAssessment, 29, pp. 299-307.