NEW RESEARCH Toward Brief “Red Flags” for Autism Screening: The Short Autism Spectrum Quotient and the Short Quantitative Checklist in 1,000 Cases and 3,000 Controls Carrie Allison, Ph.D., Bonnie Auyeung, Ph.D., Simon Baron-Cohen, Ph.D. Objective: Frontline health professionals need a “red flag” tool to aid their decision making about whether to make a referral for a full diagnostic assessment for an autism spectrum condition (ASC) in children and adults. The aim was to identify 10 items on the Autism Spectrum Quotient (AQ) (Adult, Adolescent, and Child versions) and on the Quantitative Checklist for Autism in Toddlers (Q-CHAT) with good test accuracy. Method: A case sample of more than 1,000 individuals with ASC (449 adults, 162 adolescents, 432 children and 126 toddlers) and a control sample of 3,000 controls (838 adults, 475 adolescents, 940 children, and 754 toddlers) with no ASC diagnosis participated. Case participants were recruited from the Autism Research Centre’s database of volunteers. The control samples were recruited through a variety of sources. Participants completed full-length versions of the measures. The 10 best items were selected on each instrument to produce short versions. Results: At a cut-point of 6 on the AQ-10 adult, sensitivity was 0.88, specificity was 0.91, and positive predictive value (PPV) was 0.85. At a cut-point of 6 on the AQ-10 adolescent, sensitivity was 0.93, specificity was 0.95, and PPV was 0.86. At a cut-point of 6 on the AQ-10 child, sensitivity was 0.95, specificity was 0.97, and PPV was 0.94. At a cut-point of 3 on the Q-CHAT-10, sensitivity was 0.91, specificity was 0.89, and PPV was 0.58. Internal consistency was 0.85 on all measures. Conclusions: The short measures have potential to aid referral decision making for specialist assessment and should be further evaluated. J. Am. Acad. Child Adolesc. Psychiatry, 2012;51(2):202–212. Key Words: autism spectrum conditions, red flags, refer- ral, screening, questionnaires A utism spectrum conditions (ASC) are characterized by difficulties in social in- teraction, communication, and adapting to change, alongside unusually narrow interests and strongly repetitive behavior. The diagnostic classification systems define ASC to include autistic disorder, Asperger’s syndrome (AS), atypical au- tism, and pervasive developmental disorder not otherwise specified (PDD-NOS). 1, 2 ASC are cur- rently behaviorally defined. There is much research evidence suggesting that etiology is strongly (al- though not exclusively) genetic 3-6 and neurologi- cal 7 in origin. To date, no clear biological, neuro- logical, or genetic marker can define ASC. Prospective population screening studies indicate that approximately 1% of the child and adult pop- ulation is affected by ASC. 8-10 In recent years, there has been a shift in the conceptualization of ASC from a categorical to a dimensional model, and the development of dimensional measures that can measure autistic traits as individual differences that run right through the general population. 11 Diagnosis of ASC can be a lengthy process because it varies greatly across individuals. 12 The age at which symptoms first appear also differs across individuals, 13 and changes in symptom Supplemental material cited in this article is available online. This article can be used to obtain continuing medical education (CME) category 1 credit at jaacap.org. JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY VOLUME 51 NUMBER 2 FEBRUARY 2012 202 www.jaacap.org
Transcript
NEW RESEARCH
Toward Brief “Red Flags” for AutismScreening: The Short Autism SpectrumQuotient and the Short QuantitativeChecklist in 1,000 Cases and 3,000
ControlsCarrie Allison, Ph.D., Bonnie Auyeung, Ph.D., Simon Baron-Cohen, Ph.D.
Objective: Frontline health professionals need a “red flag” tool to aid their decision makingabout whether to make a referral for a full diagnostic assessment for an autism spectrumcondition (ASC) in children and adults. The aim was to identify 10 items on the AutismSpectrum Quotient (AQ) (Adult, Adolescent, and Child versions) and on the QuantitativeChecklist for Autism in Toddlers (Q-CHAT) with good test accuracy. Method: A case sampleof more than 1,000 individuals with ASC (449 adults, 162 adolescents, 432 children and 126toddlers) and a control sample of 3,000 controls (838 adults, 475 adolescents, 940 children, and754 toddlers) with no ASC diagnosis participated. Case participants were recruited from theAutism Research Centre’s database of volunteers. The control samples were recruited througha variety of sources. Participants completed full-length versions of the measures. The 10 bestitems were selected on each instrument to produce short versions. Results: At a cut-point of6 on the AQ-10 adult, sensitivity was 0.88, specificity was 0.91, and positive predictive value(PPV) was 0.85. At a cut-point of 6 on the AQ-10 adolescent, sensitivity was 0.93, specificitywas 0.95, and PPV was 0.86. At a cut-point of 6 on the AQ-10 child, sensitivity was 0.95,specificity was 0.97, and PPV was 0.94. At a cut-point of 3 on the Q-CHAT-10, sensitivity was0.91, specificity was 0.89, and PPV was 0.58. Internal consistency was �0.85 on all measures.Conclusions: The short measures have potential to aid referral decision making forspecialist assessment and should be further evaluated. J. Am. Acad. Child Adolesc.Psychiatry, 2012;51(2):202–212. Key Words: autism spectrum conditions, red flags, refer-ral, screening, questionnaires
lPtu
a
A utism spectrum conditions (ASC) arecharacterized by difficulties in social in-teraction, communication, and adapting
to change, alongside unusually narrow interestsand strongly repetitive behavior. The diagnosticclassification systems define ASC to include autisticdisorder, Asperger’s syndrome (AS), atypical au-tism, and pervasive developmental disorder nototherwise specified (PDD-NOS).1, 2 ASC are cur-rently behaviorally defined. There is much research
Supplemental material cited in this article is available online.
This article can be used to obtain continuing medical education
a(CME) category 1 credit at jaacap.org.
JOURN
202 www.jaacap.org
evidence suggesting that etiology is strongly (al-though not exclusively) genetic3-6 and neurologi-cal7 in origin. To date, no clear biological, neuro-ogical, or genetic marker can define ASC.rospective population screening studies indicate
hat approximately 1% of the child and adult pop-lation is affected by ASC.8-10 In recent years, there
has been a shift in the conceptualization of ASCfrom a categorical to a dimensional model, and thedevelopment of dimensional measures that canmeasure autistic traits as individual differences thatrun right through the general population.11
Diagnosis of ASC can be a lengthy processbecause it varies greatly across individuals.12 Thege at which symptoms first appear also differs
cross individuals,13 and changes in symptom
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
profiles occur across the lifespan. Diagnosis isoften delayed because ASC can be difficult todetect in very young children. Parents may raiseconcerns about their child by 18 months,14 butthere is frequently a significant delay betweenthe point of first concern and an eventual diag-nosis. This may in part be due to communitypediatricians or primary care providers not beingsufficiently informed about the more subtle man-ifestations of ASC. The average age of a diagnosisfor individuals with AS is 11 years.15 However, itis clear that there are individuals with unde-tected ASC in the population9 who may be strug-gling and would benefit from support.
ASC costs the United Kingdom approximately£28 billion sterling each year,16 and similar healtheconomic estimates have been reported in theUnited States.17 Health and social care serviceshave a key responsibility to recognize ASC, yetlevels of awareness and understanding of ASCamong health care and social care agencies differgreatly from one area to another. For example,the UK National Audit Office asked GeneralPractitioners (GP) to estimate how many adultsthey had seen with suspected ASC in their prac-tice over the previous 6-month period. The aver-age response was two patients.18 Given the aver-age size of GP practices is 6,500 patients,19 onewould have expected them to see approximately65 cases of ASC per year. If we assume that halfof these might be in the adult age range, thissuggests underdetection could be 16-fold. In ad-dition, 80% of GPs indicated that they requireguidance to identify persons who may be on theautistic spectrum. GPs or family physicians maybe the first point of contact for parents of childrenwith concerns about autistic traits, as well as foradults with concerns that they themselves possi-bly have “high functioning” ASC or AS. Familyphysicians need to be able to identify childrenand adults who may require a specialist diagnos-tic assessment and therefore need to make anappropriate referral. Furthermore, child careworkers (e.g., nursery staff) have many oppor-tunities to observe children in their care and,over time, will develop a sound knowledgeof what counts as typical development. Onerecent study demonstrated that screening mea-sures designed for child care workers per-formed equally well to detect ASC as parent-report screening instruments.20
To improve diagnosis, brief instruments
would be useful for frontline clinicians and social
JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
care professionals as “red flags” to alert them tomake a referral for a full diagnostic assessment.Glascoe21 recommends that standards for theidentification of a screened condition (sensitivity)on a single administration should be between70% and 80%. Furthermore, to avoid over-referral, specificity should be close to 80%. Overthe past 20 years, most efforts have gone intodeveloping screening measures for ASC in earlychildhood. The first attempt was by our groupwhich evaluated the Checklist for Autism inToddlers (CHAT) in a large population.22-24 Keydomains assessed by the CHAT are absence ofjoint attention and pretend play in a child at 18months of age. A large population study (n �16,000) identified 10 of 12 (83.3%) children whoconsistently failed to show these key behaviors at18 months went on to develop autism23. How-ever, the CHAT had poor sensitivity (�40%),despite good specificity (�90%). The Social Com-munication Questionnaire (SCQ)25 is a parent-ated questionnaire that can be completed inpproximately 10 minutes, with a binary re-ponse format. Allen et al.26 found that sensitiv-
ity and specificity were 0.60 and 0.70 respectivelywhen the SCQ was assessed in a sample of 81preschool children, whereas investigators in an-other study27 found sensitivity and specificityboth to be 0.71 in a much larger sample. TheModified Checklist for Autism in Toddlers (M-CHAT)28 is a modified version of the CHAT,designed to be used in the American health caresystem. Robins et al.28 reported sensitivity was.97, specificity was 0.99, and PPV was 0.68.owever, these psychometric calculations were
ased on the assumption that there were no casesf ASC in those who screened negative on the-CHAT, so these findings should be treatedith caution. There are a great many other in-
truments designed to detect ASC, including thearly Screen for Autistic Traits,29,30 the Social
Responsiveness Scale,31 and the First Year Inven-tory,32,33 and have been validated in differentettings and at different ages. However, there areo fully validated measures to detect possibleSC in primary care and social care settings.The majority of measures developed to detect
SC have focused on young children. There islso a need for an instrument to measure autisticraits in adulthood. Barnard et al.34 found that
46% of those individuals with a diagnosis ofAsperger’s syndrome did not receive the diagno-
sis until late adolescence or adulthood. The Au-
203www.jaacap.org
tfhauvgqwfcoiAsfimTdtm
Apwa
ALLISON et al.
tism Spectrum Quotient (AQ)11 was developed tomeasure the degree to which adults with averageintelligence exhibit autistic traits. The 50-item AQis structured around five subdomains that arecharacteristic of individuals with ASC. The sub-domains are social interaction, communication,attention to detail, attention switching and imag-ination. Individuals diagnosed with ASC scoresignificantly higher on the AQ than persons inthe general population. The AQ has been usedextensively in research studies. It has good dis-criminative validity and screening properties at athreshold score of 26 in a clinical sample35 andexcellent discriminative validity and screeningproperties at a threshold score of 32 in a case-control sample.11 It is normally distributed, and80% of people with ASC score above 32 out of amaximum of 50, compared with only 2% ofcontrols. Child36 and adolescent37 parent-reportversions of the AQ have been developed. Both ofthese versions also discriminate between individ-uals with a diagnosis on the autism spectrum, ina cross-sectional sample. In this paper we consi-der whether the AQ—in these three differentversions—can be adapted for use in primary andsocial care as ‘red flags’ and assist with referraldecision making.
The AQ has produced consistent results acrosstime38 and culture,39-41 and scores are highlyheritable, as demonstrated in a twin study.42
Evidence suggests that the AQ is also correlatedwith biological factors such as salivary testoster-one levels,43 decreased neural white matter vol-ume in the posterior superior temporal sulcus,44
brain functional activity in the superior frontalgyrus45 and medial prefrontal cortex,46 and sin-gle nucleotide polymorphisms in candidategenes.5 Prenatal testosterone levels have alsobeen shown to predict child AQ scores.47
An early developmental version of the AQ isthe Quantitative Checklist for Autism in Tod-dlers (Q-CHAT), which is a revision of theCHAT. The Q-CHAT enables parents to quantifyautistic traits in children 18 to 30 months of ageand to discriminate children who may be on adevelopmental trajectory for ASC from thosewho are developing typically. A large-scale pop-ulation screening study (n � 4,000) is underwayto assess the validity of the Q-CHAT, but the25-item Q-CHAT has already shown excellentpower to discriminate young children with anASC diagnosis from unselected toddlers at 18 to
24 months.48 The key difference between the a
JOURN
204 www.jaacap.org
CHAT and the Q-CHAT is to move from cate-gorical to dimensional screening (Q denotes“quantitative”). Use of a quantitative measureconfers upon the instrument the power todetect more subtle manifestations of ASC. Likethe AQ, the Q-CHAT has good test–retest reli-ability and adequate internal consistency. TheQ-CHAT is also normally distributed.48 TheQ-CHAT score also reflects biological process-es; for example, it correlates with fetal testos-terone levels in typically developing toddlers49
and atypical electrophysiological response inresponse to social stimuli in infant siblings ofchildren with ASC.50
For all of these reasons, the three versions ofthe AQ and the Q-CHAT are strong candidatesfor being useful “red flag” instruments forASC. However, the AQ and Q-CHAT are 50and 25 items, respectively. Arguably these aretoo lengthy to be used in a busy primary carepractice in which the average appointmenttime is 11 minutes,51 as the full-length versionsake 10 minutes to complete. Although theull-length versions can be used comfortably atome or online by families or individuals, theim of the present study is to adapt these forse in clinics by developing short (10-item)ersions of these measures. These would fill theap for health care professionals in makinguick decisions in real clinic time abouthether to refer patients to specialist services
or ASC, without sacrificing the excellent psy-hometric properties of these instruments. Thebjective of the study was to identify which 10
tems from each of the adult AQ, adolescentQ, child AQ, and Q-CHAT would show the
ame levels of excellent sensitivity and speci-city as the full-length versions of these instru-ents in available case and control samples.
his study therefore represents the first step ineveloping the measures, rather than testing
he instruments in the context in which theyay have the most clinical utility.
METHODMeasuresFull details of the construction of all versions of the AQand the Q-CHAT can be found elsewhere.11,36,37,48 The
Q consists of a series of 50 statements to whicharticipants or parents have to indicate the degree tohich they agree or disagree with the statement. There
re four response options: strongly agree, slightly
gree, slightly disagree, strongly disagree. On half the
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
tattdoclwah
ist for
SHORT VERSIONS OF THE AQ AND Q-CHAT
items, the autistic trait requires a response of slightlyagree or strongly agree, and on half the items slightlydisagree or strongly disagree is the response thatidentifies an autistic trait. Each autistic trait endorsedscores one point, regardless of whether the individualindicated slightly or strongly agree or disagree. TheChild AQ was originally scored in a Likert 0, 1, 2, 3format, but for consistency, all versions of the AQ werescored in a binary format. A total score is determinedby summing all the items. The adult AQ is self-report,whereas the child and adolescent versions are parent-report.
The Q-CHAT consists of 25 questions focusing onbehaviors that reflect autistic traits in very early child-hood. Each item has five response options based onfrequency to which the child exhibits the behavior. Ahigh frequency of an autistic trait scores 4, and a lowfrequency of an autistic trait scores 0. Half the itemsare reverse scored. For consistency of the method todetermine the best 10 items, the Likert rating scale wasconverted to a binary scoring system so that a score of0 or 1 would score 0, and a score of 2, 3, or 4 would
TABLE 1 Participant Characteristics
Measure
ControlDerivation
SampleDe
S
AQ AdultSex
Female 249Male 170
Total 419Mean age in years (SD) 33.53 (12.48) 35.0
AQ AdolescentSex
Female 134Male 104
Total 238Mean age in years (SD) 13.46 (1.05) 13.3
AQ ChildSex
Female 256Male 214
Total 470Mean age in years (SD) 9.26 (1.30) 7.2
Q-CHATSex
Female 180Male 197
Total 377Mean age in months (SD) 20.85 (2.15) 36.3
Note: AQ � Autism Spectrum Quotient; Q-CHAT � Quantitative Checkl
score 1. P
JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
ParticipantsThis study tested a group of cases and a group ofcontrols for each of the four measures. Analysis foreach measure was further split into derivation caseand control and validation case and control samples.Therefore a total of 16 participant groups participated.A summary of participant characteristics is given inTable 1.Adult Sample. Adults with a diagnosis of ASC regis-ered as volunteers on our Web site (www.utismresearchcentre.com). They provided details aboutheir diagnosis, including information about who madehe diagnosis, where it was made, and when. Only casesiagnosed at a recognized clinic by a recognized medicr clinical psychologist using DSM-IV criteria were in-luded. After registration, volunteers completed an on-ine version of the AQ. Altogether, there were 449 adults
ith ASC (n � 402 with AS, n � 47 with HFA), of whichpproximately half formed the derivation sample andalf formed the validation sample.
Adult control data were collected at the Cambridge
psychology.com). This site is for people from thegeneral population who are interested in taking part inresearch. The registration procedure for control volun-teers is identical to the procedure for adults with anASC diagnosis. Control adults also completed the AQonline. Only adults more than 16 years of age who didnot report any neurodevelopmental diagnosis wereincluded in the study. The derivation and validationcontrol samples comprised a total of 838 adults.Adolescent Sample. Parents of adolescents between theages of 12 and 15 with a diagnosis of ASC registered onour Web site and completed the AQ-adolescent. Again,parents provided details about their child’s diagnosis,including information about who made the diagnosis,where it was made, and when. Only cases diagnosedat a recognized clinic by a recognized medic or clinicalpsychologist using DSM-IV criteria were included.Altogether, there were 162 adolescents with ASC (n �91 with AS, n � 26 with HFA, n � 37 with autism, n �4 with PDD, and n � 4 with atypical autism). n � 81formed the derivation case sample, and n � 81 formedthe validation case sample.
Parents of adolescents who were participating in alarge epidemiological study of social communicationskills9 were sent the AQ-adolescent through the post.Only adolescents (aged 12-15 years) whose parents didnot report any neurodevelopmental diagnosis wereincluded in the study. The derivation and validationcontrol samples comprised 475 adolescents.Child Sample. Recruitment for the child samples wasthe same as for the adolescent samples. Altogether,there were 432 children (aged 4-11 years) with ASC(n � 158 with AS, n � 81 with HFA, n � 160 withautism, n � 26 with PDD, and n � 7 with atypicalautism). n � 216 formed the derivation case sampleand n � 216 formed the case validation sample. Onlycontrol children, who were 4-11 years of age, takenfrom the dataset published in Auyeung et al.36 andwhose parents did not report any neurodevelopmentaldiagnosis were included. The derivation and valida-tion control samples comprised 940 children.Preschool Sample. Parents of preschool childrenbetween the ages of 15 and 47 months with adiagnosis of ASC registered on our Web site (www.autismresearchcentre.com) and completed the Q-CHAT. Altogether, there were 126 preschool chil-dren with ASC (n � 10 with AS, n � 11 with HFA,n � 90 with autism, n � 11 with PDD, and n � 4with atypical autism) for whom Q-CHAT data wereavailable. Again, parents provided details aboutwho made the diagnosis, where it was made andwhen. Only cases diagnosed at a recognized clinicby a recognized medic or clinical psychologist usingDSM-IV criteria were included. n � 63 formed thederivation case sample and n � 63 formed thevalidation case sample. The sample (N � 754) pub-lished by Allison et al.48 comprised the control
derivation and validation samples. s
JOURN
206 www.jaacap.org
ProcedureParticipants were randomly allocated to derivationand validation samples. The best 10 items from eachmeasure were determined from the derivation samplesby calculating a discrimination index (DI) for eachitem.52 This is calculated by subtracting the proportion
f participants who scored 1 (autism trait positiveesponse) on each item in the control group from theroportion of participants who scored 1 in the ASCroup. Good items on a measure are indicated by aiscrimination index of 0.3 to 0.7. On all versions of theQ, the two items with the highest DI within each
ubscale were chosen. On the Q-CHAT, the 10 itemsith the highest DI were chosen.Receiver operating characteristic (ROC) curves
omprising the 10 most discriminating items for eacheasure were produced on the validation samples.OC curves plots sensitivity and 1-specificity of allossible scores on the measure. The presence of aiagnosis of ASC was the dependent variable and AQr Q-CHAT score was the independent predictor vari-ble. The area under the curve (AUC) is a measure ofhe overall predictive validity, where an AUC � 0.50ndicates random prediction of the independent vari-ble. An AUC of �0.90 indicates excellent validity. TheUC was calculated for each 10-item measure, and
ompared with the AUC for the full versions.Independent-samples t tests were conducted to
ompare the 10-item measures between case individ-als and controls. Internal consistency (Cronbach’slpha) was calculated for each measure. Correlationsere examined between total scores on the short and
ong forms of all questionnaires. The collection of theQ and Q-CHAT online at our Web sites received a
avorable ethical opinion from the University of Cam-ridge Psychology Research Ethics Committee.
RESULTSResults from the item analysis for all measuresare presented in Tables S1 to S4, available online.
he 10 items with the highest DI are presented inable 2 (AQ) and Table 3 (Q-CHAT). The AUC
or all the measures (long and short versions) ishown in Table 4, indicating that all the shortersions all had AUC of �0.90. The AUC valueas marginally higher for the short version on
he Q-CHAT and Child AQ than the long ver-ion. ROC curves for the long and short versionsf each measure are displayed in Figure S1,vailable online. The coordinates of the curvendicating the score at various sensitivities andpecificities are shown in Table S5, availablenline.ase-Control Comparisons. Adult AQ. There was a
ignificant difference in AQ-10 Adult scores for
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
case individuals (mean � 7.93, standard devia-tion [SD] � 1.93) and controls (mean � 2.77,SD � 2.00); t(642) � �31.71, p � .0001 (equalvariances assumed). The magnitude of the differ-ences in the means was large (eta squared �0.62). Cronbach’s alpha for the AQ-10 (Adult)was 0.85. The AQ-10 (Adult) significantly corre-
TABLE 2 Most Discriminating 10 Items on All Versions oPredictive Value (PPV)
Subscale AQ Adult
Attention toDetail
I often notice small sounds whenothers do not (5). PPV � 0.46
S/
I usually concentrate more onthe whole picture, rather thanthe small details (28). PPV �
0.53
S/
AttentionSwitching
I find it easy to do more thanone thing at once (32). PPV �
0.61
In
If there is an interruption, I canswitch back to what I wasdoing very quickly (37).PPV � 0.57
If
Communication I find it easy to ‘read betweenthe lines’ when someone istalking to me (27). PPV �
0.70
S/
I know how to tell if someonelistening to me is gettingbored (31). PPV � 0.76
S/
Imagination When I’m reading a story I findit difficult to work out thecharacters’ intentions (20).PPV � 0.76
W
I like to collect informationabout categories of things(e.g., types of car, types ofbird, types of train, types ofplant, etc) (41). PPV � 0.56
S/
Social I find it easy to work out whatsomeone is thinking or feelingjust by looking at their face(36). PPV � 0.70
S/
I find it difficult to work outpeople’s intentions (45). 0.63
S/
Note: Numbers in parentheses indicate item number.
lated with the AQ-50 (Adult) (r � 0.92, p � .0001).
JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
Adolescent AQ. There was a significant differ-ence in AQ-10 adolescent scores for case individ-uals (mean � 8.40, SD � 1.69) and controls (mean �1.78, SD � 1.80); t(146.52) � �29.96, p � .0001(equal variances not assumed). The magnitude ofthe differences in the means was large (etasquared � 0.74). Cronbach’s alpha for the AQ-10
Autism Spectrum Quotient (AQ), Including Positive
AQ Adolescent AQ Child
tices patterns in thingse time (23). PPV �
S/he often notices small soundswhen others do not (5).PPV � 0.49
ually concentrateson the whole picture,
r than the small detailsPPV � 0.50
S/he usually concentrates moreon the whole picture, ratherthan the small details (28).PPV � 0.51
ial group, s/he cankeep track of several
ent people’srsations (10). PPV �
In a social group, s/he caneasily keep track of severaldifferent people’sconversations (10). PPV �
0.68is an interruption, s/hewitch back to whatwas doing veryly (37). PPV � 0.65
S/he finds it easy to go backand forth between differentactivities (32). PPV � 0.74
equently finds that s/esn’t know how toa conversation goingPPV � 0.65
S/he does not know how tokeep a conversation goingwith his/her peers (26).PPV � 0.87
good at social chit-(38). PPV � 0.70
S/he is good at social chit-chat(38). PPV � 0.82
/he was younger, s/ed to enjoy playings involving pretending
other children (40).0.66
When s/he is reading a story,s/he finds it difficult to workout the characters’ intentionsor feelings (20).PPV � 0.73
ds it difficult toine what it would be
be someone elsePPV � 0.63
When s/he was in preschool,she used to enjoy playinggames involving pretendingwith other children (40). PPV� 0.73
ds social situations(11). PPV � 0.66
S/he finds it easy to work outwhat someone is thinking orfeeling just by looking attheir face (36). PPV � 0.77
ds it hard to makefriends (22). PPV �
S/he finds it hard to make newfriends (22). PPV � 0.74
f the
he noall th0.48he usmorerathe(28).a soceasilydifferconve0.67therecan ss/hequickhe frhe dokeep(26).he ischat
hen she usgamewithPPV �
he finimaglike to(42).
he fineasy
he finnew0.63
(Adolescent) was 0.89. The AQ-10 (Adolescent)
207www.jaacap.org
ALLISON et al.
significantly correlated with the AQ-50 (Adoles-cent) (r � 0.95, p � 0.0001).Child AQ. There was a significant difference inAQ-10 child scores for case individuals (mean �8.64, SD � 1.43) and controls (mean � 1.81, SD �1.57); t(684) � �54.33, p � .0001 (equal variancesassumed). The magnitude of the differences inthe means was large (eta squared � 0.81). Cron-bach’s alpha for the AQ-10 (Child) was 0.90. TheAQ-10 (Child) significantly correlated with theAQ-50 (Child) (r � 0.94, p � .0001).Q-CHAT. There was a significant difference inQ-CHAT-10 scores for case individuals (M �6.90, SD � 2.70) and controls (mean � 1.03, SD �1.32); t(67.01) � �16.94, p � .0001 (equal vari-ances not assumed). The magnitude of the differ-ences in the means was large (eta squared �0.40). Cronbach’s alpha for the Q-CHAT-10 was0.88. The Q-CHAT-10 significantly correlatedwith the Q-CHAT-25 (r � 0.79, p � .0001).Cut-Points on the Short Screeners. All versions ofthe AQ and the Q-CHAT had very high testaccuracy properties in their short (10 item) forms.
TABLE 3 Most Discriminating 10 Items on theQuantitative Checklist for Autism in Toddlers (Q-CHAT)
Q-CHAT
Does your child look at you when you call his/hername? (1). PPV � 0.80
How easy is it for you to get eye contact with yourchild? (2). PPV � 0.78
Does your child point to indicate that s/he wantssomething (eg, a toy that is out of reach) (5). PPV �
0.55Does your child point to share interest with you (eg,
pointing at an interesting sight)? (6). PPV � 0.55Does your child pretend (e.g., care for dolls, talk on a
toy phone)? (9). PPV � 0.51Does your child follow where you’re looking? (10). PPV
� 0.55If you or someone else in the family is visibly upset,
does your child show signs of wanting to comfortthem? (eg, stroking their hair, hugging them)? (15).PPV � 0.28
Would you describe your child’s first words as (typical):(17). PPV � 0.70
Does your child use simple gestures (eg, wavegoodbye)? (19). PPV � 0.76
Does your child stare at nothing with no apparentpurpose? (25). PPV � 0.48
On all versions of the AQ, the cut-point that best
JOURN
208 www.jaacap.org
balanced sensitivity and specificity was 6, and onthe Q-CHAT it was 3. At a cut-point of 6 on theAQ-10 adult, sensitivity was 0.88, specificity was0.91, and positive predictive value (PPV) was0.85 (pretest odds � 0.54). At a cut-point of 6 onthe AQ-10 adolescent, sensitivity was 0.93, spec-ificity was 0.95 and PPV was 0.86 (pretest odds �0.33). At a cut-point of 6 on the AQ-10 child,sensitivity was 0.95, specificity was 0.97 and PPVwas 0.94 (pretest odds � 0.85). At a cut-point of3 on the Q-CHAT-10, sensitivity was 0.91, speci-ficity was 0.89, and PPV was 0.58 (pretest odds �0.16). Internal consistency was high on all mea-sures (�0.85).
DISCUSSIONThis study set out to adapt the AQ (child, ado-lescent, and adult versions) and the Q-CHAT intoshort versions for use in primary or social caresettings by busy frontline health care profession-als as rapid screeners or “red flags” to serve asguides for referral. The study demonstrated thatall versions of the AQ and the Q-CHAT havevery high test accuracy properties in their short(10-item) forms. Internal consistency was high onall measures (�0.85). Anastasi53 suggested thatCronbach’s alpha should be at least 0.85 if aninstrument is to be used to draw inferencesconcerning an individual. These results demon-strate that the short versions are as good (if not
TABLE 4 Area Under the Curve for All Measures (Shortand Long Versions)
Note: AQ � Autism Spectrum Quotient; CI � confidence interval; SE �
standard error; Sig. � significance; Q-CHAT � Quantitative Check-
list for Autism in Toddlers.
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
(e
aiwdoer
pgmtfbvefcwA
SHORT VERSIONS OF THE AQ AND Q-CHAT
better in the cases of the AQ-10 child and Q-CHAT-10) than the long versions.
The items that were selected for the shortforms of the questionnaires were derived andtested on independent samples. That is, the itemswere derived in one sample and test accuracywas assessed on a separate, nonoverlapping,independent sample, to avoid circularity. Thesame items appeared in the short forms on atleast two of the three versions of the AQ, with theexception of one of the items in the imaginationsubdomain that was different for all three ver-sions of the AQ. This suggests that autistic traitsare stable across the lifespan.
The Q-CHAT cut-point is lower comparedwith the cut-point for the versions of the AQ.Since the control sample of Q-CHAT data werecollected when the child was 18 to 24 months, itis likely that there are children within this samplewith high scores who may have subsequentlyreceived a diagnosis. If anything, this wouldhave served to reduce the size of the groupdifferences. This may also apply to the controlsamples for the versions of the AQ but to a lesserextent, as we would expect autistic traits tobecome more stable with age. Diagnoses are lessstable in high-functioning children under the ageof 2 years.54-56 Möricke et al.54 argue that tran-sience may exist for subtle subclinical autistictraits in very young infants that may go unno-ticed, but these traits may become more visiblewith the increasing demands for reciprocal socialinteraction with others in adolescence. Alterna-tively, these traits may remain subclinical, reduceover time, or resolve altogether, which couldpotentially explain the somewhat lower positivepredictive value of the Q-CHAT as comparedwith the other measures presented.
There are limitations to this study that must beacknowledged. First, the analyses presented herewere conducted retrospectively. That is, all indi-viduals (or their parents) in the case groups forwhom AQ or Q-CHAT data were available pro-vided the data following diagnosis. Increasedawareness about ASC may have led respondentsto answer in the expected direction (i.e., endors-ing the presence of the autistic trait). Futureresearch should repeat this study using a pro-spective design, aiming to replicate our results indifferently ascertained samples, particularly insettings where children and adults interface withnonspecialists who have limited knowledge and
experience of ASC. Measurement equivalence c
JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
the comparability of data obtained from differ-nt groups57) is a discussion beyond the scope of
this article, but we acknowledge that this shouldbe addressed in future studies. Second, themethod of administration across samples was notconsistent; with the exception of the AQ adultsamples, all case data were completed online,and control data by post. Although we do notthink that this will have had a significant effecton the results, this issue too could be addressedin future studies.
Third, because of resource limitations, it wasnot possible to independently validate diagnosticstatus in either the case or control groups. How-ever, we adopted a strict and conservativeapproach; individuals were included as case in-dividuals in this study only if sufficient informa-tion was available about their diagnosis. Thislimitation is balanced by the large samples ofindividuals diagnosed with ASC that we wereable to include in this study; in total, more than1,000 individuals with ASC and 3,000 controlswere included. Given the lengthy process re-quired to make an independent research diagno-sis of ASC, this study would not have beenpossible without taking the reported diagnosison trust. A recent study in the United Statesfound that 98% of individuals who had reporteda diagnosis of ASC were validated through med-ical record checks.58 Furthermore, the accuracy of
Web-based approach to autism phenotypingmplemented within the Interactive Autism Net-
ork (IAN) has been examined. This studyirectly assessed participants with a diagnosisf ASC on their network. The clinician’s best-stimate diagnosis agreed with the diagnosiseported by the families in 98% of cases.59 Sup-
port for using the Internet for data collection isprovided by Gosling et al.,60 who found that data
rovided by Internet methods are of at least asood quality as those provided by traditionalethods. Taken together these findings suggest
hat scientists can confidently recruit participantsor autism research through Web-based data-ases. As Daniels et al.58 point out, participants inoluntary research projects do not represent thentire population of children with ASC and theiramilies. However, all of the participants in-luded in this study share something in commonith nonresearch participants with a diagnosis ofSC, that is, they have all at some stage had
oncern expressed about them, been referred,
209www.jaacap.org
tjpwatpabaa
ALLISON et al.
assessed, and diagnosed by a suitably qualifiedhealth professional.
A final limitation is that there are unequalproportions of individuals with different subtypediagnoses within the case groups. For example,in the adult case sample, 90% of the participantshave a diagnosis of Asperger syndrome, com-pared with 37% in the child case sample. Adultswith autism and learning disability are underrep-resented in the adult volunteers who register onthe Web site. Therefore, it must be acknowledgedthat the composition of the case samples differ byage which may reflect systematic bias in relationto the participants who register on the Web siteto be volunteers.
We are not proposing that these instrumentsbe used as population screening instruments,as taking that step would require evaluation inan unselected population, and this has not yetbeen done. We believe that it is unlikely thatgeneral population screening will ever be apractical solution to detection for ASC acrossthe lifespan. There are many costs associatedwith population screening, including the psy-chological impact for an incorrect positivescreening result, a delay in accessing help foran incorrect negative screening result, and apotentially costly increased demand for diag-nostic and intervention services. Rather, at thisstage, we propose that these may be used asreferral tools that is, where concern has alreadybeen expressed, and/or the individual is expe-riencing difficulties, as a guide for primary carehealth or social care professionals (includingGPs, family physicians, social workers, nurseryworkers and Health Visitors) to help them todecide whether a referral to a specialist servicefor ASC is appropriate. The current decision-making process of primary health, social care,and early education practitioners with regardto referral for specialist ASC assessment isunknown and should be investigated. In real-ity, decision making is most likely dependenton knowledge, training, and prior experiencein relation to ASC, meaning that many individ-uals who warrant a referral may not be re-ferred. The predictive value of measures suchas these can be increased by applying them incontexts where concerns have already beenraised.61 It must be cautioned that the psycho-metric properties obtained in these samples
may not be generalizable. It is not known how
JOURN
210 www.jaacap.org
he samples derived for this study differ notust from referred samples but also from sam-les where a decision is being made abouthether to make a referral. The performance of
ny measure is dependent on the prevalence ofhe disorder being measured in the sample. Theredictive value of the measure will change as
function of the prevalence in the sampleeing assessed, given the known sensitivitynd specificity of the instruments (even if thesere high).62,63 The proportion of cases in each
validation sample ranged from 14% to 46%. Toillustrate an example using the adult AQ data,the proportion of cases of ASC in the validationsample was 35% and the resulting PPV was0.85 (197 true positives, 36 false positives).However, in a true population sample in whichthe prevalence of ASC is approximately 1%,8,9
the PPV would have been 0.09 (given thesensitivity and specificity of 0.88 and 0.91 re-spectively, values that do not vary). Therefore,screening for ASC in a sample enriched byindividuals for whom there are concerns aboutpossible ASC may result in a substantiallyhigher predictive value than if the measure istested in a population sample in which the preva-lence is substantially lower. Similarly, it is inevita-ble that screening the general population for a rarecondition will result with many test false-positiveresults and therefore a low PPV.64
This study represents the first step in the devel-opment of short instruments designed to helphealth care and social care professionals in thereferral pathway for ASC. The short forms aremore suitable for busy health care professionalsthan the long forms when time is limited. Respon-dent burden is also reduced with the short forms.Further work is required to examine how thesetools perform in primary and social care, by track-ing individuals who are referred to specialist diag-nostic services and determining their outcome byindependent expert diagnostic observations. &
Accepted November 14, 2011.
Drs. Allison, Auyeung, and Baron-Cohen are with the Autism ResearchCentre, Cambridge University.
Funding was made possible from grants from the Big Lottery Fund, theMedical Research Council, the Three Guineas Trust, and the NationalInstitute for Health Research Collaboration for Leadership in AppliedHealth Research and Care.
We are grateful to all of the individuals and families that completed thequestionnaires. This study was conducted in association with NationalInstitute of Health Research (NIHR) Collaborations for Leadership inApplied Health Research and Care (CLAHRC) for Cambridgeshire andPeterborough. We thank Carol Brayne, Fiona Matthews, and Liliana
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
SHORT VERSIONS OF THE AQ AND Q-CHAT
Ruta of the University of Cambridge; Tony Charman and GregPasco of the Institute of Education; Sally Wheelwright of theUniversity of Southampton and Rosa Hoekstra of the Open Univer-sity, for valuable discussions. We thank the anonymous reviewersfor their helpful comments.
Disclosure: Drs. Allison, Auyeung, and Baron-Cohen report no biomed-ical financial interests or potential conflicts of interest.
J Autism Dev Disord. 2011. Available at: http://www.springerlink.com/content/mk01522176825656/. Accessed August 23, 2011.
2
2
2
2
JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
VOLUME 51 NUMBER 2 FEBRUARY 2012
Correspondence to Carrie Allison, Ph.D., Autism Research Centre,Department of Psychiatry, Cambridge University Douglas House,18B Trumpington Road, Cambridge, CB2 8AH, UK; e-mail [email protected]
REFERENCES1. World Health Organisation. The ICD-10 Classification of Mental
and Behavioural Disorders: Diagnostic Criteria for Research.Geneva: World Health Organisation; 1993.
2. American Psychiatric Association. Diagnostic and Statistical Man-ual of Mental Disorders (DSM-IV). 4th ed. Washington, DC:American Psychiatric Association; 1994.
3. Bailey A, Le Couteur A, Gottesman I, et al. Autism as a stronglygenetic disorder: evidence from a British twin study. PsycholMed. 1995;25:63-77.
4. Bolton P, Macdonald H, Pickles A, et al. A case-control familyhistory study of autism. J Child Psychol Psychiatry. 1994;35:877-900.
5. Chakrabarti B, Dudbridge F, Kent L, et al. Genes related to sexsteroids, neural growth, and social-emotional behavior are asso-ciated with autistic traits, empathy, and Asperger syndrome.Autism Res. 2009;2:157-177.
6. Geschwind DH. Autism: many genes, common pathways? Cell.2008;135:391-395.
7. Courchesne E, Carper R, Akshoomoff N. Evidence of brainovergrowth in the first year of life in autism. JAMA. 2003;290:337-344.
8. Baird G, Simonoff E, Pickles A, et al. Prevalence of disorders of theautism spectrum in a population cohort of children in SouthThames: the Special Needs and Autism Project (SNAP). Lancet.2006;368:210-215.
9. Baron-Cohen S, Scott FJ, Allison C, et al. Prevalence of autism-spectrum conditions: UK school-based population study. Br JPsychiatry. 2009;194:500-509.
10. Brugha T, McManus S, Meltzer H, et al. Autism spectrum disor-ders in adults living in households throughout England: reportfrom the Adult Psychiatric Morbidity Survey 2007. Leeds, UK:Health and Social Care Information Centre, Social Care Statistics;2009.
11. Baron-Cohen S, Wheelwright S, Skinner R, Martin J, Clubley E.The autism-spectrum quotient (AQ): evidence from Aspergersyndrome/high-functioning autism, males and females, scientistsand mathematicians. J Autism Dev Disord. 2001;31:5-17.
12. Wing L. The continuum of autistic characteristics. In: Schopler E,Mesibov G, eds. Diagnosis and Assessment in Autism. New York:Plenum; 1988.
13. Charman T. What does the term ‘working diagnosis’ mean? JAutism Dev Disord. 2005;35:539-540.
14. Wing L. The Autistic Spectrum. Oxford: Pergamon; 1997.15. Howlin P, Asgharian A. The diagnosis of autism and Asperger
syndrome: findings from a survey of 770 families. Dev Med ChildNeurol. 1999;41:834-839.
16. Knapp M, Romeo R, Beecham J. Economic cost of autism in theUK. Autism. 2009;13:317-336.
17. Ganz M. The costs of autism. In: Moldin S, Rubenstein J, eds.Understanding Autism: From Basic Neuroscience to Treatment.Boca Raton, FL: Taylor & Francis; 2006.
18. National Audit Office. Supporting People with Autism ThroughAdulthood. London: Stationery Office; 2009.
19. Information Centre for Health and Social Care. Average PracticeList Sizes in the UK NHS Technical Steering Committee; 2008.
20. Dereu M, Raymaekers R, Warreyn P, Schietecatte I, MeirsschautM, Roeyers H. Can child care workers contribute to the earlydetection of autism spectrum disorders? A comparison betweenscreening instruments with child care workers versus parentsas informants. [published online ahead of print June 21 2011].
1. Glascoe FP. Screening for developmental and behavioral prob-lems. Ment Retard Dev Disabil Res Rev. 2005;11:173-179.
2. Baron-Cohen S, Allen J, Gillberg C. Can autism be detected at 18months? The needle, the haystack, and the CHAT. Br J Psychiatry.1992;161:839-843.
3. Baron-Cohen S, Cox A, Baird G, et al. Psychological markers in thedetection of autism in infancy in a large population. Br J Psychi-atry. 1996;168:158-163.
4. Baird G, Charman T, Baron-Cohen S, et al. A screening instrumentfor autism at 18 months of age: a 6-year follow-up study. J AmAcad Child Adolesc Psychiatry. 2000;39:694-702.
25. Berument SK, Rutter M, Lord C, Pickles A, Bailey A. Autismscreening questionnaire: diagnostic validity. Br J Psychiatry.1999;175:444-451.
26. Allen CW, Silove N, Williams K, Hutchins P. Validity of the socialcommunication questionnaire in assessing risk of autism inpreschool children with developmental problems. J Autism DevDisord. 2007;37:1272-1278.
27. Corsello C, Hus V, Pickles A, et al. Between a ROC and a hardplace: decision making and making decisions about using theSCQ. J Child Psychol Psychiatry. 2007;48:932-940.
28. Robins DL, Fein D, Barton ML, Green JA. The Modified Checklistfor Autism in Toddlers: an initial study investigating the earlydetection of autism and pervasive developmental disorders. JAutism Dev Disord. 2001;31:131-144.
29. Swinkels SH, Dietz C, van Daalen E, Kerkhof IH, van Engeland H,Buitelaar JK. Screening for autistic spectrum in children aged 14to 15 months. I: the development of the Early Screening ofAutistic Traits Questionnaire (ESAT). J Autism Dev Disord.2006;36:723-732.
30. Dietz C, Swinkels S, van Daalen E, van Engeland H, Buitelaar JK.Screening for autistic spectrum disorder in children aged 14-15months. II: population screening with the Early Screening ofAutistic Traits Questionnaire (ESAT). Design and general find-ings. J Autism Dev Disord. 2006;36:713-722.
31. Constantino JN, Davis SA, Todd RD, et al. Validation of a briefquantitative measure of autistic traits: comparison of the so-cial responsiveness scale with the autism diagnostic interview-revised. J Autism Dev Disord. 2003;33:427-433.
32. Reznick JS, Baranek GT, Reavis S, Watson LR, Crais ER. Aparent-report instrument for identifying one-year-olds at risk foran eventual diagnosis of autism: the first year inventory. J AutismDev Disord. 2007;37:1691-1710.
33. Watson LR, Baranek GT, Crais ER, Steven Reznick J, Dykstra J,Perryman T. The first year inventory: retrospective parent re-sponses to a questionnaire designed to identify one-year-olds atrisk for autism. J Autism Dev Disord. 2007;37:49-61.
34. Barnard J, Harvey V, Prior A, Potter D. Ignored or Ineligible? TheReality for Adults with Autistic Spectrum Disorders. London:National Autistic Society; 2001.
35. Woodbury-Smith MR, Robinson J, Wheelwright S, Baron-CohenS. Screening adults for Asperger syndrome using the AQ: apreliminary study of its diagnostic validity in clinical practice. JAutism Dev Disord. 2005;35:331-335.
36. Auyeung B, Baron-Cohen S, Wheelwright S, Allison C. TheAutism Spectrum Quotient: Children’s Version (AQ-Child). JAutism Dev Disord. 2008;38:1230-1240.
tient—Revised (SQ-R) and Empathy Quotient (EQ). Brain Res.2006;1079:47-56.
39. Hoekstra RA, Bartels M, Cath DC, Boomsma DI. Factor structure,reliability and criterion validity of the Autism-Spectrum Quotient(AQ): a study in Dutch population and patient groups. J AutismDev Disord. 2008;38:1555-1566.
40. Wakabayashi A, Baron-Cohen S, Wheelwright S, Tojo Y. TheAutism-Spectrum Quotient (AQ) in Japan: a cross-cultural com-parison. J Autism Dev Disord. 2006;36:263-270.
41. Ruta L, Mazzone D, Mazzone L, Wheelwright S, Baron-Cohen S.The Autism-Spectrum Quotient—Italian Version: a cross-culturalconfirmation of the broader autism phenotype [published onlineahead of print May 28 2011]. J Autism Dev Disord. 2011. Avail-able at: http://www.springerlink.com/content/n71j631157373513/.Accessed August 23, 2011.
42. Hoekstra RA, Bartels M, Verweij CJH, Boomsma DI. Heritabilityof autistic traits in the general population. Arch Pediatr AdolescMed. 2007;161:372-377.
43. Takagishi H, Takahashi T, Yamagishi T, et al. Salivary testoster-one levels and autism-spectrum quotient in adults. Neuro Endo-crinol Lett. 2011;31:837-841.
44. von dem Hagen EA, Nummenmaa L, Yu R, Engell AD, EwbankMP, Calder AJ. Autism spectrum traits in the typical populationpredict structure and function in the posterior superior temporalsulcus. Cereb Cortex. 2011;21:493-500.
45. Gomot M, Belmonte MK, Bullmore ET, Bernard FA, Baron-CohenS. Brain hyper-reactivity to auditory novel targets in childrenwith high-functioning autism. Brain. 2008;131:2479-2488.
46. Lombardo MV, Barnes JL, Wheelwright SJ, Baron-Cohen S. Self-referential cognition and empathy in autism. PLoS ONE. 2007;2:e883.
47. Auyeung B, Baron-Cohen S, Ashwin E, Knickmeyer R, Taylor K,Hackett G. Fetal testosterone and autistic traits. Br J Psychol.2009;100:1-22.
48. Allison C, Baron-Cohen S, Wheelwright S, et al. The Q-CHAT(Quantitative CHecklist for Autism in Toddlers): a normallydistributed quantitative measure of autistic traits at 18-24 monthsof age: preliminary report. J Autism Dev Disord. 2008;38:1414-1425.
49. Auyeung B, Taylor K, Hackett G, Baron-Cohen S. Foetal testos-terone and autistic traits in 18 to 24-month-old children. MolAutism. 2010;1:11.
50. Elsabbagh M, Mercure E, Hudry K, et al. The utility of ERPmeasures as putative intermediate phenotypes in infancy. Paper
JOURN
212 www.jaacap.org
presented at the International Meeting for Autism Research(IMFAR) 2010; Philadelphia.
51. Stones R, Janvier N, Robbins K. 2006/7 UK General PracticeWorkload Survey. Leeds, UK: The Information Centre; 2007.
52. Gillis JM, Callahan EH, Romanczyk RG. Assessment of socialbehavior in children with autism: the development of the Behav-ioral Assessment of Social Interactions in Young Children. ResAutism Spectrum Disord. 2011;5:351-360.
53. Anastasi A. Psychological Testing. 6th ed. New York: Macmillan;1990.
54. Möricke E, Swinkels S, Beuker K, Buitelaar J. Predictive value ofsubclinical autistic traits at age 14–15 months for behavioral andcognitive problems at age 3–5 years. Eur Child Adolesc Psychia-try. 2010;19:659-668.
55. Kleinman JM, Ventola PE, Pandey J, et al. Diagnostic stability invery young children with autism spectrum disorders. J AutismDev Disord. 2008;38:606-615.
56. Landa RJ. Diagnosis of autism spectrum disorders in the first 3years of life. Nat Clin Pract Neurol. 2008;4:138-147.
57. Kankaras M, Vermunt JK, Moors GBD. Measurement equivalenceof ordinal items: a comparison of factor analytic, item responsetheory, and latent class approaches. Sociol Methods Res. 2011;40:279-310.
58. Daniels AM, Rosenberg RE, Anderson C, Law JK, Marvin AR,Law PA. Verification of parent-report of child autism spectrumdisorder diagnosis to a Web-based autism registry [publishedonline ahead of print April 6 2011]. J Autism Dev Disord. 2011.Available at: http://www.springerlink.com/content/r3102357gg7rt30p/. Accessed April 23, 2011.
59. Lee H, Marvin AR, Watson T, et al. Accuracy of phenotyping ofautistic children based on Internet implemented parent report.Am J Med Genet B Neuropsychiatr Genet. 2010;153B:1119-1126.
60. Gosling SD, Vazire S, Srivastava S, John OP. Should we trustWeb-based studies? A comparative analysis of six preconceptionsabout Internet questionnaires. Am Psychologist. 2004;59:93-104.
61. Williams J, Scott F, Allison C, Bolton P, Baron- Cohen S, Brayne C.The CAST (Childhood Asperger Syndrome Test): test accuracy.Autism. 2005:45-68.
62. Clark A, Harrington R. On diagnosing rare disorders rarely:appropriate use of screening instruments. J Child Psychol Psychi-atry. 1999;40:287-290.
63. O’Toole BI. Screening for low prevalence disorders. Aust N Z JPsychiatry. 2000;34(Suppl):S39-S46.
64. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BrMed J. 1994;309:102.
AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY
