Injury Prevention 1996; 2: 221-227 METHODOLOGIC ISSUES Evaluation of a systematic approach for identifying injury scenarios Katherine Kaufer Christoffel, Joseph L Schofer, and the Kids'n'Cars Team* Abstract Objective-To assess the effectiveness of a new multidisciplinary method for recon- structing the causal sequences that lead to child pedestrian injuries. Setting-Subjects were 5-12 year old residents of Chicago, Illinois, USA, pres- enting for care due to pedestrian injury at one pediatric trauma center. Methods-The interactions of medical, child, psychosocial, and traffic factors contributing to the injury were analysed. For 142 cases, information about the vic- tim, his/her family, the injury site, and the activities just before the injury, was used in a structured manner by a multi- disciplinary team to produce injury scenarios. Each scenario comprised a list of contributing factors, an estimate of the importance of each, and a narrative des- cription of the causal sequence leading to the injury event. Face validity was assessed by two outside teams that per- formed a structured review of a subsam- ple of cases (n = 11). Reliability was evaluated by comparison of the results of parallel teams assessing the same cases (n = 14). Process consistency and bias were assessed by analysis of the correla- tions of factor-importance rating pat- terns between members and over time. Results-The outside team's agreement scores were based on a 1-5 Likert scale; these showed a mean of 3X6 and median of 4 0. Parallel teams consistently showed agreement greater than 85% on global attributes of cases. Intraclass correlation coefficient scores showed fair or better agreement for all classes of contributors, and excellent agreement for more than one third. Rating pattern analyses showed strong agreement by team members. Agreement did not increase over the period of the study. Conclusions-This causal sequence re- construction method has acceptable face validity, reliability, and internal con- sistency. Although labor intensive and thus costly, it can produce unique, rich information for understanding injury causation and for guiding the search for promising interventions. (Injury Prevention 1996; 2: 221-227) Keywords: pedestrian, causal sequence, evaluation, safety. The use of multidisciplinary teams to inves- tigate motor vehicle crashes is not new. Some of the earliest work was done by Baker and others at the Traffic Institute of Northwestern University in the late 1950s.I Their team included an engineer, psychologist, and physician, who performed 'hot pursuit' inves- tigations of urban auto crashes. The team received notification of events from police and traveled to the scene to examine the site and vehicles, and to interview drivers, victims, and witnesses. Some interviews were conducted at the scene and others in the study office within a few days of the event. Only a small number of crashes was investigated, and most of these were vehicle-vehicle events. A brief description of the event was produced based on group consensus. These researchers were not able to reach consensus on causal factors and their importance. Instead, each disciplinary specialist prepared a separate, descriptive report identifying factors but not assessing their importance. Snyder and Knoblauch also conducted a 12 city investigation of 2157 pedestrian injury events involving persons of all ages2; 40%O of the victims were under 9 years of age. Each case was investigated by one or two specialists trained to use a consistent data collection protocol created by a multidisciplinary team. Victims, drivers, and witnesses were inter- viewed to develop a description of the 'behavioural sequence' leading to the injury. About 16% of the cases were investigated at the scene within two hours of the event and nearly 55 % were investigated one or more weeks later. Data were not collected on the detailed charac- teristics of injured pedestrians (beyond age and gender), although driver characteristics were analyzed. No formal process was reported for integrating information into injury scenarios, though cases were classified into 26 scenario categories. The work described here builds on these earlier efforts. When we began to study child pedestrian injury, we were impressed by the complexity of the causal processes and frust- rated by the absence of an appropriate method to weave them into comprehensive scenarios. We sought to identify the psychosocial and physical contributors to the event, to assess their relative importance, and to understand their interactions. This demanded not only a systematic approach to data collection, but also an assessment of these data. We designed a team process that enabled us to integrate several sources of data: descriptions of the event, the scene, and the child's physical, Children's Memorial Hospital, Chicago, and Northwestern University Medical School, Chicago, Illinois, USA K Kaufer Christoffel Department of Civil Engineering and The Transportation Center, Northwestern University, Evanston, Illinois, USA JL Schofer *The Kids'n'Cars Team members are listed at the end of the paper Correspondence to: Dr K Kaufer Christoffel, 2300 Children's Plaza, 46, Chicago, IL 60614, USA. 221 on May 7, 2020 by guest. Protected by copyright. http://injuryprevention.bmj.com/ Inj Prev: first published as 10.1136/ip.2.3.221 on 1 September 1996. Downloaded from
Transcript
Injury Prevention 1996; 2: 221-227
METHODOLOGIC ISSUES
Evaluation of a systematic approach foridentifying injury scenarios
Katherine Kaufer Christoffel, Joseph L Schofer, and the Kids'n'Cars Team*
AbstractObjective-To assess the effectiveness ofanew multidisciplinary method for recon-structing the causal sequences that lead tochild pedestrian injuries.Setting-Subjects were 5-12 year oldresidents of Chicago, Illinois, USA, pres-enting for care due to pedestrian injury atone pediatric trauma center.
Methods-The interactions of medical,child, psychosocial, and traffic factorscontributing to the injury were analysed.For 142 cases, information about the vic-tim, his/her family, the injury site, andthe activities just before the injury, wasused in a structured manner by a multi-disciplinary team to produce injuryscenarios. Each scenario comprised a listofcontributing factors, an estimate oftheimportance of each, and a narrative des-cription of the causal sequence leading tothe injury event. Face validity wasassessed by two outside teams that per-formed a structured review of a subsam-ple of cases (n = 11). Reliability wasevaluated by comparison of the results ofparallel teams assessing the same cases(n = 14). Process consistency and biaswere assessed by analysis of the correla-tions of factor-importance rating pat-terns between members and over time.
Results-The outside team's agreementscores were based on a 1-5 Likert scale;these showed a mean of 3X6 and median of4 0. Parallel teams consistently showedagreement greater than 85% on globalattributes of cases. Intraclass correlationcoefficient scores showed fair or betteragreement for all classes of contributors,and excellent agreement for more thanone third. Rating pattern analyses showedstrong agreement by team members.Agreement did not increase over theperiod of the study.Conclusions-This causal sequence re-construction method has acceptable facevalidity, reliability, and internal con-sistency. Although labor intensive andthus costly, it can produce unique, richinformation for understanding injurycausation and for guiding the search forpromising interventions.(Injury Prevention 1996; 2: 221-227)
The use of multidisciplinary teams to inves-tigate motor vehicle crashes is not new. Some ofthe earliest work was done by Baker and othersat the Traffic Institute of NorthwesternUniversity in the late 1950s.I Their teamincluded an engineer, psychologist, andphysician, who performed 'hot pursuit' inves-tigations of urban auto crashes. The teamreceived notification of events from police andtraveled to the scene to examine the site andvehicles, and to interview drivers, victims, andwitnesses. Some interviews were conducted atthe scene and others in the study office within afew days of the event. Only a small number ofcrashes was investigated, and most of thesewere vehicle-vehicle events. A brief descriptionof the event was produced based on groupconsensus. These researchers were not able toreach consensus on causal factors and theirimportance. Instead, each disciplinaryspecialist prepared a separate, descriptivereport identifying factors but not assessingtheir importance.
Snyder and Knoblauch also conducted a 12city investigation of 2157 pedestrian injuryevents involving persons of all ages2; 40%O ofthevictims were under 9 years of age. Each casewas investigated by one or two specialiststrained to use a consistent data collectionprotocol created by a multidisciplinary team.Victims, drivers, and witnesses were inter-viewed to develop a description of the'behavioural sequence' leading to the injury.About 16% ofthe cases were investigated at thescene within two hours of the event and nearly55% were investigated one or more weeks later.Data were not collected on the detailed charac-teristics of injured pedestrians (beyond age andgender), although driver characteristics wereanalyzed. No formal process was reported forintegrating information into injury scenarios,though cases were classified into 26 scenariocategories.The work described here builds on these
earlier efforts. When we began to study childpedestrian injury, we were impressed by thecomplexity of the causal processes and frust-rated by the absence of an appropriate methodto weave them into comprehensive scenarios.We sought to identify the psychosocial andphysical contributors to the event, to assesstheir relative importance, and to understandtheir interactions. This demanded not only asystematic approach to data collection, but alsoan assessment of these data. We designed ateam process that enabled us to integrateseveral sources of data: descriptions of theevent, the scene, and the child's physical,
social, and psychological characteristics.3 Thisprocess, which we refer to as causal sequencereconstruction (CSR), differs from previousefforts principally because of this multidiscip-linary integration and interpretation phase.This report describes an evaluation of the CSRmethod.
MethodsThe methods of the Kids' n' Cars study havebeen described,34 and are summarized brieflyhere. Pedestrian injury victims aged 5-12 yearswere identified at Children's Memorial Hos-pital, a pediatric trauma center on the northside of Chicago. Data on each child and injuryevent were gathered from parents, victims(where feasible), teachers, on-scene investiga-tions, and police accident reports, which oftenincluded statements from drivers, witnesses,and victims.The injury site was visited, measured,
videotaped, and diagrammed, and relevanttraffic measures were collected at the same timeof day and day of week as the event. Each childwas characterized by his or her physical att-ributes and limitations, medical history,injuries, and psychological profile measuredwith standardized test scores. The Achenbachchild behavior checklist was used to evaluatebehavior,5-7 and the Alpern-Boll developmen-tal profile to assess developmental level.8 Thechild's family environment was described byscores on the Moos family environmentscale,9 10 the Coddington life stress scale, 11 12 theThurstone temperament schedule (formothers),'3 and the multidimensional scales ofperceived social support (also about themother).'4 To minimize the effect of the injuryon parental reports of behavior, all measureswere obtained as soon as possible after injury.
TEAM PROCESSOur multidisciplinary team included expertsfrom traffic engineering, medicine, clinicalpsychology, and social work. Each case wasinvestigated by each ofthese disciplines follow-ing an established and consistent protocol. Thefindings of each expert were integrated at agroup meeting during which each field inves-tigator presented his or her findings in objec-tive terms, for example, interview content, testscores, site measurements, and videotape. Thiswas followed by an open discussion in which adescription of the causal sequence was for-mulated and contributing factors identifiedfrom a list of 75 factors in 11 realms, developedinteractively early in our study (see Appendixfor factors and table 2 for realms). The impor-tance of these factors in contributing to theinjury event was determined by an open ratingprocess in which each team member assignedweights on a scale of 0 (not a contributor) to 5(decisive factor). This process fostered debateand discussion, leading to selection of consen-sus weights, the modal value of the groupratings. A qualitative (yes/no) judgment wasmade as to whether the child moved quickly orappeared suddenly (sudden appearance pedest-
rian injury, SAPI).4 Finally, a narrative des-cription of each case was prepared by thediscussion leader and ratified by the team,which then voted its confidence in the resultingnarrative. The mean and mode confidencescore was 4 on a scale of 1-5. Meetings took35-75 minutes; cases were reassessed if newinformation became available, for example, if ateacher's child behavior checklist arrivedbelatedly.
EVALUATIONEfforts to assess the soundness of the CSRprocess and its product scenarios includedinternal and external reliability, face validity,and analysis of internal consistency and biases(figure).
Internal reliability of the process wasassessed by applying it through two parallelteams formed by dividing the Kids'n'Cars(henceforth 'Kids') team into two - Kids Aand Kids B- with each discipline representedon each team. The A and B teams were createdby random assignment of each discipline'smembers (usually two) and of team membersnot tied to a discipline (for example, projectcoordinator), with new assignments after everytwo split cases. Six cases were analyzed in thisway.The second component of the evaluation was
an assessment of external reliability for eightcases by an outside consultant team (team 1): atwo member team from another institution thatreviewed the scenarios produced by the Kidsteam. Team 1 was provided with the sameinformation as the Kids team, that is, 1-2 pagesummaries ofthe medical records review, socialwork interview, psychological scale scores, andtraffic site investigations, along with avideotape of the injury site. This allowed us tocompare the Kids team results with team l'sresults as if they were split teams.The third component was an assessment of
face validity by team 1 and another, team 2.Team 2 was also a two person team, from athird institution. It received the same kind ofinformation as team 1 received for assessmentof external reliability. These teams were alsogiven summaries of the Kids team results foreach case, including the factors selected asoperative, the importance ratings, the SAPIand confidence judgments, and the narrativedescriptions. For logistical reasons, one con-sulting team reviewed eight cases and the otheronly five; two cases were reviewed by bothteams. The two outside teams were asked toreview the Kids team's CSR process for thecases they received. Based on those reviews,they were asked to assess, for each of the fourdata types used - medical, social work,psychological, and traffic - data usefulness,data comprehensiveness, the Kids team'sconfidence in the data, the contributors theKids team identified and the weights it assignedto these. The consultants were also asked to ratethe soundness of the Kids team's narrativedescription and the confidence ratings assignedto these. Teams 1 and 2 reported their degree ofagreement with the Kids team results, using
222 on M
ay 7, 2020 by guest. Protected by copyright.
http://injuryprevention.bmj.com
/Inj P
rev: first published as 10.1136/ip.2.3.221 on 1 Septem
five point Likert scales for all statements asser-ting a positive view of the items beingevaluated.The fourth evaluation component assessed
internal consistency and bias by examining thecorrelations among Kids team members' ratingpatterns for three realms- road hazards, childdistraction, and supervision. The aim was toevaluate (a) whether team leaders exertedundue influence on the rating patterns, and (b)whether the correlations increased over thefour year period of the CSR meetings (whichwould suggest a group dynamic affectingscenario content).
DATA ANALYSISOutside team evaluations of the Kids teamassessments were described using mean andmodal Likert scale scores for each component.The two types of two team data (Kids A v KidsB and Kids v team 1) were analyzed in twoways. First, the per cent agreement was cal-culated on several types of team ratings:identification of sudden appearance events,narrative confidence score, and contributorselection and weights. These analyses used the11 contributor realms listed in table 2. For eachrealm, the maxium score for any selected con-tributor for each case was the indicator of therealm's contribution to the causal sequence.The agreement measures used were (a) per centagreement on selected (and unselected) indi-vidual contributors; (b) per cent agreement onselected, that is, realms with any contributoridentified and unselected; (c) per cent agree-ment on maximum contributor weights inselected realms; and (d) disagreement on max-imum contributor weights, as indicated by themean square difference in weights for eachrealm.
For the Kids A v Kids B and Kids v team 1analyses, we also calculated three variancecomponents: team, case, and residual; this wasbased on a random effects analysis of variance.For each category, the intraclass correlationcoefficient (ICC), equal to the case variancecomponent divided by the sum of the threevariance components, was calculated. (Thisstatistic is equivalent to a weighted K analysiswith weights equal to the square of thedifference between scores.) Key statistics in-clude the size of the ICC and the size of therandom variance component. We evaluatedICCs as follows: > 0-80, excellent agreement;> 0 60, good agreement; > 040, moderateagreement; > 0 20, fair agreement; and > 0-20,poor agreement.'5
Internal consistency and bias were analyzedby obtaining correlations for every combina-tion oftwo voters for each realm analyzed. Eachvoter was then characterized by the meancorrelation with other voters. In addition, caseswere divided into four equal sized groups infour successive time periods. A general linearmodels procedure was used to generate anintraclass correlation coefficient and meansquare error for votes in each time period.These results were examined for evidence of atemporal learning effect, as evidenced by adecreasing mean square error and increasingICC.
ResultsFACE VALIDITYAs described under the methods section facevalidity was assessed using a five point Likertscale (1 = strongly disagree to 5 = stronglyagree). The mean scores for 24 items, for eachoutside team, and an overall mean, across cases
223 on M
ay 7, 2020 by guest. Protected by copyright.
http://injuryprevention.bmj.com
/Inj P
rev: first published as 10.1136/ip.2.3.221 on 1 Septem
and teams were > 3 5 for 21 items and > 4 for11 items. For the cases staffed by both outsideteams, the mean was 3 6, and the median 4 0.
TWO TEAM DATATable 1 shows that there was relatively strongagreement between parallel teams' results.Agreement was strongest for the six Kids A vKids B comparison and was not substantiallyweaker for the eight Kids v team 1 cases.Table 2 shows the variance analyses for the
Kids A v Kids B and Kids v team 1 com-parisons. The ICC values were above 0-80 forsix of 13 Kids v Kids comparisons. One com-parison showed an ICC of 0; this was due tolack of subject variability: the realm wasselected in only two cases and scored as low inimportance in both. ICC values were alsoabove 0-80 for five of the Kids v team 1comparisons and were 0-20 or better for allremaining comparisons. This table shows thatboth kinds of split teams functioned equallywell.
In the comparisons in both tables 1 and 2, theteams agreed best on the most global measures(for example, sudden appearance), and least onthe most detailed assessments, that it, max-imum factor weightings within realms.
RATING PATTERNSBecause of the large number of team membersand contributing factors rated, the inter-rateranalyses generated numerous correlation
Table 1 Internal and external reliability agreement between two teams
Kids A v Kids B Kids v team I(n=6) (n=8)
Confidence (weights within 1) (%) 100 88Agreement on sudden appearance (°) 100 88Mean No of selected realms* (range) 5-5 (4-8) 4-8 (3-10)Agreement on selected realms*: % where both teams 72 (50-83) 72 (56-100)weighted 2 or greater (range)Agreement on weights, selected realms*: ° factor 68 (50-83) 61 (13-88)weights within one (range)Agreement on weights, all realmst: 0% factor weights 86 (73-91) 73 (36-91)within one (range)For selected realms*: mean square difference in factor 4-0 (1 2-5-3) 3-3 (1 1-6-3)importance rating (range)For all realmst: mean square difference in factor 0 9 (0-0-1 8) 1 0 (0-2-2 3)importance rating (range)
*Selected realms (number varies): realms for which at least one factor was rated > 2 by either team,that is, identified as a contributor.tAll 11 realms: selected and unselected realms.
Table 2 Contributingfactor ratings: variance componentsfor two team assessmentsKids A v Kids B Kids v team I
Study staffProject coordinator Traffic research assistant0 90 (0 83-0 94) 0 88 (0 79-0 96)
values. Table 3 shows selected values thatexemplify these analyses. For each teammember listed, the mean and range of thecorrelations with all other team members isshown only for the road characteristics factorrealm. Correlations were high and did notdiffer much among team members nor werethey higher for the senior team members thanfor others. The other results analyzed weresimilar.The ICC and mean square errors for votes on
road hazards in successive time periods variedfrom 0 92 in the first period to 0-76 in the third(mean square error varied from 0 303 to 0 706).The differences were not statisticallysignificant. Thus the pattern observed does notsupport an inference of tightened teamdynamics (and so possibly bias) over time.
DiscussionPreventing injury requires an understanding ofthe factors and events leading to the tissuedamaging energy exchange. Only with thisunderstanding is it possible to identify effectivemeans to interrupt the process. Examples ofthis principle include the development ofmotor vehicle occupant restraint systems tosafely displace crash forces, the identification offour sided fences with self locking gates as ameans to delay pool access long enough toprevent toddler drownings, and the develop-ment of 'Willy Whistle', an approach toteaching children to cross streets safely.'6One approach commonly used to reach thisunderstanding is an examination of thesequence of events to detect patterns thatsuggest causal factors and prevention oppor-tunities. This process generates scenarios thatallow a qualitative understanding ofhow injuryevents happen; it therefore enables investi-gators to set priorities for data collection, dataanalysis, and the design and evaluation ofpreventive interventions. The scenarios arealso useful for communicating with otherinvestigators and injury control practitioners.Yet scenario construction has been a hiddenstep in injury prevention work.
Despite widespread use of this general app-roach, consistent, verifiable procedures arerarely used and the process may therefore beseen as unscientific. The utility of prior multi-disciplinary efforts is based on a presumptionof reliability and validity rather than any sortsof quantitative evaluation. Current standardsrequire rigorous scrutiny of complex, expen-sive, and/or novel methods. Evaluation is par-
on May 7, 2020 by guest. P
rotected by copyright.http://injuryprevention.bm
j.com/
Inj Prev: first published as 10.1136/ip.2.3.221 on 1 S
ticularly needed for the types of process wedescribe because of the qualitative nature of theresulting scenarios, which is the source of boththeir value and their uncertainty. If any CSRmethod does not produce consistent, reliable,and unbiased results, it is merely an anecdotalprocess without credibility. Ifthis CSR processcan be structured with rigor and demonstratedto be valid, the importance of scenario con-structions could be more fully acknowledged,its products more credible, and its results morewidely shared.The CSR method described here responds to
a problem faced by all who study rare, randomevents: almost never do investigators witnessthe event or the steps leading to it. Instead,researchers must rely on witness reports thatare deficient in many ways. Nor do trainedprofessionals (for example, police officers)often witness these events directly. Our workdid not involve immediate visits to the injurysite because of the costs and risks involved.Instead, we reconstructed each event usinginformation from several different sources.These include parental descriptions that oftenwere secondhand (from victims, friends, andsiblings). When possible, however, we alsointerviewed the injured child. Police reportsimplicitly or explicitly include the driver'sexplanation, and sometimes witness descrip-tions. Some or all of these observers andreporters may distort the facts, unconsciously(for example, to provide a rationale for ahorrible event) or consciously (for example, toavoid or assign blame, cover up a misdeed orrule infraction).4 In such cases it is not possibleto establish a 'gold standard' measurement ofwhat really happened.The CSR process described here integrates
information from several sources through alogical, open replicable procedure that pro-duces 'most likely' process description. At thecore of this method is the team meeting - aninteractive process for challenging and crosschecking reports from various data sources toreduce bias and increase objectivity.The nature ofthis approach demands evalua-
tion to assess (1) internal and externalreliability, accomplished here by two teamanalyses; (2) face validity, here done throughevaluation by consultant teams; and (3) internalconsistency and bias, pursued here through anassessment of the contribution of groupdynamics to process outcomes. The number ofcases studied by consultant teams includedabout 10% of the sample of cases for whichscenarios were constructed.
INTERNAL AND EXTERNAL RELIABILITY (TWOTEAM ANALYSES)In view ofthe structural uncertainties built intothe CSR method, the level of agreement shownin the two team analyses is remarkably high,with substantial difficulties only on a fewspecific contributor weightings. Reliability wasextremely good for the least detailed analyses(for example, confidence in narrative and sud-den appearance), and excellent on more thanone third of the contributing realms.Agreement was moderate to good on the role
ofthe driver, although we expected it to be poorbecause ofthe uncertainty inherent in this area.Driver behavior is typically recorded in policereports, with drivers themselves as the infor-mant. Driver bias and social expectations leadto a tendency to blame the child. Our approachwas more comprehensive, combining reportsfrom police, the child, parents, and otherwitnesses.Agreement was also surprisingly good for
newly identified contributors, such as suddenappearance. For these, such agreement mighthave resulted from shared experience, whichthe Kids split team members had in greaterabundance than the team 1 members. Excellentagreement across all teams on these is a reassur-ing indication of the utility of these newlyidentified contributors.
Reliability was weaker for several cont-ributor realms. In the case of traffic cont-ributors, which seem objective, this result iscounterintuitive. However, as the injury eventwas not witnessed, the contribution of partic-ular traffic configurations is somewhatspeculative. Transient conditions are partic-ularly susceptible to variable reporting fromdifferent sources. Thus, our modest reliabilityresults on traffic contributors are not surpris-ing. What is most important is that agreementon the injury narrative was not adverselyaffected by modest agreement concerningtraffic contributors.For a few contributor realms, agreement
levels were notably discrepant between the twosets of two team comparisons (that is, Kids A vKids B, Kids v team 1). Lack of agreement on'medical' contributors could easily have beencaused by their rarity. Further, the role ofmedical problems was not always clear, and theteam 1 physician may not have seen this quite asclearly as the three medical people on the Kidsteam (two physicians (MDs) and one registeredmurse), who discussed many more cases andbecame used to the fact that medical issues onlyrarely contributed to the causal sequence.The Kids v team 1 comparison for 'child
inner processes' was moderate, but substan-tially lower than the excellent agreement for theKids A v Kids B comparison. This may havereflected some disagreement between the Kidsand team 1 psychologists about the interpreta-tion of subscale scores from the Achenbachchild behaviour checklist. In addition, theprecise connection between child characteris-tics and unobserved behaviors during thecausal sequence is, like the role of traffic,subject to speculation. Again, what is mostimportant is that the existing level of uncer-tainty did not undermine agreement on theinjury narratives.The discrepancy on the view/visibility cont-
ributors, with very good reliability for the Kidsv team 1 comparison but only fair agreementfor the Kids A v Kids B comparison, isperplexing, particularly given the highinterteam consistency on 'sudden appearance'.
FACE VALIDITYThe evaluations indicate that this CSR methodappears reasonably sound to outside reviewers.
225 on M
ay 7, 2020 by guest. Protected by copyright.
http://injuryprevention.bmj.com
/Inj P
rev: first published as 10.1136/ip.2.3.221 on 1 Septem
INTERNAL CONSISTENCY AND BIAS (RATINGPATTERNS)This analysis was conducted for the 120 caseswith complete records of the relevant ratings.Intervoter agreement was high, but not perfect.Clearly, the open voting process did not sup-press all disagreement. We were concerned thatit might have given undue influence to teamleaders but this did not occur. The analyses ofsequential subgroups of cases also provideassurance that team interactions did not lead toincreasingly idiosyncratic results over time.
SIGNIFICANCE OF EVALUATION RESULTSThe specific type of data used differed some-what from previous CSR efforts: our studycollected more psychosocial information thanearlier efforts but did not interview drivers orexamine vehicles. Although driver reports canbe expected to carry substantial biases, theabsence of all but the simplest objective in-formation about drivers from the police reportsrepresents a gap in our data. However, thisCSR method can readily accommodate datafrom various sources.
APPLICATION OF THE METHODOur CSR method produced reliable scenarios,with face validity, that were not shaped byidiosyncratic group dynamics. This methodmay be useful to other investigators studyingother complex types of injury. The resultingscenarios appear to be credible process descrip-tions, and thus components of future injuryresearch. Thus, this method appears to beeffective for understanding complex interac-tions of qualitative contributors. These includedefinitional, measurement, and conceptualiza-tion issues which may be at the core of under-standing injury event causality.For example, while it is intuitive that super-
vision of children is important in, and protec-tive against, pedestrian injury, early in ourwork we found that children were getting hit bycars even when supervised by parents. Furtherdiscussions made it clear that there was no
operational definition of supervision to help usunderstand such cases. Our method led us to afour dimensional definition of supervision:supervised or not, age of the supervisor, dis-tance of the supervisor from the child, andwhether the child is alone or with a group ofpeers.17
Similarly, when we tried to fit our cases intothe accepted pedestrian injury taxonomy,2 wefound ambiguities that led us to separate so-called dart-out events into two distinct butoften associated phenomena: moving quicklyinto traffic (running across the street or chang-ing directions rapidly while in the street); andappearing suddenly, not necessarily due torapid movement, but because of view obstruc-tions from parked cars, cars stopped in traffic,or street furniture.4 This distinction clarifiesthe process of the injury event and may lead toseparate interventions designed to slow chil-dren down and make them more visible.This CSR approach is particularly appropri-
ate where (1) many factors are likely to bringabout injuries; (2) information about the eventscomes from several, disparate, often conflictingsources; (3) distinctly different disciplinaryviewpoints are needed to understand theevents; (4) the research need is more forhypothesis generation and the identification ofcausal paths than for statistical evidence.The method is most helpful for teasing out
causal hypotheses because the synergies ofmultidisciplinary debate often generate newperspectives- different from the beliefs ofanysingle individual or discipline. It is least helpfulfor reviewing routine cases where causality isobvious or where patterns are often repeated.While the process of developing the initialframework of contributing factors was timeconsuming and expensive, it is probably neces-sary when starting a new CSR effort. Theseevaluation results suggest that, once theframework of factors has been established, asmaller, less costly team may reliably apply it tomany cases.
The Kid 'n' Cars Team also includes: Martha Barthel, RN(Children's Memorial Hospital); Mark Donovan, MS (Child-ren's Memorial Hospital), Jeffrey Jenq, MS (NorthwesternUniversity Transportation Center); Cecilia Klinger (Children'sMemorial Hospital); John V Lavigne, PhD (Children'sMemorial Hospital and Northwestern University MedicalSchool); Susan LeBailly, PhD (Children's Memorial Hospital);Patricia McGuire (Children's Memorial Hospital); KristinMalmstrom (Children's Memorial Hospital); Robert R Tanz,MD (Children's Memorial Hospital and Northwestern Univer-sity Medical School); Barbara White, MSW (Children'sMemorial Hospital); and Karen Wills, PhD (Loyola Universityof Chicago and Children's Memorial Hospital).We wish to thank our outside consultants, Lizette Peterson,
PhD and Bernard Ewigman, MD, MSPH at the University ofMissouri in Columbia, and Phyllis Agran, MD, MPH andDiane Winn, RN at the University of California in Irvine. Wealso wish to thank Mark Donovan, MS for statistical assistance,and Jean Gagliardi for secretarial assistance.
Appendix
Possibly contributing factors in child pedestrianinjury (ages 5-12)
MEDICALWeight/height > 90 centileNeuromuscular or orthopedic abnormalityCognitive deficitSeizure disorderIngestion of substance that can alter behaviorClumsyVisual disabilityAuditory disability
SOCIALStress at homeMother: depressed/low supportMother: impulsive/immature/over hurriedMother: antisocial/endorses risk taking
SUPERVISIONNeeded supervision absentNeeded supervision present but inadequateNo supervision neededSupervision present and adequatePossible overcontrolling/overprotective parent, so childmay lack street crossing skills
Rule breaking: child in forbidden location or notsupposed to cross alone
Underestimating risk/unrealistic expectationsMother developmentally handicappedTrait high energy/activity/hyperactive (any or all)Trait impulsive/distractible-inattention/social immaturity
(any or all)Trait conduct disorder/non-compliant/aggressive (any
or all)
226 on M
ay 7, 2020 by guest. Protected by copyright.
http://injuryprevention.bmj.com
/Inj P
rev: first published as 10.1136/ip.2.3.221 on 1 Septem
Trait discrepancy, for example, physical less than socialInternalizing psychiatric disturbanceWide streetNormnally high speed streetComplex street configurationComplex operationsEnvironmental impediment to pedestrian traffic makes
pedestrian unexpectedWet pavementHigh traffic volumeLight traffic volume (deceptively safe)Distracted by events/conditions inside vehicleDistracted by events/conditions outside vehicleDriving too fast for conditionsOther dangerous/reckless driver actionChild exited car, entering traffic inappropriatelyView (by or of child) obstructed by fixed objectsView (by or of child) obstructed by standing or moving
vehiclesView (by or of child) obstructed by legally parked
vehiclesView (by or of child) obstructed by double parked or
illegally parked vehiclesView (by or of child) restricted because of child's heightLow visibility (for driver, child) due to weatherLow visibility (for driver, child) due to lighting (night)Child negotiated part-way across street and was hit insecond half of street (continued across)
Child negotiated part-way across street (and was hit)returning to safety of original side
Child negotiated part-way across street (and was hit)facilitated by encouragement by driver(s)
Child attracted/distracted by people at siteChild attracted/distracted by place or object (not in
street)Child followed peers into street ('trailer')Child followed adult into streetChild playing, game spilled into streetChild attracted by area conductive to playChild escaping a situation by going into street or being
chasedChild distracted by complexity of visual environmentChild distracted/confused by high noise levels at sitePressure specific: 'hurry and cross'Group crossingSpecial eventTime pressuredChild attempted midblock crossing for convenience,
'propinquity' of attractionChild attempted midblock crossing appeared lesscomplex than intersection option
Moved quicklyShoe troubleEntered street to retrieve objectSkateboard or other object used as scooter
Child's experience with this site: unfamiliar, notexperienced
Child's experience with this site: very familiar, carelessIntersection crossing, avoiding more complex siteChild's experience with site of this type: unfamiliar, not
experiencedChild's experience with site of this type: very familiar,
carelessLack of appropriate controlCrosswalk (deceptively safe)
1 Baker JS. Experimental case studies of traffic accidents: ageneral discussion of procedures and conclusions.Selected reports from case studies of traffic accidents.Evanston, IL: Traffic Institute at Northwestern Univer-sity, sponsored by the US Bureau of Public Roads, theNational Institutes of Health, the US Public HealthService, and the Automotive Safety Foundation, 1960.
2 Snyder MD, Knoblauch RL. Pedestrian safety: theidentification of precipitating factors and possiblecountermeasures. Vol I. Prepared by Operations ResearchInc, for the National Highway Traffic Safety Administra-tion, 1971.
3 Christoffel KK, Schofer JL, Lavigne JV, et al. Kids 'n' Cars:an ongoing study of pedestrian injuries. Description andearly findings. Children's Environment Quarterly 1991; 8:41-50.
4 Schofer JL, Christoffel K Kaufer, and the Kids 'n' CarsTeam. Child pedestrian injury taxonomy based onvisibility and action. Accid Anal Prev 1995; 27: 317-33.
5 Achenbach TM. The child behavior profile, boys aged 6-11.J Consult Clin Psychol 1978; 46: 478-88.
6 Achenbach TM, Edelbrook CS, Howell C. Empiricallybased assessment of the behavioral/emotional problem of2- and 3-year-old children. J Abnorm Child Psychol 1987;15: 629-50.
7 Edelbrock C, Achenbach TM. Manual for the teacher'sreportform and teacher version of the child behavior profile.Burlington, VT: University of Vermont, 1985.
8 Alpem G, Boll T. Developmental profile manual. Aspen, CO:Psychological Development Publications, 1972.
9 Moos R, Moos B. A typology of family social environments.Fam Proc 1976; 15: 357-72.
10 Fowler PC. Factor structure of the family environmentscale: effects of social desirability. J Clin Psychol 1982;138: 285-92.
11 Baumrind D. Child care practices anteceding three pattemsof preschool behavior. Gen Psych Monographs 1967; 75:43-88.
12 Baumrind D, Block A. Socialization practices associatedwith dimensions of competence in preschool boys andgirls. Child Dev 1967; 38: 291-327.
14 Zimet GD. The multidimensional scale of perceived socialsupport. J Pers Assess 1988; 52: 30-41.
15 Kramer MS, Feinstein AR. Clinical statistics: the biostatis-tics of concordance. Clin Pharmacol Ther 1981; 29:111-23.
16 Preusser DF, Blomberg RD. Reducing child pedestrianaccidents through public education. J Safety Res 1984;15: 47-56.
17 Wills KE, Christoffel K Kaufer, Lavigne JV, Schofer JL,Donovan M, Malmstrom K, and the Kids 'n' CarsResearch Team. Patterns and correlates of supervision inchild pedestrian injury. J Pediatr Psychol 1996 (in press).
227 on M
ay 7, 2020 by guest. Protected by copyright.
http://injuryprevention.bmj.com
/Inj P
rev: first published as 10.1136/ip.2.3.221 on 1 Septem
