Presentation and production: The role of gesture in spatial communication Elizabeth E. Austin ⇑ , Naomi Sweller Department of Psychology, Macquarie University, Sydney, NSW 2109, Australia article info Article history: Received 27 September 2013 Revised 20 December 2013 Available online 15 February 2014 Keywords: Gesture Spatial Route Directions Recall Encoding abstract During social interaction, verbal language as well as nonverbal behavior is exchanged between speakers and listeners. One social task that often involves nonverbal behavior is the relaying of spatial direction information. The questions addressed in this study were whether presenting gesture during encoding (a) enhanced corresponding spatial task performance and (b) elicited gesture production at recall for adults and children. Children (3–4 years) and adults were presented with verbal route directions through a small-scale spatial array and, depending on the assigned condition (i.e., no gestures, beat gestures, or representational gestures), the accompanying gestures. Children, but not adults, benefited from the presence of gesture during encoding of the spatial route direction task, as measured by recall at test. Results suggest that the presence of gesture during encoding plays an integral part of effectively communicating spatial route direction information, particularly for children. Ó 2014 Elsevier Inc. All rights reserved. Introduction Nonverbal behaviors such as hand gestures are integral components of communication and are often produced when conveying spatial information across all ages and cultures (Alibali, 2005; Driskell & Radtke, 2003; Feyereisen & de Lannoy, 1991; Goldin-Meadow, 2000). Our gestures are thought to directly enhance comprehension, learning, and memory for verbal messages by illustrating concepts, conveying additional information, and providing additional cues (Hostetter, 2011; McNeill, 1992). http://dx.doi.org/10.1016/j.jecp.2013.12.008 0022-0965/Ó 2014 Elsevier Inc. All rights reserved. ⇑ Corresponding author. E-mail address: [email protected](E.E. Austin). Journal of Experimental Child Psychology 122 (2014) 92–103 Contents lists available at ScienceDirect Journal of Experimental Child Psychology journal homepage: www.elsevier.com/locate/jecp
Transcript
Journal of Experimental Child Psychology 122 (2014) 92–103
Contents lists available at ScienceDirect
Journal of Experimental ChildPsychology
journal homepage: www.elsevier .com/locate/ jecp
Presentation and production: The role of gesturein spatial communication
http://dx.doi.org/10.1016/j.jecp.2013.12.0080022-0965/� 2014 Elsevier Inc. All rights reserved.
During social interaction, verbal language as well as nonverbalbehavior is exchanged between speakers and listeners. One socialtask that often involves nonverbal behavior is the relaying ofspatial direction information. The questions addressed in this studywere whether presenting gesture during encoding (a) enhancedcorresponding spatial task performance and (b) elicited gestureproduction at recall for adults and children. Children (3–4 years)and adults were presented with verbal route directions through asmall-scale spatial array and, depending on the assigned condition(i.e., no gestures, beat gestures, or representational gestures), theaccompanying gestures. Children, but not adults, benefited fromthe presence of gesture during encoding of the spatial routedirection task, as measured by recall at test. Results suggest thatthe presence of gesture during encoding plays an integral part ofeffectively communicating spatial route direction information,particularly for children.
� 2014 Elsevier Inc. All rights reserved.
Introduction
Nonverbal behaviors such as hand gestures are integral components of communication and areoften produced when conveying spatial information across all ages and cultures (Alibali, 2005; Driskell& Radtke, 2003; Feyereisen & de Lannoy, 1991; Goldin-Meadow, 2000). Our gestures are thought todirectly enhance comprehension, learning, and memory for verbal messages by illustrating concepts,conveying additional information, and providing additional cues (Hostetter, 2011; McNeill, 1992).
E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103 93
However, there may also be indirect benefits such as improving a speaker’s verbal message, maintain-ing a listener’s attention, and building rapport, thereby enhancing communication (Maricchiolo,Gnisci, Bonaiuto, & Ficca, 2009). Directly and indirectly, the gestures we produce and observe appearto play an important role in shaping the way we think and communicate with other people.
An alternative mechanism by which observing gestures may enhance comprehension is co-speechgestural imitation, where observing gestures elicits overt gesture production from listeners, therebyenhancing communication (Holler & Wilkin, 2011). The current study, therefore, aimed to examinewhether the presence of gesture at encoding has an impact on the extent to which the spoken messageis retained and subsequently recalled by adults and children. In addition, the extent to which individ-uals gestured at recall was examined, as was the effect that gesture at recall had on the reporting ofthe encoded message.
Gesture in communication
The communication of spatial information is a dynamic process involving representational gesturessuch as deictic, metaphoric, and iconic gestures (Allen, 2003; McNeill, 1992). These gestures enhanceword and narrative comprehension in adults and children aged 4 years (Driskell & Radtke, 2003;McNeil, Alibali, & Evans, 2000; Thompson, Driscoll, & Markson, 1998). Deictic gestures are pointingmovements, conveying directional information that orientates the listener (Allen, 2003; McNeill,1992). Iconic gestures provide an image by depicting concrete referents that directly relate to theaccompanying speech (e.g., making ball-shaped motions with hands to convey ‘‘ball’’). Metaphoric ges-tures also provide an image, but of abstract concepts rather than concrete referents (e.g., open palmrotating left and right to represent ‘‘almost’’), and can also indicate spatial locations to metaphoricallyrefer to characters, locations, or parts of a story (e.g., pointing at a story character) (Allen, 2003; McNe-ill,1992; Özçalis�kan & Goldin-Meadow, 2005). Nonrepresentational hand gestures, called beat gestures,are simple rhythmic hand movements that hold no apparent semantic value (Allen, 2003; McNeill,1992). The range of gestures suggests that there may be both direct and indirect mechanisms throughwhich the presence of gestures may enhance spatial direction communication (Hostetter, 2011).
Underlying mechanisms
Representational gestures often occur spontaneously during speech production and are thought bysome theorists to be two expressions of a linguistics-based communication system (Allen, 2003;Cameron & Xu, 2011; McNeill, 1992). The gesture as simulated action (GSA) framework, as proposedby Hostetter and Alibali (2008), holds that the perceptual and cognitive processing of messages con-veyed in speech and gesture activates perceptual and motor states in speakers and listeners. In thisway, gestures that accompany speech contribute to the listener’s comprehension by creating in the lis-tener a cognitive simulation or mental representation of the message (Alibali & Hostetter, 2010). Thisconceptualization of gesture leads to two possible pathways whereby gesture might benefit cognitiveprocesses of listeners. First, the presence of gesture may activate cognitive processes by eliciting mentalrepresentations of the message (Hostetter & Skirving, 2011; Sassenberg & van der Meer, 2010).Alternatively, the presence of gesture may activate cognitive processes by eliciting overt mimicry(Alibali & Hostetter, 2010; McNeil et al., 2000; Morsella & Krauss, 2004; Sassenberg & van der Meer,2010; Wheeler, 1966). Both pathways are thought to contribute to the listener’s comprehension and re-call of the message (Alibali & Hostetter, 2010; Hostetter & Alibali, 2008). However, this framework doesnot provide a complete model for how gestures benefit cognition and communication because it doesnot explain the role of beat gestures. It is possible that beat gestures enhance communication indirectlyby emphasizing specific words or phrases (Hostetter & Potthoff, 2012). Comprehending and communi-cating spatial route directions is a task in which the use of gesture may benefit both adults and children.
Communication of spatial route directions
To date, only one study has examined the role of gesture in spatial term communication and com-prehension (Driskell & Radtke, 2003). Adults performed a word association type task, where they saw
94 E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103
a spatial term (e.g., ‘‘under’’), conveyed either with only speech or with speech and gesture (Driskell &Radtke, 2003). Listener comprehension was enhanced when accompanying gestures were used, espe-cially for spatial location terms (e.g., ‘‘on’’), to a greater extent than manipulation/movement terms(e.g., ‘‘open’’) (Driskell & Radtke, 2003). Similarly, Emmorey, Tversky, and Taylor (2000) observed thatwhen adults were recalling previously memorized maps, they produced gesture strings where individ-uals used one hand to symbolize a landmark and its location relative to other landmarks, referred to bythe other hand. Adults’ speech production has also been demonstrated to improve when producinggesture while providing large-scale verbal route directions to locations unfamiliar to the listener (Al-len, 2003). However, research has not yet provided a complete picture for the role of gestures in thecommunication of route directions.
Existing research demonstrates that children of different ages use gestures in spatial tasks, includ-ing problem solving (Goldin-Meadow, Cook, & Mitchell, 2009), Piagetian conservation reasoning(Church, Kelly, & Lynch, 2000), communication of route directions (Iverson, 1999), and comprehensionof requests (Kelly, 2001; McNeil et al., 2000; Morford & Goldin-Meadow, 1992). Although child perfor-mance on these spatial tasks suggests that children comprehend deictic and iconic gestures presentedat encoding from 12 months of age, it has not yet been established whether these conclusions can beapplied to other gesture types, including metaphoric and beat gestures, specifically within the contextof verbal route directions. In addition, due to differences in gesture and vocabulary development ofpreschool-aged children, the comprehension of spatial directions may be affected by the presenceof gesture at encoding (Coplan & Gleason, 1988; McNeil et al., 2000). Therefore, this study examinedwhether the presence of gesture and/or the types of gesture presented enhance spatial message reten-tion and recall compared with spatial messages presented without any accompanying gestures.
The current study
The current study was designed to examine whether the presence of gesture at encoding has animpact on the extent to which that information is retained and subsequently recalled in adults andchildren. Based on existing literature and theory, it was predicted that the presence of gesture atencoding would lead to better task performance than hearing the message conveyed with no accom-panying gesture. It was also expected that the type of gesture presented would influence task perfor-mance. Individuals presented with a combination of gestures at encoding, including beat, deictic,metaphoric, and iconic gestures (combined gesture condition), were expected to perform the task bet-ter and produce more gestures at recall than individuals presented with beat gestures or no gestures atencoding. Similarly, exposure to beat gestures at encoding was expected to enhance task performanceand elicit gesture production to a greater extent than the absence of gesture at encoding. Regardingdevelopmental differences, it was expected that adults would perform the task better and producemore gestures at recall compared with children.
Regarding spatial language, it was expected that more location terms would be recalled comparedwith movement terms, in particular when participants were exposed to gesture during encoding.Lastly, this research aimed to explore the effect of presenting gestures at encoding on the productionof gestures at recall. This particular research question was exploratory in nature, with no directionalhypotheses.
To that end, a single small-scale spatial direction task was employed to compare the effects of thepresence of gesture (no gesture, beat gesture, or combined gesture) on adults’ and children’s task per-formance. In the current study, adults and children were presented with a verbal description of a paththrough a small-scale scene and, depending on the assigned condition, with accompanying gestures.
Method
Participants
A total of 93 children were recruited from preschools in the area of Sydney, Australia (44 boys and49 girls, mean age = 4 years 3 months, SD = 4 months, range = 3 years 4 months to 4 years 9 months).
E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103 95
A total of 95 adults were recruited from an introductory psychology unit at Macquarie University andfrom a gym located in Sydney. The combined adult group consisted of 50 male and 44 female partic-ipants (mean age = 28 years, SD = 7.6, range = 17 to 49). One child was removed from analysis due to afailure to provide verbal or gestural responses. One additional child and one adult were removed dueto technological malfunction. As a result, 91 children and 94 adults composed the final sample.
Design and materials
Participants were presented with a single small-scale spatial array constructed from Lego materialsadapted from Iverson (1999) and Levinson (1997). The Lego base plate measured approximately38 � 38 cm (15 � 15 inches) and consisted of objects made from Lego pieces arranged such that thetarget path was not obvious (see Fig. 1). A Lego character was described by the researcher to take acertain route through the spatial array.
Participants in each age group (child or adult) were randomly allocated to one of three conditions:combined gesture, beat gesture, or no gesture. Participants in the combined gesture condition receivedthe verbal path description and 20 accompanying gestures: deictic (n = 5), beat (n = 5), iconic (n = 5),and metaphoric (n = 5). For example, ‘‘At the police station’’ (researcher points at the police stationwith right hand as the words ‘‘police station’’ are verbalized), ‘‘Lego man shakes hands’’ (researcherrepeatedly moves right hand up/down with open palm facing left, fingers extended horizontally awayfrom the body as the words ‘‘shakes hands’’ are verbalized) ‘‘with the policeman and then starts walk-ing’’ (researcher performs a beat gesture as the words ‘‘starts walking’’ are verbalized), ‘‘passing infront of the windmill’’ (both palms facing chest, researcher makes an arc in the air, hands to finishin the same orientation approximately 20 cm away from chest when the words ‘‘in front of’’ are ver-balized). These are examples of deictic, iconic, beat, and metaphoric gestures, respectively, presentedto participants during the task. The beat gesture condition received the verbal description and a totalof 20 accompanying rhythmic hand movements that contained no semantic information. To enhancethe consistency of the experimenter’s performance, prescribed gestures for the combined and beatgesture conditions were performed at the same set points within the verbal script for each participant(see Appendix A for full script and gesture set points). Participants in the no gesture condition receivedonly the verbal description while the experimenter’s hands remained still.
Procedure
Adult participants were presented with the spatial array and instructed to examine it visually untilthey felt ready to begin. Child participants were presented with the spatial array, and together theexperimenter and children named the objects in the spatial array. The order of object naming wasrandomized.
All participants were then provided with the verbal description of the target path and, dependingon the assigned condition, the accompanying gestures. On the rare occasion when a participant
Fig. 1. Participant view of the Lego landscape.
96 E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103
appeared distracted during the demonstration, the participant was asked, ‘‘Are you listening, [name]?’’This applied to a small number of child participants and was not necessary for adults. Participantswere then given 120 s to complete a join-the-dots filler task. On completion of the filler task or after120 s had elapsed (whichever occurred first), participants were asked, ‘‘Can you tell me the path thatLego man took through the scene without taking Lego man along with you?’’ Following this, partici-pants were asked, ‘‘Can you now show me the path that Lego man takes through the scene, taking Legoman along with you?’’ If any participant asked for help, failed to respond, or did not interact with thespatial array, the experimenter said, ‘‘Where do you think Lego man went?’’ and/or ‘‘Where else did hego?’’ No other cues were given, and any requests for direction were answered with ‘‘Where do youthink he went?’’ Because participants’ spontaneous actions were of interest, specific instructionsregarding gesturing were explicitly avoided. No directional feedback or confirmation was providedby the experimenter. All participant responses were videotaped for later analysis. The entire proce-dure took 10 to 15 min per participant, depending on the length of each participant’s response. All par-ticipants completed the task in one session.
Coding
Speech and gesture produced at all time points were transcribed and coded following the codingprocedure used by Cook, Yip, and Goldin-Meadow (2010). Speech was transcribed verbatim, includingfilled pauses (e.g., ‘‘ummm’’) and hesitations. A location or movement was considered recalled cor-rectly if the speaker’s description of the target path included the location or movement. For example,along the target path, Lego man walks through a café and stops for a glass of milk. If the participantsaid, ‘‘He goes to the coffee shop and has a coffee,’’ the location and the movement were counted asrecalled because the location and the drinking movement were mentioned. A location or movementterm was counted as correct if the participant demonstrated the term through gesture (e.g., pointingat/touching the police station).
A location or movement was not counted as recalled correctly if the speaker’s description of thetarget path did not include the location or movement or if the location was not referred to by touch.Movements were not counted as recalled correctly if the movement reported was incorrect. For exam-ple, along the target path, Lego man rides past the front of the house and laughs at the boy balancingon the chair. If the participant said, ‘‘He rode past the guy balancing on the chair and waved at him,’’the location and movement were not counted as recalled correctly even though the boy and chair werementioned because the participant incorrectly identified the chair as the location and waving as themovement. The maximum scores for location and movement recall were 10 and 8, respectively, withtotal recall scored out of 18 (see Appendix B for list of items to be recalled). Total recall scores repre-sent the total recall from both recall occasions (i.e., both describing where Lego man went and show-ing the path). Participant gesture was coded when hand movements that accompanied speech did notserve a functional purpose (e.g., scratching), when direct manipulation of an object occurred (e.g.,picking up Lego figurine), when an emblem was displayed (e.g., thumb and pointer finger joined to-gether to mean ‘‘okay’’), or when the participant was instructed to demonstrate the path taken. Par-ticipant gesture was coded by type and categorized into one of four categories (i.e., beat, deictic,iconic, or metaphoric) according to descriptions outlined by Allen (2003) and McNeill (1992).
Reliability
Interrater reliability was assessed by having a second coder independently code 11% of the spokenand gesture transcripts. Reliability was evaluated by obtaining single-rater intraclass correlations(ICCs) assessed through a consistency model. Intraclass correlations were highly significant for loca-tion total, movement total, and overall total verbal recall (all ps < .001). For gesture coding, intraclasscorrelations were obtained for gestures made during the first recall for beat gesture (ICC = .998,p < .001), deictic gesture (ICC = .997, p < .001), iconic gesture (ICC = .890, p < .001), metaphoric gesture(ICC = .997, p < .001), first recall gesture total (ICC = .998, p < .001), and gesture total (ICC = .998,p < .001).
E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103 97
Results
Preliminary analyses
A preliminary analysis was conducted to determine whether recall and gesture production differedbetween the adult university and gym participants. A series of t tests revealed two significant differ-ences. First, gym participants were significantly older (M = 31 years, SD = 6.463) compared with uni-versity participants (M = 21 years, SD = 3.708), t(92) = 8.457, p = .002. This represents a medium sizeeffect (d = 1.763). Second, gym participants reported significantly more location items (M = 9.19,SD = 0.906) compared with university participants (M = 8.67, SD = 1.348), t(92) = 2.210, p = .03. Thisrepresents a small size effect (d = 0.461). Because the independent samples t tests established no sig-nificant differences between gym and university participants on any other measure of recall or ges-ture, the two samples were collapsed to form a single adult sample used in the main analysis. Errorrates of subsequent analyses are Bonferroni adjusted to control the family-wise error rate at .05.
Main analyses
Effect of presenting gesture at encoding on total recallA 2 (Age Group: child or adult) � 3 (Gesture Condition: no gesture, beat gesture, or combined ges-
ture) � 2 (Gender: males or females) between-participants analysis of variance (ANOVA) was carriedout to examine the effect of presenting gesture at encoding on recall.1 The analysis revealed a signif-icant effect of age group, F(1,173) = 272.73, p < .001, partial g2 = .61, such that adults recalled more infor-mation than children. There was a main effect of gesture condition, F(2,173) = 3.17, p = .045, partialg2 = .03, such that participants who were presented with gesture (beat or combined) scored higher ontotal recall than individuals who were not presented with gesture. There was no main effect of gender,F(1,173) = 0.023, p = .88, partial g2 < .01. There were similarly no two- or three-way interactions involv-ing gender (all ps > .05). Because analyses revealed no significant main effects or interactions involvinggender, further analyses involving the variable total recall did not include gender. The significant maineffects of age group and gesture condition remained after gender was removed from the analysis.Fig. 2 presents the mean numbers of items recalled by adults and children for each gesture condition.
Further analysis revealed a significant interaction between age group and the difference betweenthe no gesture condition and the average of the two gesture conditions (beat and combined),F(1,179) = 5.16, p = .024, partial g2 = .03, such that the difference in performance between adultsand children was greater for the no gesture condition than for the average of the two gesture condi-tions (see Fig. 2). Examination of simple effects revealed a significant difference between the no ges-ture condition and the average of the two gesture conditions on total recall for children,F(1,179) = 9.75, p = .002, partial g2 = .05, but not for adults, F(1,179) = 0.006, p = .934, partialg2 < .01. There was no significant difference between the beat and combined conditions for either chil-dren or adults (both ps > .05). There was no interaction between age and the two gesture conditions,F(1,179) = 0.36, p = .551, partial g2 < .01.
Effect of presenting gesture at encoding on location and movement item recallTotal recall was then split into recall of location and movement items. Because location and move-
ment item recall scores had different maximum scores, raw scores were transformed into percentagesfor this analysis. A 2 (Age Group) � 3 (Gesture Condition) � 2 (Feature Type: locations or movements)mixed design ANOVA was conducted, with age group and condition as between-participants factorsand feature type as a within-participants factor. Because there were no effects of gender for total re-call, gender was not included in this analysis. The analysis of feature type recall revealed a significanteffect of age, F(1,179) = 302.40, p < .001, partial g2 = .63, with adults performing better at recall thanchildren. There was similarly a significant main effect of feature type, F(1,179) = 587.65, p < .001, par-tial g2 = .77, with more locations recalled than movements recalled across all gesture conditions (see
1 No differences were found within the current child sample based on age.
Children Adults
Mea
n to
tal r
ecal
l
Age group
No Gesture
Beat Gesture
Combined Gesture
Fig. 2. Mean scores for the total verbal recall of children and adults for each gesture condition. Error bars represent standarderrors.
98 E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103
Figs. 3 and 4). There was a main effect of gesture condition, F(2,179) = 3.31, p = .040, partial g2 = .04,such that participants who were presented with either gesture condition recalled more location andmovement terms than participants who were not presented with gesture. Fig. 3 displays the recallof location items across age groups and gesture conditions. Fig. 4 displays the recall of movementitems across age groups and gesture conditions.
There were no significant two-way interactions between gesture condition and age group or be-tween gesture condition and feature type (all ps > .05). However, the analysis did reveal a highly sig-nificant interaction between feature type and age group, F(1,179) = 68.11, p < .001, partial g2 = .28,with the difference in performance between adults and children being greater for movements thanfor locations (see Figs. 3 and 4).
Analysis also revealed a significant three-way interaction among age group, gesture condition, andfeature type, F(2,179) = 3.37, p = .037, partial g2 = .04, such that for children there was a significant dif-ference between the no gesture condition and the combined gesture condition for the recall of loca-tions, F(1,179) = 9.81, p = .002, partial g2 = .05, but not for the recall of movements. For adults, onthe other hand, there was no significant difference between any gesture conditions for either locationsor movements (all ps > .05).
Effect of presenting gesture at encoding on total gesture produced at recallThe effect of presenting gesture at encoding on the dependent variable total gesture produced at
recall was examined using a two (age group) by three (gesture condition) between-subjects ANOVA.
0
10
20
30
40
50
60
70
80
90
100
Children Adults
Mea
n lo
catio
n re
call
Age group
No Gesture
Beat Gesture
Combined Gesture
Fig. 3. Mean scores for the recall of locations for children and adults for each gesture condition. Error bars represent standarderrors.
0
10
20
30
40
50
60
70
80
90
100
Children Adults
Mea
n m
ovem
ent
reca
ll
Age group
No Gesture
Beat Gesture
Combined Gesture
Fig. 4. Mean scores for the recall of movements for children and adults for each gesture condition. Error bars represent standarderrors.
E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103 99
The analysis revealed a significant effect of age group (F(1,179) = 8.10, p = .005, partial g2 = .04), withadults producing more gestures at recall than children, but no effect of gesture condition(F(2,179) = 1.236, p = .293, partial g2 = .01). There was no significant interaction between gesture con-dition and age group (F(2,179) = 2.375, p = .096, partial g2 = .03).
Effect of gesture produced at recall on total recallA Pearson product–moment correlation analysis was used to examine the relationship between the
amount of gesture produced at recall and the amount participants recalled. There was a significant po-sitive correlation between the amount of gesture produced at recall and the amount participants re-called verbally (r = .345, p = .01), such that as the amount of gestures adults and children producedincreased, so did the amount recalled. This correlation holds for both children (r = .476, p < .001)and adults (r = .204, p = .049) separately, although it should be noted that the relationship is strongerfor children. This analysis may be confounded, however, with the number of words spoken by partic-ipants. Participants who produced longer spoken responses may have had a chance to both gesturemore and recall more. It is important, therefore, to determine whether this relationship betweenamount gestured and amount recalled holds once the number of words spoken is held constant. Amultiple regression analysis was conducted to test whether total gesture produced at recall signifi-cantly predicted participants’ total recall above and beyond the number of words spoken. The resultsof the regression indicated that the two predictors (total gesture produced and number of words spo-ken) explained 51.6% of the variance (R2 = .516), F(2,182) = 97.08, p < .001. Both total gesture produced(b = .054, p = .016) and total words produced (b = .030, p < .001) significantly predicted total recall,indicating that total gesture produced significantly affected total recall beyond the effect of numberof words spoken.
Discussion
The current study has two main findings. First, the recall of spatial information was affected by thepresence of gesture at encoding for children but not for adults. Second, the presence of gesture atencoding affected the type of spatial information recalled by children but not by adults. The presenceof representational gestures (combined condition) may have aided task performance directly by illus-trating spatial concepts, conveying additional information and cues, or indirectly by maintainingattention and building rapport (Dijksterhuis & Bargh, 2001; McNeill, 1992). Beat gestures may alsohave aided message comprehension indirectly by emphasizing specific words or phrases (Hostetter& Potthoff, 2012). According to the GSA framework, the presentation of representational gestures acti-vates visual and motor mental representations of the spatial message to be recalled, improving recall
100 E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103
when representational gestures are present at encoding for adults and children (Alibali & Hostetter,2010). The current study provides some evidence to support this model.
For children, the presence of a wide variety of gestures at encoding (i.e., the combined gesture con-dition) elicited greater recall of spatial information than the presence of no gestures at encoding. Inaddition, the presence of beat gestures tended to enhance the recall of spatial information comparedwith no gestures at encoding. For adults, gestures accompanying speech at encoding did not appear toaid the recall of the spatial message. Only one other study has attempted to document the effect oficonic gestures, beat gestures, or no gesture presentation at encoding on recall for adults and childrenaged 4 or 5 years (So, Chen-Hui, & Wei-Shan, 2012). Greater recall was found for words accompaniedby iconic gestures. Data from this study are generally consistent with these findings regarding thebeneficial effects of being exposed to gestures during encoding. There are, however, two ways inwhich the current research extends that of So and colleagues (2012). First, the current study presentedparticipants with words in the context of sentences in a scenario as compared with the single wordspresented in So and colleagues’ study. Second, So and colleagues limited the comparison of gesture toiconic, beat, and no gestures, whereas the current study provides data on other gesture types, includ-ing metaphoric and deictic gestures.
There is evidence, in contrast to the current study’s findings, that gesture presented at encodingenhances adult comprehension and recall for single words (Driskell & Radtke, 2003; So et al., 2012),sentences (Thompson et al., 1998), and reasoning (Church et al., 2000). For example, Thompson andcolleagues (1998) found that when presented with video-recordings of a female narrator reciting shortsentences either with or without accompanying gestures, adults recalled more sentences than chil-dren aged 9 years and their recall of meaningful sentences increased when accompanied by a combi-nation of iconic and metaphoric gestures. This suggests that gesture presented at encoding is used byadults. However, there are fundamental differences between these studies and the current study,including verbal message length and content. Although statistically there was no ceiling effect in adultperformance, the task may have lacked sufficient sensitivity to determine the scope of benefits of ges-ture accompanying verbal route direction recall, particularly for adults. It is possible that adult routedirection recall would benefit from the presence of gesture at encoding for tasks when adult cognitiveresources are challenged to a greater extent.
Although spatial location information was recalled to a greater extent than spatial movementinformation across conditions, we found a significant relationship between age and the type of featurerecalled. The difference between adult and child recall was greater for movement information than forlocation information, and this difference was influenced by the presence of gesture at encoding. Thatis, compared with no gesture at encoding, the presence of representational gestures (combined con-dition) enhanced the recall of location information but not movement information for children. Foradults, the presence of gestures did not appear to enhance the recall of either location or movementterms. As indicated previously, existing research has found that gestures accompanying speech im-prove recall and task performance for adults and children (Church et al., 2000; Driskell & Radtke,2003; So et al., 2012; Thompson et al., 1998). To date, existing literature has not examined the recallof location and movement terms when presented with accompanying gestures in a route directioncontext. As such, the contrasting findings of the current study may reflect unique characteristics ofspatial route direction language and the accompanying gestures.
On the other hand, the greater recall of location terms compared with movement terms for childrenmay be due to characteristics of English language development, specifically the acquisition of nounsand verbs (Behrend, 1990; Gentner, 1982; Tomasello, Akhtar, Dodson, & Rekau, 1997). Because loca-tion terms are primarily nouns and movement terms are primarily verbs, it is possible that children’svocabulary knowledge may have affected language production at recall and, therefore, affected taskperformance (Gentner, 1982; Tomasello et al., 1997). Although this effect may at least partially explainthe greater recall of location terms than movement terms, it cannot explain the effects of gesture con-dition or the interaction among age group, gesture condition, and feature type where children pre-sented with representational gestures at encoding recalled more location terms (nouns) but notmovement terms (verbs) than children presented with verbal directions alone.
An alternative explanation for the mechanisms underlying the benefits of the presence of gestureon recall is listener gesture production. Gesture production at recall appears to aid recall of the spatial
E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103 101
message, such that the more participants gestured, the more they recalled correctly, and this effectwas not simply due to participants who spoke more both gesturing more and reporting more correctinformation. Previous research has also observed adults and children producing gestures when recall-ing spatial information (Allen, 2003; Cameron & Xu, 2011) and reproducing modeled actions, includ-ing causally irrelevant actions, to a high degree of fidelity (McGuigan, Makinson, & Whiten, 2011). Assuch, the presence of gestures at encoding may provide listeners with retrieval cues but also may elicitovert gesture production, thereby transferring the indirect benefits of gesture to the listener. The cur-rent study expands on existing research by examining effects of gesture presentation at encoding onrecall and gesture production during the communication of spatial route directions.
Overall, the results suggest that the use of gesture in social interactions is beneficial, particularlyfor the communication of spatial route direction information (Allen, 2003). However, the results of thisinvestigation should be interpreted in light of a number of limitations that can be addressed in futurestudies. Individuals interact differently with small-scale tasks compared with large-scale tasks(Iverson, 1999). The decision to use a small-scale array was to standardize the task between individ-uals at different locations. Future research could examine the effect of presenting gestures during theencoding of large-scale route directions on recall and gesture production at recall. Future researchcould also examine whether the presence of gestures at encoding may affect the communication ofroute directions using small-scale tools such as paper-based and electronic maps.2 In addition, thedifficulty of the task may have placed different cognitive demands on children compared with adults.Although the current study did not reveal floor or ceiling performances in either sample, the task mayhave lacked sufficient sensitivity to determine the scope of benefits of the presence of gesture on taskperformance. Future research could examine the role of gesture in spatial tasks of increasing difficultyor complexity.
Conclusions
The current study found that children, but not adults, benefited from the presence of gesture duringencoding of a spatial route direction task. This study increases the body of knowledge and providesvaluable insight into how individuals convey spatial information, indicating that gesture is an integralpart of effectively communicating spatial route direction information.
Acknowledgments
We thank Julie Austin for her assistance in coding and Mick Shaw for his assistance throughouttesting. We also thank the children and adults who participated.
Appendix A
Verbal description of the target path
Take a moment to look at the scene without touching it and let me know when you are ready tobegin.
Is it all right if I record you later on because your response is important to me and I want to be asaccurate as I can?
I am going to describe for you the path that Lego man takes through the scene, so it is importantthat you concentrate. Okay?
Note. Underlined words indicate gesture points.
(1) At the police station, Lego man shakes hands with the policeman and then starts walking(2) Passing in front of the windmill(3) Going through the mechanics workshop, Lego man picks up his skateboard
2 We thank an anonymous reviewer for this suggestion.
102 E.E. Austin, N. Sweller / Journal of Experimental Child Psychology 122 (2014) 92–103
(4) He rides his skateboard between the back of the ninja hut and the broken car(5) Getting off the skateboard to walk through the café, where he stops for a glass of milk(6) Patting the cat, Lego man rides his board around the apple tree(7) And around behind the swimming pool, waving at the girl swimming(8) He rides past the front of the house, laughing at the boy balancing on the chair, he leaves his
skateboard at the house(9) Lego man walks on under the crane
(10) To end up in the audience in front of the stage to cheer and clap at the singers’ performance.
If you would just like to have a go at this, join the dots task = 2 minutesCan you tell me the path Lego man takes through the scene without taking Lego man along the
path?Can you show me the path Lego man takes through the scene, taking Lego man along the path?What do you think the goal of the task was?
Appendix B
List of locations and movements
Location terms to be recalled:(1) Police station(2) Windmill(3) Mechanics workshop(4) Ninja hut/Broken car*
(5) Café(6) Apple tree(7) Swimming pool(8) House(9) Crane
(10) StageMovement terms to be recalled:
(1) Shakes hands(2) Picks up/rides skateboard⁄
(3) Drinks milk(4) Pats the cat(5) Waves at girl(6) Laughs at boy(7) Leaves skateboard at house(8) Clap/Cheer at singer
* Gestures were presented accompanying two items not included in the recall list: a location (ninja hut) and a movement (ridesskateboard). The location term was excluded because the path that Lego man takes through the scene passes between the ninjahut and the broken car, making it difficult to distinguish between whether participants remembered just one item or bothitems. Therefore, recall of the items ninja hut and broken car was counted as one item. Similarly, the movement ‘‘ridesskateboard’’ occurs immediately after the movement ‘‘picks up skateboard.’’ As such, we cannot distinguish between whetherparticipants recalled the first item only or both items (because to ride the skateboard, Lego man needs to first pick it up).Therefore, these items were collapsed into one item.
References
Alibali, M. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. SpatialCognition & Computation, 5, 307–331.
Alibali, M., & Hostetter, A. (2010). Mimicry and simulation in gesture comprehension. Behavioral and Brain Sciences, 33, 433–434.Allen, G. (2003). Gestures accompanying verbal route directions: Do they point to a new avenue for examining spatial
Church, R., Kelly, S., & Lynch, K. (2000). Immediate memory for mismatched speech and representational gesture acrossdevelopment. Journal of Nonverbal Behavior, 24, 151–174.
Cook, S., Yip, T., & Goldin-Meadow, S. (2010). Gesturing makes memories that last. Journal of Memory and Language, 63, 465–475.Coplan, J., & Gleason, J. (1988). Unclear speech: Recognition and significance of unintelligible speech in preschool children.
Pediatrics, 82, 447–452.Dijksterhuis, A., & Bargh, J. (2001). The perception–behavior expressway: Automatic effects of social perception on social
behavior. Advances in Experimental Social Psychology, 33, 1–40.Driskell, J., & Radtke, P. (2003). The effect of gesture on speech production and comprehension. Human Factors, 45, 445–454.Emmorey, K., Tversky, B., & Taylor, H. (2000). Using space to describe space: Perspective in speech, sign, and gesture. Spatial
Cognition and Computation, 2, 157–180.Feyereisen, P., & de Lannoy, J.-D. (1991). Gestures and speech: Psychological investigations. New York: Cambridge University Press.Gentner, D. (1982). Why nouns are learned before verbs: Linguistic relativity versus natural partitioning (Report No. 527). Retrieved
from https://www.ideals.illinois.edu/bitstream/handle/2142/17514/ctrstreadtechrepv01982i00257_opt.pdf?sequence=1.Goldin-Meadow, S. (2000). Beyond words: The importance of gesture to researchers and learners. Child Development, 71,
231–239.Goldin-Meadow, S., Cook, S., & Mitchell, Z. (2009). Gesturing gives children new ideas about math. Psychological Science, 20,
267–272.Holler, J., & Wilkin, K. (2011). Co-speech gesture mimicry in the process of collaborative referring during face-to-face dialogue.
Journal of Nonverbal Behavior, 35, 133–153.Hostetter, A. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137, 297–315.Hostetter, A., & Alibali, M. (2008). Visible embodiment: Gesture as simulated action. Psychonomic Bulletin & Review, 15, 495–514.Hostetter, A., & Potthoff, A. (2012). Effects of personality and social situation on representational gesture production. Gesture, 12,
62–83.Hostetter, A., & Skirving, C. (2011). The effect of visual vs. verbal stimuli on gesture production. Journal of Nonverbal Behavior, 35,
205–223.Iverson, J. (1999). How to get to the cafeteria: Gesture and speech in blind and sighted children’s spatial descriptions.
Developmental Psychology, 35, 1132–1142.Kelly, S. (2001). Broadening the units of analysis in communication: Speech and nonverbal behaviours in pragmatic
comprehension. Journal of Child Language, 28, 325–349.Levinson, S. (1997). Language and cognition: The cognitive consequences of spatial description in Guugu Yimithirr. Journal of
Linguistic Anthropology, 7, 98–131.Maricchiolo, F., Gnisci, A., Bonaiuto, M., & Ficca, G. (2009). Effects of different types of hand gestures in persuasive speech on
receivers’ evaluations. Language and Cognitive Processes, 27, 239–266.McGuigan, N., Makinson, J., & Whiten, A. (2011). From over-imitation to super-copying: Adults imitate causally irrelevant
aspects of tool use with higher fidelity than young children. British Journal of Psychology, 102, 1–18.McNeil, N., Alibali, M., & Evans, J. (2000). The role of gesture in children’s comprehension of spoken language: Now they need it,
now they don’t. Journal of Nonverbal Behavior, 24, 131–150.McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Morford, M., & Goldin-Meadow, S. (1992). Comprehension and production of gesture in combination with speech in one-word
speakers. Journal of Child Language, 19, 559–580.Morsella, E., & Krauss, R. (2004). The role of gestures in spatial working memory and speech. American Journal of Psychology, 117,
411–424.Özçalis�kan, S., & Goldin-Meadow, S. (2005). Do parents lead their children by the hand? Journal of Child Language, 32, 481–505.Sassenberg, U., & van der Meer, E. (2010). Do we really gesture more when it is more difficult? Cognitive Science, 34, 643–664.So, W., Chen-Hui, C. S., & Wei-Shan, J. L. (2012). Mnemonic effect of iconic gesture and beat gesture in adults and children: Is
meaning in gesture important for memory recall? Language and Cognitive Processes, 27, 665–681.Thompson, L., Driscoll, D., & Markson, L. (1998). Memory for visual–spoken language in children and adults. Journal of Nonverbal
Behavior, 22, 167–187.Tomasello, M., Akhtar, N., Dodson, K., & Rekau, L. (1997). Differential productivity in young children’s use of nouns and verbs.
Journal of Child Language, 24, 373–387.Wheeler, L. (1966). Toward a theory of behavioral contagion. Psychological Review, 73, 179–192.
