EMBODIED INTUITIVE INTERACTION IN
CHILDREN
Shital Desai
B.E (Hons), M.S
Submitted in fulfilment of the requirements for the degree of
Doctor of Philosophy
School of Design, Faculty of Creative Industries
Queensland University of Technology
2017
Embodied intuitive interaction in children i
Keywords
Children
Directly Manipulated Interfaces (DMI)
Embodiment
Embodied Interaction
Embodied Intuitive Interaction
Intuitive Interaction
Interaction model
Physical products
Tangible Embodied Embedded Interfaces (TEIs)
Tactile Interactions
Virtual Interfaces
ii Embodied intuitive interaction in children
Abstract
Products and interfaces for children have continued to evolve in terms of
complexity—moving from traditional physical products to virtual interfaces and
products with embedded electronics (referred to as Tangible Embodied Embedded
Interfaces, or TEIs). As the result of this evolution, there is an increasing need to
design children’s products through the lens of human factors and child-centred
design.
Physical products, virtual interfaces, and TEIs can be placed on a physical-
virtual continuum, and can be interacted with through various interaction modalities.
While Embodied interactions and intuitive interactions with products can result in
positive experiences for children, there is limited empirical research to investigate
the potential for product design to facilitate children’s Embodied intuitive
interaction. To address this research gap, this study focussed on tactile interaction as
an interaction modality to investigate children’s Embodied intuitive interaction.
Two experiments were carried out, with children playing with three types of
toys: physical, virtual, and TEI. Experiment 1 compared a physical toy with an
equivalent virtual app for intuitive interaction and aspects of Embodiment that
facilitate intuitive interaction. Experiment 2 investigated a physical product and a
TEI for intuitive interaction, and the impact of aspects of Embodiment on this
intuitive interaction.
A methodology that enabled a thorough elicitation of aspects of Embodiment
that facilitate children’s intuitive interaction was developed. Children from 5–12
years of age were observed playing with physical, virtual, and TEI products. In
Experiment 1, observations were followed with retrospective interviews. A co-
discovery method was used during observations and retrospective interviews. Data
was analysed using both qualitative thematic analysis and quantitative analysis.
The results suggest that physical products are more intuitive than virtual
interfaces. They also suggest that TEIs can also be intuitive, depending on the
configuration and integration of the system’s physical and virtual elements. Physical
affordances is the primary contributor to intuitive interaction in physical products
Embodied intuitive interaction in children iii
and TEIs, while virtual interfaces rely on perceived affordances for intuitive
interaction. People use natural and deliberate clues to detect physical and perceived
affordances, respectively. In the absence of such clues, affordances can remain
hidden. Symbolic, deliberate clues are often difficult for children to understand, and
they could be misinterpreted (as in the virtual interfaces). This study suggests the use
of Embodied representations as deliberate clues. Emergence, scaffolding, and co-
operative activity were the next most important contributors to intuitive interaction in
physical products and TEIs, and were found to be strongly correlated to physical
affordances.
The results are transferrable to an Enhanced Framework of Intuitive Interaction
(EFII), and provide an enhanced version of the concept. This enhanced version, in
turn, leads to new directions for research in the area. More specifically, the outcome
of this study is the model for Embodied intuitive interaction (MEII) that represents
children’s interaction with interfaces, using tactile interactions. MEII can be used to
design and evaluate children’s interfaces for Embodied intuitive interactions. MEII is
innovative in its evaluation of children’s interactions for Embodied intuitive
interaction. Furthermore, it could be used to design Embodied intuitive products for
children.
The significance of this study is its empirical verification of claims (in the
research literature) that physical products are more intuitive than virtual interfaces.
The study has conceptualised TEIs using a physical-virtual continuum and
interaction modalities. In so doing, it has paved the way for future research on
various configurations that TEIs could offer in terms of physical and virtual
integration. Furthermore, the child-centric and design-centric approach to
Embodiment and intuitive interaction taken in this study, means that Embodiment
need not be visualised as a concept involving bodily movements only, but could also
be conceptualised in terms of design aspects.
iv Embodied intuitive interaction in children
Table of Contents
Keywords .................................................................................................................................. i
Abstract .................................................................................................................................... ii
Table of Contents .................................................................................................................... iv
List of Figures ........................................................................................................................ vii
List of Tables ........................................................................................................................... ix
Statement of Original Authorship ............................................................................................ x
Acknowledgements ................................................................................................................. xi
Chapter 1: Introduction .................................................................................... 13
1.1 Definitions of concepts and terms ................................................................................ 13
1.2 Physical-Virtual continuum ......................................................................................... 17
1.3 Interaction modalities ................................................................................................... 17
1.4 Research Problem ........................................................................................................ 20
1.5 Aims and objectives ..................................................................................................... 23
1.6 Research significance ................................................................................................... 23
1.7 Thesis overview ........................................................................................................... 27
1.8 Summary ...................................................................................................................... 27
Chapter 2: Embodiment and Intuitive Interaction ......................................... 28
2.1 Embodiment ................................................................................................................. 28
2.2 Embodiment in Interaction Design .............................................................................. 36
2.3 Intuition and Intuitive Interaction ................................................................................ 39
2.4 Embodied Intuitive interaction ..................................................................................... 51
2.5 Summary ...................................................................................................................... 52
Chapter 3: Children’s Embodied Intuitive Interaction .................................. 55
3.1 Embodiment in children ............................................................................................... 55
3.2 Directly manipulated interfaces for children ................................................................ 60
3.3 Intuition in children ...................................................................................................... 62
3.4 Children’s Embodied intuitive interaction ................................................................... 64
3.5 Summary ...................................................................................................................... 67
Chapter 4: Aspects of Embodiment .................................................................. 69
4.1 Cognitive aspects of Embodiment ............................................................................... 69
4.2 Design aspects of embodiment..................................................................................... 75
4.3 Relationship between cognitive and design aspects of embodiment ........................... 81
4.4 Summary ...................................................................................................................... 83
Chapter 5: Research Design .............................................................................. 85
Embodied intuitive interaction in children v
5.1 Methodology ................................................................................................................. 85
5.2 Participants ................................................................................................................... 91
5.3 Data collection Methods ............................................................................................... 94
5.4 Analysis ........................................................................................................................ 98
5.5 Summary ..................................................................................................................... 107
Chapter 6: Embodied Intuitive Interaction: Physical and Virtual ............. 109
6.1 Methodology ............................................................................................................... 109
6.2 Analysis ...................................................................................................................... 114
6.3 Discussion ................................................................................................................... 126
6.4 Summary ..................................................................................................................... 129
Chapter 7: Primary Predictors of Embodied Intuitive Interaction ............ 131
7.1 Methodology ............................................................................................................... 132
7.2 Analysis ...................................................................................................................... 136
7.3 Discussion ................................................................................................................... 150
7.4 Summary ..................................................................................................................... 155
Chapter 8: Discussion ...................................................................................... 157
8.1 Design Implications .................................................................................................... 157
8.2 Model for Embodied intuitive interaction (MEII) for children .................................. 164
8.3 Summary ..................................................................................................................... 172
Chapter 9: Contributions and Future Work ................................................. 172
9.1 Contributions to knowledge ........................................................................................ 173
9.2 Research outcomes ..................................................................................................... 176
9.3 Research limitations ................................................................................................... 177
9.4 Future research ........................................................................................................... 179
9.5 Conclusions ................................................................................................................ 180
Bibliography ........................................................................................................... 183
Appendices .............................................................................................................. 207
Appendix A Strength of agreement based on ICC values ..................................................... 207
Appendix B Parametric and non-parametric statistical analysis ........................................... 208
Appendix C Thresholds for Effect size ................................................................................. 209
Appendix D Coding heuristics for Types of Interaction ....................................................... 210
Appendix E Coding heuristics for Aspects of Embodiment ................................................. 211
Appendix F Consent form for children and parents .............................................................. 212
Appendix G Consent form for school principal .................................................................... 214
Appendix H Image release consent forms ............................................................................ 215
Appendix I Image release information sheet itle .................................................................. 217
Appendix J Participant information sheet for experiment 1 at school .................................. 219
vi Embodied intuitive interaction in children
Appendix K Participant information sheet for experiment 1 at PAS lab ............................. 222
Appendix L Participant information sheet for experiment 2 ................................................ 225
Appendix M Recruitment email ........................................................................................... 228
Appendix N Education Queensland approval ...................................................................... 229
Appendix O QUT Human Research Ethics committee approval ......................................... 232
Appendix P Arrangements in Monkey Blocks given to children in Experiment 2 .............. 234
Embodied intuitive interaction in children vii
List of Figures
Figure 1 Physical-Virtual product continuum ............................................................ 17
Figure 2 Comparison of Intuitive Interaction Continua, shown through blue arrows (adapted from Blackler & Hurtienne, 2007) .................................... 45
Figure 3 Design aspects of Embodiment derived from cognitive aspects ................. 82
Figure 4 Research Design to investigate children’s Embodied intuitive interaction .................................................................................................... 88
Figure 5 XY plot of power for a range of sample size (N) value, 1 to 100 ............... 93
Figure 6 XY plot of power for a range of sample size (N) value, 1 to 100 ............... 93
Figure 7 Coding scheme showing theme groups and the corresponding sub-themes .......................................................................................................... 99
Figure 8 Physical Jenga toy (left), and virtual Jenga app (right) ............................. 110
Figure 9 Swiping and tapping to remove the block (above); placing the block on the top of the stack (below) ................................................................... 111
Figure 10 Colour-coded warnings of the danger of crashing the stack.................... 111
Figure 11 Children playing with the physical toy .................................................... 114
Figure 12 Children playing with the virtual interface .............................................. 114
Figure 13 Coding environment in Observer XT ...................................................... 115
Figure 14 Comparison of Number of Intuitive Interactions; Number of Layers Added; and Latency to decide for physical and virtual Jenga ................... 121
Figure 15 Box plot of use of aspects of Embodiment in physical and virtual toys ............................................................................................................. 124
Figure 16 Pair-wise comparisons of aspects of Embodiment for interactions with physical toy (left) and virtual toy (right) with mean ranks for each aspect ................................................................................................. 126
Figure 17 Osmo setup and Newton app ................................................................... 133
Figure 18 Children playing Osmo. The view of the tablet screen (on the left) shows Newton game in action and the view of children manipulating objects and drawing in the physical space (on the right) ........................... 134
Figure 19 Three types of blocks in Monkey Blocks: orange blocks with weights at one of the ends, green blocks with weights in the middle, blue blocks with no weights. .............................................................................. 135
Figure 20 Blocks and monkeys in arrangements ..................................................... 135
Figure 21 Example of an arrangement: black and white image given to children (left), equivalent coloured image (right) .................................................... 136
Figure 22 Number of intuitive, non-intuitive, and partially-intuitive interactions for the TEI Osmo ....................................................................................... 144
viii Embodied intuitive interaction in children
Figure 23 Intuitive, non-intuitive, and partially-intuitive interactions in the physical toy, Monkey Blocks ..................................................................... 145
Figure 24 Relative contributions of aspects of Embodiment to intuitive interaction with TEI game .......................................................................... 148
Figure 25 Relative contributions of aspects of Embodiment to intuitive interaction with Monkey Blocks ................................................................ 148
Figure 26 Artefact-Artefact Affordances: stack-ability and slide-ability of blocks in Monkey Blocks (above); pull-ability and slide-ability of blocks in Jenga (below) .............................................................................. 159
Figure 27 Research results incorporated into an Enhanced Framework of Intuitive Interaction (EFII) (adapted from Blackler et al., 2018) .............. 161
Figure 28 Embodied Cognition as distributed perceptual systems, (adapted from Gaines (1989) and Hinton (2014)) .................................................... 167
Figure 29 Perception side of the Model for Embodied intuitive interaction (MEII) in children ...................................................................................... 168
Figure 30 Actions side of the Model for Embodied Intuitive Interaction (MEII) in children. ................................................................................................. 170
Figure 31 Arrangements for Monkey Blocks game in black and white and colour ......................................................................................................... 234
Embodied intuitive interaction in children ix
List of Tables
Table 1 The type of products on the physical-virtual continuum .............................. 19
Table 2 Summary of Experiment 1 ............................................................................ 89
Table 3 Summary of Experiment 2 ............................................................................ 90
Table 4 Dependent for measures of intuitive interaction, successfulness and aspects of Embodiment .............................................................................. 120
Table 5 Descriptive statistics for Number of intuitive interactions, Number of layers added and Latency to decide corresponding to Type of Toy: Physical and Virtual Jenga ......................................................................... 121
Table 6 Mean Rank Values of Number of Intuitive Interactions, Number of Layers Added, and Latency to decide for each type of toy ....................... 122
Table 7 Mann-Whitney U Test Statistic of Number of Intuitive Interactions, Number of Layers Added, and Latency to decide for each type of toy ..... 122
Table 8 Descriptive statistics for Number of uses of aspects of Embodiment for physical and virtual Jenga .......................................................................... 124
Table 9 Mean Rank Values of Number of Intuitive Interactions, Non-Intuitive Interactions and Partially Intuitive Interactions ......................................... 146
Table 10 Regression coefficients and VIF values for the MRS system ................... 147
Table 11 Correlations between aspects of Embodiment (predictors) ...................... 149
Table 12 Strength of agreement based on ICC values, adapted from (Koo & Li, 2016) .......................................................................................................... 207
Table 13 Differences between assumptions for parametric and non-parametric data analysis methods, adapted from Field (2008) .................................... 208
Table 14 Thresholds for interpreting effect size, adapted from Cohen (1992, p.40) for correlation and Rosenthal & Rosnow (1991, p.361) for Mann Whitney U test. .......................................................................................... 209
Table 15 Coding heuristics for Types of Interaction ............................................... 210
Table 16 Coding heuristics for Aspects of Embodiment ......................................... 211
x Embodied intuitive interaction in children
Statement of Original Authorship
The work contained in this thesis has not been previously submitted to meet
requirements for an award at this or any other higher education institution. To the
best of my knowledge and belief, the thesis contains no material previously
published or written by another person except where due reference is made.
Signature: QUT Verified Signature
Date: October 2017
Embodied intuitive interaction in children xi
Acknowledgements
This thesis would not have been possible without the continued support,
mentorship and advice from my supervisors, Associate Professor Thea Blacker and
Professor Vesna Popovic. I have received immense support and advice for which I
am very grateful. Your continued enthusiasm, vision, and patience during this
process have been invaluable. I thank you very much for that. I also thank Mr. Ray
Duplock and Denise Scott. The assistance and knowledge you have both contributed
to this work has been of great importance.
Children and parents who were kind enough to help me out in the study,
without your help, it would not have been possible. I thank all my colleagues from
the People and Systems Lab (PAS lab), Creative Industries Faculty (CIF), Science
and Engineering Faculty (SEF), QUT-wide and world-wide. There are many to
thank individually, but specific mentions go out to Marianella Chamorro-Koc,
Marissa Lindquist, Deborah Fels, Mini Suresh, Shayne Beaver, Levi Swaan and
Mitch McEwan: You all have supported me at different times and in different ways.
I cannot thank you enough. Specific mention goes to Helena Papageorgiou from
Creative Industries Higher Degree Research support team, thank you for all the help
and for being patient with my constant demands and queries.
I am very thankful to the entire research community for your constructive
feedback and interesting discussions, it has greatly helped me in putting this thesis
together. I thank all reviewers and examiners for your invaluable feedback.
To my students for their invaluable and interesting discussions which have
informed my research and my knowledge and understanding of other areas of Design
Research, I thank you all.
Lastly, my family for all the love and support, I thank you. To my daughter
Vishakha, I hope this journey of mine will inspire you to reach greater heights and
goals in your own life. To my husband, Gaurang, I thank you for the strength,
patience, and encouragement all through this journey. Lastly, love and affection
shown by my furry mate, Archie, in difficult times meant a lot to me.
xii Embodied intuitive interaction in children
Embodied intuitive interaction in children 13
Chapter 1: Introduction
This chapter introduces Embodiment and intuitive interaction, and the research
problem pertaining to children’s Embodied intuitive interaction. Key concepts and
terms relevant to the study are defined in Section 1.1. The physical virtual
continuum that is used to discuss the range of products used by children is discussed
in Section 1.2. Section 1.3 then discusses interaction modalities in children as ways
in which they interact with interfaces and products. Section 1.4 discusses the
research problem, and identifies the gaps in research that lead to the research
questions. The objectives of this study are stated in Section 1.5, and the significance
of its findings is detailed in Section 1.6. Finally, Section 1.7 provides an overview of
the thesis.
1.1 Definitions of concepts and terms
Embodiment is described as an idea that an organism’s sensory-motor
capacities, body and environment not only play an important role in cognition, but
the manner in which these elements interact enables particular cognitive capacities to
develop and determines the precise nature of those capacities (Clark, 1997; Cowart,
2004; Dourish, 2001).
Cognitive processes in Embodiment rely on perception action couplings
(Gibson, 1979) which is discussed in Chapter 2 and Chapter 4 in detail. Perception
action coupling is a circular relationship where perception drives action which in turn
aids perception. The perception identifies and uses invariant information in the
environment. The action component involves movement and control features being
set and regulated to achieve an action goal.
Intuitive interaction is a subconscious application of one’s prior knowledge,
and this knowledge is derived from various sources. Familiarity with features of
products, or prior experience, is one of the sources of prior knowledge in intuitive
interaction (Blackler, Popovic, & Mahar, 2010). The definition of intuitive
interaction thus draws upon prior knowledge and its subconscious application:
Intuitive use of products involves utilising knowledge gained through other gervaisexperience(s). Therefore, products that people use intuitively are
14 Embodied intuitive interaction in children
those with features they have encountered before. Intuitive interaction is fast and generally non-conscious, so people might be unable to explain how they made decisions during intuitive interaction. (Blackler et al., 2010, p2)
Researchers at the Intuitive Use of User Interfaces (IUUI) research group in
Germany—Hurtienne & Blessing, 2007; Israel, Hurtienne, Pohlmeyer, Mohs,
Kindsmüller, et al., 2009; Mohs et al., 2006; Naumann et al. (2007)—defined
intuitive interaction as a relationship between a person and an artefact, rather than as
an attribute of an object: “A technical system is intuitively usable if the user’s
subconscious knowledge leads to effective interaction.” Embodiment and intuitive
interaction are discussed in further detail in Chapter 2, Chapter 3 and Chapter 4.
Interactions with interfaces and systems integrated and/or embedded with
technology are grouped under the common umbrella term Tangible Embodied and
Embedded Interfaces (TEIs). The broad spectrum of products under TEIs is evident
from the recent call for papers at the TEI 2017 conference (TEI 2017, 2017). The
conference asked for submissions on wearables; products that bridge physical and
digital worlds; shape-changing displays; and on the role of physicality in Embodied
sense making and tangible interactions. As advances in technology progress, and the
uses of TEIs in interaction design are explored, new configurations of TEIs will
continue to emerge. Papers presented at past TEI conferences have variously
discussed TEI systems as ‘Tangibles’ (Gervais, Frey, Gay, Lotte, & Hachet, 2016);
‘Embodied interfaces’ (Malinverni, Ackermann, & Pares, 2016); ‘Embedded
systems’ (Yoon, Huo, & Ramani, 2016); and ‘Mixed reality systems’ (Robert,
Wistorrt, Gray, & Breazeal, 2011).
‘Tangibles’ are defined as things that are “capable of being perceived
especially by the sense of touch” (Merriam-Webster, 2004). Tangible User Interfaces
(TUIs) was the term introduced by Ishii (2008) to represent objects that embed digital
information so that it can be directly manipulated and investigated. As products and
appliances became more intelligent, tangible interfaces gained popularity in
disciplines such as Industrial Design, Product Design, and Interactive Arts
(Djajadiningrat et al., 2004). Hornecker & Buur (2006) used the term ‘tangibles’ to
describe the meaning of tangibility in disciplines other than Human Computer
Interaction (HCI). They described tangibles as user interfaces and interactions that
draw on the tangibility and materiality of the interface, the physical Embodiment of
Embodied intuitive interaction in children 15
data, whole body interaction, and the Embodiment of interface and user interaction in
real spaces and contexts. ‘Tangibles’ are thus defined as
Systems that rely on Embodied interaction, tangible manipulation, physical representation of data, and embeddedness in real space. (Hornecker & Buur, 2006, p. 1)
‘Embedded interfaces’ refer to systems with computing elements with a
dedicated function, often with real-time computing constraints (Heath, 2002, p2).
The computing elements control the behaviour and operations and, in some cases,
even the look of the systems in which they are embedded. ‘Embodied interfaces’
refer to sensory interfaces that involve Embodied awareness of the human body and
the physical world (Moeller & Kerne, 2012).
Mixed reality systems are TEIs consisting of both physical elements and
virtual elements (Milgram & Colquhoun, 1999). They were developed to allow users
to take advantage of the continuous innovations in technology. Mixed reality
systems commonly referred to as Augmented systems, could either consist of real
physical world augmented with virtual objects, as in Augmented Reality (Azuma et
al., 2001), or the virtual world augmented with real physical objects, as in
Augmented Virtuality (Regenbrecht et al., 2004).
The primary differences among the TEIs discussed above are the objectives
and functionalities that they were designed for. For example, Mixed Reality systems
were developed to create immersive environments with technology. TUIs were
intended to provide a way of interacting with digital information, using various
interaction modalities. TEIs, in general, differ in their system configurations,
depending on the way their physical and virtual elements are configured and coupled.
There is ongoing research into the use of TEIs for children (Enyedy et al., 2017;
Fan, Antle, & Cramer, 2016). That research is, however, mostly focussed on the use
of various TEIs in the context of children’s learning. There is some limited research
on embedded systems in the context of children’s play (Sakai & Sugano, 2016), such
as embedding a tracking device in stuffed toys. TEIs are further discussed in the
context of Embodiment and children’s intuitive interaction in Chapter 2 and Chapter
3.
Affordances are the possibilities of actions that can be performed on an object
or environment (Gibson, 1979). Children should be able to discover these possible
16 Embodied intuitive interaction in children
actions through the interpretation of clues (Dotov, Nie, & De Wit, 2012). Inherent
properties of objects and environments such as shape, material, size, and colour
could offer clues for possible actions (Harrison & Hudson, 2009; Kim, 2015;
Norman, 1999). For example, children could interpret a soft and squishy object as
something to be pressed and squeezed; or a spherical object could be interpreted as
something to be rolled or bounced. Since these clues are natural to the objects, they
are referred to as ‘natural clues’. However, clues could also be deliberately
incorporated into the design of an object to facilitate the discoverability of its
possible operations; these clues are referred to as ‘deliberate clues’ (Norman, 1999).
The absence of such clues could result either in incorrect actions (actions that are
unintended or unexpected), or no action with the objects at all. Gaver (1991) referred
to such affordances as ‘false affordances’ and ‘hidden affordances’, respectively.
Thus, the term ‘clues’ in this research means
a set of features that either exist naturally in objects and environments or incorporated deliberately in design to facilitate discoverability of possible actions and operations on the objects and environment.
The most common forms of deliberate clues are symbolic in nature, and have a
definite meaning. The understanding of such deliberate clues is developed through
experience and/or learning of the meaning of the symbols in the same context of use,
or from another context of use. These deliberate clues are referred to as ‘symbolic
clues’. Another form of deliberate clues, suggested by Pezzulo (2011), is called
‘Embodied representations’. These are re-enactments of motor processes in the mind
that are generated by watching something in action, rather than by performing
actions oneself. In other words, Embodied representations are action-based clues that
trigger the same neural processes of action and perception as does the performance of
those actions. Animated icons and simulations of objects depicting their functionality
are examples of Embodied representations. Natural and deliberate clues are further
discussed in relation to affordances in Section 5.4.1.
When interpreted and acted upon, clues, both natural and deliberate, could
reveal another clue (or clues) for further possible action. Gaver (1991) refers to such
affordances as ‘sequential in time affordances’. Similarly, affordances could be
nested in space; that is, more than one clue is available at different places to facilitate
the perception of possible action (or actions) with an element.
Embodied intuitive interaction in children 17
1.2 PHYSICAL-VIRTUAL CONTINUUM
Children have access to products that range from those with physical elements
only, to those with virtual elements only. These products can be placed on a
physical–virtual continuum (as shown in Figure 1). The physical environments, on
the left, define products and interfaces consisting solely of real physical objects;
these include objects such as blocks, sticks, pens, and paper. These objects lack
computing elements, and people mostly rely on their sensorimotor knowledge to
interact with them. The virtual environments, on the right, define products and
interfaces consisting solely of virtual elements; these include conventional computer
graphic simulations, and apps on a screen (such as a monitor, tablet, or phone).
Figure 1 Physical-Virtual product continuum
Anything between the two extremes of the continuum are TEIs, that is, systems
or interfaces with both physical and virtual environments, where both real world and
virtual world objects co-exist. TEIs towards the extreme left of the continuum would
allow more interactions with the physical elements, and those towards the extreme
right would allow more interactions with the virtual elements.
1.3 INTERACTION MODALITIES
Children’s actions with artefacts are the result of the simultaneous deployment
of various semiotic resources, such as speech and body gestures, to convey
information (Goodwin, 2000). Streeck (2013) and Streeck, Goodwin, & LeBaron
(2011) further added that children’s interactions with the physical, material, and
social world also contribute to Embodiment. Thus, interaction modalities—that is,
the ways in which children carry out actions with products, interfaces, and the social
world—have been explored for Embodied interaction. Traditional interaction
modalities have limited children’s interactions to tactile and visual. However, with
the integration of technology in TEIs, new interfaces with various interaction
modalities, that accept physical interactions (gestures, touch, and body movements)
as inputs to digital environments, have further strengthened the possibility of
18 Embodied intuitive interaction in children
incorporating Embodiment with technology in children’s learning and playing (van
den Hoven & Mazalek, 2011).
Franinović & Serafin (2013) discussed the use of sound as an interaction
modality. Fink et al. (2014) studied the use of robots in encouraging children to tidy
their rooms. They developed robotic toy boxes that initiated an interaction by
encouraging children to pick up toys after play and put them in the box. The toy box
then provides auditory and visual feedback to the children. For example, Bubu
Monstry makes chewing and eating sounds when a toy is placed in the box; when all
toys are back in the box, Bubu vibrates and burps. Bubu‘s moving eyebrows and
eyes, provides positive visual feedback. Other interaction modalities with TEIs are
full body interactions, gestural interactions, and direct manipulations. Full body
interaction is the use of full-body movement as an interaction modality for physical
and digital artefacts. Gestural interactions allow individuals to foster bodily and
physical engagement with an artefact (Aslan, Primessnig, Murer, Moser, &
Tscheligi, 2013).
Natural user interfaces (NUIs) allow interactions in ways that are more like
those used to interact with people and objects in the real world. NUIs include natural
forms of interaction, such as bodily movements (Hoffmann, Schuster, Schilberg, &
Jeschke, 2016); spatial gestures (Hay, Newman, & Harle, 2008); facial expressions
(Veeriah, Pilarski, & Sutton, 2016); speech (Dahl, 2017); and touch (Anthony,
Stofer, Luc, & Wobbrock, 2016). Because they are natural, NUIs are thought to be
natural to use, and require very little training and learning. Fitts’s law has played an
important role in Human Computer Interaction (HCI), and is often used to design and
evaluate NUIs for ease of use. The law states that the time of completion of an
interaction is a function of distance, and the size of the target which is acted upon
(Fitts, 1954). Time of completion is very important in interaction design, as the
longer it takes a user to navigate an interface, the higher the possibility of their
becoming frustrated with, or even giving up on the interface. Thus, NUIs that are
designed for faster time of completion are interfaces that excel in terms of
learnability. Interactions with NUIs require knowledge that users derive from their
everyday lives. Thus, NUIs exploit metaphors with real-world objects to facilitate
interaction with digital objects in the digital world.
Embodied intuitive interaction in children 19
Gestural interactions and NUIs were popularised in futuristic movies such as
Minority Report (Arthur, 2010; Dick, 2002) and Mission Impossible (Cruise,
Wagner, & Brian, 1996) and, in the children’s context, have been popular in gaming
environments such as XBOX Kinect (Microsoft, 2010), and interactive tables and
walls (Anthony et al., 2016). Recent research efforts have focussed on the role of
gestural interactions and NUIs in Mixed reality systems (Santos, Cardoso, Ferreira,
Ferreira, & Dias, 2016). There could be overlaps between full body interactions and
gestural interactions, as a combination of both is often used in NUIs (Macaranas,
2013).
However, Mistry & Maes (2009), Norman (2010) and Shaer & Hornecker
(2010) suggested that gestural interactions are neither easy to learn nor remember.
Some gestures can be confusing as they differ from one culture to another, and the
derivation of meaning from gestures and body movements relies heavily on an
individual’s experience and knowledge. Norman (2010) cites Indian head shake and
hand waving gestures as some of these confusing gestures. Furthermore, gestural
interactions do not leave behind any traces of interaction. Thus, if an interaction does
not result in an appropriate response, it is impossible for an individual to determine
the cause of the failure.
Direct manipulations with tactile interactions are an alternative to body and
gestural interactions. They eliminate the coupling issues between the user’s
interaction and the artefact. Children are familiar with tactile interactions from birth,
and through their developmental stages as they get older. Thus, tactile interactions
are the most common modes of incorporating Embodiment in children’s learning
activities, especially in the early years (Robins & Dautenhahn, 2014). Direct access
to the spatial and material properties of the elements and features that are touched,
allows children to provide meaning to their interactions.
This research studied children’s Embodied intuitive interaction, using tactile
interactions as an interaction modality. To this end, it explored three types of
products, as identified on the physical-virtual continuum: physical, virtual, and TEIs
(Table 1).
Table 1 The type of products on the physical-virtual continuum
20 Embodied intuitive interaction in children
1.4 RESEARCH PROBLEM
In their everyday lives, children are increasingly expected to learn to use
complex concepts, devices, and products. As these concepts and products become
increasingly complex, they become more difficult to understand and use. The
essential criteria for usability of a device is that its operation should be easy to learn,
and that children can, either consciously or subconsciously, anticipate its behaviour
(Hartson, Andre, & Williges, 2003).
Embodied interactions result in experiences that are engaging, magical, fun,
emotional, and affective (Fincham, 2016; King & Chang, 2016). Wensveen et al.
(2004) suggested that children’s intuitive interactions with products and interfaces
result in positive experiences. Israel, Hurtienne, Pohlmeyer, Mohs, Kindsmuller, et
al. (2009) suggested that knowledge from Embodied interactions with the physical
world contributes to intuitive interactions. Blackler & Hurtienne (2007) suggested
that the simplest form of intuitive interaction is based on Embodied knowledge
learned early in life. There is an increasing push to incorporate Embodiment into
children’s learning (Björk-Willén & Cromdal, 2009; Breathnach, O’Gorman, &
Danby, 2016; Whitebread, Basilio, Kuvalja, & Verma, 2012). Embodied play and
Embodied learning are powerful forces in children’s interactions with the world
(Whitebread et al., 2012).
An Embodied view of cognition could contribute immensely to an
understanding of how Embodiment impacts and influences intuitive interaction.
Nevertheless, the bodies of literature on intuitive interaction and Embodied
interaction are largely independent of each other. For example, Hummels, Smets, &
Overbeeke (1998) discussed the possibilities that gestures could offer to support
intuitive human-computer interfaces for product design; however, their study was
Type of Product
Interaction modality Physical virtual continuum
Product/Interface used in this study
Physical Tactile interaction with the physical elements
Extreme left of the continuum
Jenga blocks Monkey Blocks
Virtual Tactile interaction with the virtual elements
Extreme right of the continuum
Jenga app
TEIs Tactile interaction with the physical elements and visual interaction with the virtual elements
Middle of the continuum, towards the Physical, left end of the continuum
Osmo with Newton app
Embodied intuitive interaction in children 21
focussed on gesture, while the meaning of Embodiment was limited to the hand
movements used in gestures.
Embodiment and Embodied interactions are considered to offer natural and
intuitive form of interactions. This claim has been made in literature, few studies
such as Antle, Corness, & Droumeva (2009a), discuss the cognitive structures of
intuition underlying Embodiment. Research in Embodied interaction design has
emphasised the affordances offered by body movements in interacting with a
product, and has described Embodiment in relation to the role of body and space in
interaction design. This has led to the use of the term ‘Embodiment’ to mean bodily
action or physicality (Ahmet, Jonsson, Sumon, & Holmquist, 2011; Grufberg &
Jonsson, 2012; Holmquist et al., 2010; Johansson et al., 2011; Schafer et al., 2013;
Søndergaard, 2013).
There is more to Embodiment than simply physical body movements.
Hurtienne, Löffler, Gadegast, & Hußlein (2015), and Macaranas, Antle, & Riecke
(2015) suggested the use of metaphors in interaction. The meanings of these
metaphors are derived from people’s Embodied knowledge of the world, or their
sensorimotor knowledge. To support intuitive interaction, interfaces can be designed
with interactional metaphors based on abstract representations with physical body
movements as means to incorporate those metaphors. Antle & Wise (2013) studied
an interactive musical sound-making environment that uses an Embodied metaphor
to map body-based input with audio output. In their empirical experiments, some
children (aged 7 to 10 years) were asked to perform specific sound sequences by
varying a single sound parameter (volume, tempo, pitch, and rhythm) associated with
specific physical movements, while others used a system without Embodied
metaphors. They found that the Embodied metaphor-based system was more intuitive
than the system without the Embodied metaphors. The experiments have proved the
effectiveness of Embodied metaphors in the intuitive use of interfaces. However,
further and ongoing research needs to focus on the appropriateness of Embodied
metaphors for children, as the discoverability of metaphors depends on children’s
past experience and knowledge which, when compared to an adult’s, is quite varied
(Bakker, Van Den Hoven, & Antle, 2011).
Direct manipulation and interaction with objects through tactile or haptic
interactions rely on Embodied interactions (Hornecker & Buur, 2006). Haans &
22 Embodied intuitive interaction in children
IJsselsteijn (2006) suggested that interacting with the environment around us through
touch offers a phenomenal extension of the self, enabling genuine Embodied
interaction. They further added that tactile interactions could offer Embodied
interactions with technology, eventually blurring the boundary between
“…unmediated self and the mediating technology...” (p156). Thus, directly
manipulated tactile interactions are one of the modalities used in Embodied
interaction. Being intuitive is attributed to touch-based interactions—tactile or haptic
interactions with static system properties such as being direct (Dix, 2011), easy to
learn, natural (Muller-Tomfelde & Fjeld, 2012), fast, simple, and effective (Jacoby et
al., 2009). However, few studies have attempted to empirically prove the validity of
the intuitive claims for tactile interactions with physical products. Tactile interactions
with TEIs can become complex as they often involve interactions at different places
or nodes on the product, and the corresponding feedback from the product is also at
different places/nodes (Wensveen, Overbeeke, Djajadiningrat, & Kyffin, 2004).
This research investigated Embodied intuitive interaction in three types of
interfaces and systems, as highlighted on the physical virtual continuum (Figure 1) -
physical products, TEIs and virtual interfaces—with tactile interaction as an
interaction modality.
Research in intuitive interaction has mostly focussed on adults (Blackler et al.,
2010; Lawry, Popovic, & Blackler, 2011) and adults in contexts such as airport
navigation (Cave, Blackler, Popovic, & Kraal, 2014) and web interaction (Mohan,
Blackler, & Popovic, 2015).There has been limited research into children’s intuitive
interaction (Desai, Blackler, & Popovic, 2015, 2016). Although there is ample
research into Embodied interaction in children (Marshall et al., 2009; Montemayor et
al., 2002; Sherman, Druin, & Montemayor, 2001; Zaman, Abeele, Markopoulos, &
Marshall, 2009), there is limited research that investigates Embodiment and intuitive
interaction in children (Antle, Droumeva, & Corness, 2008). To address this lack,
this study investigated the role of Embodiment in intuitive interaction through
children’s tactile interaction with physical products, virtual interfaces and TEIs It
investigated how children use their environment and social interactions to inform
their mind and body when interacting with an interface or a product. The study was
supported by two empirical experiments, as detailed in Chapter 6 and Chapter 7.
Embodied intuitive interaction in children 23
Specifically, the study focussed on the following research question and three sub-
questions:
What is the role of Embodiment in children’s intuitive interaction?
i. Can Embodiment facilitate intuitive interaction in children?
ii. Which design aspects of Embodiment facilitate children’s intuitive
interaction
iii. To what extent do the design aspects of Embodiment facilitate
intuitive interaction in children?
Design aspects of Embodiment were derived through the literature review of
Embodied cognition (Chapter 4) and were then used in Experiment 1 and Experiment
2 to address the three sub-questions.
1.5 AIMS AND OBJECTIVES
The aim of this research was to gain an understanding of aspects of
Embodiment and the role that they play in children’s intuitive interaction. Using
children’s play with toys as the context, the research objectives were to:
Understand the cognitive aspects of Embodiment in the literature, and
study its design aspects to facilitate children’s Embodied intuitive
interaction
Investigate children’s Embodied intuitive interaction with physical
products, a virtual interface (app), and TEIs
Investigate the influence of the design aspects of Embodiment on
children’s intuitive interaction when playing with physical toys, virtual
interface, and TEIs
Develop an interaction model that represents the use of Embodiment in
children’s intuitive interactions with physical and virtual products and
TEIs.
1.6 RESEARCH SIGNIFICANCE
This research makes a number of significant contributions through its
generation of new knowledge and its outcomes. New insights gained contribute to
24 Embodied intuitive interaction in children
advancing knowledge of children’s interactions with physical and virtual products
and TEIs, and in turn to advance the theory of children’s Embodied intuitive
interaction. A significant contribution is the development of an interaction model –
Model for Embodied Intuitive Interactions (MEII) that illustrates children’s
Embodied intuitive interactions with physical and virtual products, and TEIs. Further
significance of this research study is demonstrated by the transferability of the
study’s outcomes to the continua of intuitive interaction, resulting in an Enhanced
Framework for Intuitive Interaction (EFII). The following sub-sections provide an
overview of these key contributions.
1.6.1 Contribution to knowledge
There are three significant contributions of this research study: (i) It contributes
to knowledge on Embodied intuitive interactions in children with tactile interactions
as an interaction modality; (ii) It provides an understanding of the role of
Embodiment in facilitating children’s intuitive interaction; and (iii) It provides a
methodology for investigating children’s Embodied intuitive interaction.
(i) Knowledge on Embodied intuitive interactions in children with tactile
interactions as an interaction modality.
This research strengthens the understanding of aspects of Embodiment in
children’s intuitive interaction. In particular, it investigates Embodiment in
relation to tactile interactions as an interaction modality. The research study
contributes new insight into Embodied intuitive interaction, using tactile
interactions with physical products, virtual interfaces, and TEIs. This is a
significant step towards incorporating Embodiment in the design of
products on the physical-virtual continuum, as it is the nature of the tactile
interactions with products and interfaces that contributes to intuitive
interaction. Previous research mostly focussed on full body interactions and
Embodied metaphors as facilitators of children’s intuitive interaction.
(ii) Understanding of the role of Embodiment in facilitating children’s
intuitive interaction
The results of this study contribute to an Enhanced Framework for
Intuitive Interaction (EFII) (Blackler, Desai, McEwan, Dieffenbach, &
Popovic, 2018). This framework could lead to new directions in research
Embodied intuitive interaction in children 25
into intuitive interaction, such as intuitive experiences, and the factors
responsible for these experiences (e.g. transfer distance, indirection, and
ubiquity).
The transferability of the results to Blackler & Hurtienne’s (2007) continua
of intuitive interaction suggests that the findings could also be applicable
to adults. This is an important contribution because Embodied intuitive
interaction is conventionally associated with children as sensorimotor
knowledge is considered to be more accessible to children than to adults
(Brandenburg & Sachse, 2012). The applicability of the EFII to adults
could lead to future research that investigates Embodied intuitive
interaction in adults.
The EFII allows discussion of intuitive interaction with products on the
physical-virtual continuum: physical products, virtual interfaces, and TEIs.
The findings from the research study concur with Blackler's (2008)
findings that physical affordances should be incorporated in design
whenever possible. Where physical affordances are not possible, perceived
affordances should be used. The study has further suggested in form of
recommendations that physical and perceived affordances could be used
together, sequential in time, or/and nested in space. This study thus has
implications for design, and provides design recommendations for
Embodied intuitive interactions with physical products, virtual interfaces,
and TEIs for children.
This study further highlights the role of cooperative activity and
scaffolding as Embodied aspects in the design of children’s products.
Appropriate scaffolds can assist children in offloading cognitive activity
onto the environment around them while carrying out perceptual motor
activities. Emergence is an aspect of design that is representative of
dynamic processes in systems where interactions, behaviours, and
environments evolve over time. This study has suggested that dynamic
processes are conducive to Embodied intuitive interactions as they
facilitate the updating of existing knowledge and the generation of new
knowledge; in other words, learning of new concepts. This contribution is
26 Embodied intuitive interaction in children
significant because it provides guidelines for the design of Embodied
intuitive products (physical, virtual, and TEIs) for children.
(iii) Methodology to investigate children’s Embodied intuitive interaction
This research contributes a novel methodology for the study of aspects of
Embodiment and children’s intuitive interaction. The context of play and
co-discovery between two children in an observational setting allowed data
to be collected, with little disruption to the children’s cognitive processes.
Retrospective interviews (performed after observations) provided
clarification and a better understanding of the use of aspects of
Embodiment in intuitive interaction. This methodological contribution is
significant as it can be transferred to other application domains involving
children, such as spatial interactions.
1.6.2 Research Outcome
The findings from this research study have contributed to the model for
Embodied Intuitive Interaction (MEII)—an interaction model that represents
children’s behaviour with artefacts and the social world. It illustrates that children are
distributed anticipatory systems; that is, they use their sensory perceptual systems,
sensorimotor knowledge, and Embodied experiences to interact with artefacts and the
social world. They also use physical affordances, perceived affordances, and
scaffolding in cooperation with other children to perceive the clues in the social
world and the artefacts, to decide on the actions to be performed.
Having decided on these actions, they use scaffolding in cooperation with other
children to perform the actions on artefacts and the social world. While these actions
are being performed, however, the properties of the artefacts and the social world
change. Thus, over multiple perception and action cycles over time, the artefacts,
social world, and children’s interactions evolve, resulting in ‘emergence’. This
dynamic nature of Embodied intuitive interaction is important for children’s
development of new knowledge, and for the updating of their existing knowledge
and learning. This is a significant contribution of the study as it offers insights into
how children interact with their physical and social world. MEII (Figure 29 and
Figure 30) could be applied in various contexts relating to children, such as
designing interactions for children. This model will allow designers to evaluate and
Embodied intuitive interaction in children 27
design interactions for Embodied intuitiveness, and will inform the further design
and development of tools to design Embodied intuitive products for children.
1.7 THESIS OVERVIEW
Chapter 2 reviews the literature on intuitive interaction, and on the nature and
processes of Embodiment in the context of cognition and interaction design. Chapter
3 discusses Embodied intuitive interaction in the context of children as user groups.
Chapter 4 addresses the first part of the research question, ‘What are the
aspects of Embodiment?’ It discusses the aspects of Embodiment in cognitive
science and derives design aspects of Embodiment which were then used in
Experiment 1 and Experiment 2 to investigate Embodied intuitive interaction in
children.
Chapter 5 describes the research design and methodology, and justifies the
methodology according to the needs of the research question and the available
methods. The two experiments are briefly explained. Heuristics for intuitive
interaction, and design aspects of Embodiment derived from the respective literature,
are explained.
Chapter 6 and Chapter 7 describe the two empirical experiments, Experiment 1
and Experiment 2, respectively. The results are described and discussed from the
perspective of implications to design. Chapter 8 discusses the results of Experiment 1
and Experiment 2 in reference to the continuum of intuitive interaction. The
contribution of this study to the Enhanced Framework of Intuitive Interaction (EFII)
(Blackler et al., 2018) (Figure 27) is discussed. The outcome of this study is The
Model of Embodied Intuitive Interaction (MEII) which is discussed in Chapter 8.
Chapter 9 highlights the contributions of the study, and ends with a statement of the
future scope of the research.
1.8 SUMMARY
Chapter 1 has introduced the research study with a brief background on the role
that Embodiment and intuitive interaction could play in children’s interaction with
complex products. Key concepts and terms relevant to this study were defined and
explained. A physical virtual continuum was used to discuss the range of products
that children use, and how they vary in their physicality and virtuality. Interaction
28 Embodied intuitive interaction in children
modalities that form the basis of children’s interaction with products were discussed.
A discussion on the research problem identified research gaps leading to the research
questions. Aims and objectives of the study were defined. Finally, the chapter
outlined the contributions made by this research study to new knowledge, and the
outcomes of the research were discussed.
Chapter 2 now reviews the literature on intuitive interaction and the nature and
processes of Embodiment, in the context of cognition and interaction design.
Literature on intuition and intuitive interaction is discussed. Different perspectives of
Embodiment based on the role of brain, body and environment in the cognitive
processes of Embodiment are discussed.
Chapter 2: Embodiment and Intuitive Interaction
This thesis brings together two important frameworks in interaction design,
Embodiment and Intuitive Interaction. To this end, this chapter reviews the literature
on: i) Embodiment (Section 2.1)—the alternative to the traditional theory of
cognition, and considers that the brain, body, and the environment work together to
process any stimuli; ii) Embodiment in interaction design (Section 2.2)—interaction
design that is based on the concept of Embodiment; iii) Intuition and intuitive
interaction (Section 2.3)—intuitive interaction in design from the perspective of
cognitive psychology; and, finally, the research on iv) Embodied intuitive
interaction—that discusses the relationship between Embodiment and intuitive
interaction, and the limited research carried out in the field of interaction design.
2.1 EMBODIMENT
The basic idea of Embodiment is that the brain alone should not be treated as
the only place where mental processes occur. The body and its movement, guided by
Embodied intuitive interaction in children 29
perceptual coupling with the environment, facilitate the goal achievement, thus could
eliminate the need for complex internal mental representations. Embodied theories of
cognition present different principles by which aspects of perceptual and motor
processes are tightly coupled, not only to each other, but also to higher order
cognitive processes, including language (Clark, 2008) and mathematics (Barsalou,
Santos, Simmons, & Wilson, 2008).
The theory of Embodiment contradicts the traditional view on cognition that
suggests the separation of mind, body, and environment. The traditional view of
cognition has been discussed in various fields and has been referred to by different
names: Cartesianism or Dualism in cognitive theory (Husserl, 1960/2013); and
Brainbound (Haugeland, 1989) in Artificial Intelligence. Descartes (1911, 1985)
suggested that the mind is the locus of intelligence, and the body is simply a sensing
tool. The concept of Brainbound suggests that the body is just the sensor and effector
system of the brain, and that the rest of the world is the arena in which adaptive
problems are posed, and the brain-body system must sense and act (Haugeland,
1998). Traditional theories of motor development are based on the belief that
movements are the result of commands from the brain, and that all movements result
in actions, and that repetition of movements leads to action learning. Embodiment,
on the other hand, rejects the understanding that intelligence is the most important
feature of the notions of thought and reason.
Researchers in Artificial Intelligence have been investigating whether
machines can be designed to think like human beings. Turing (1950) formulated a
test, popularly known as the ‘Turing test’, to determine whether a machine can do
what humans, as thinking entities, can do. In an attempt to develop machines that
think and act like humans, machines were formulated as entities that manipulate
abstract representations by explicit formal rules. These rules came to be known as
GOFAI (Good Old Fashioned Artificial Intelligence). Searle (1980) was critical of
GOFAI and argued that machines could pass the Turing test simply by manipulating
symbols of which they had no understanding. Machines without appropriate
understanding of those symbols could not be considered to be thinking like humans.
Cartesianism explains functions of mind such as thinking in terms of symbolic
manipulations according to explicit rules (Vogt, 2002). Internal representations,
formal abstractions and rule based transformations are used to manipulate symbols. It
30 Embodied intuitive interaction in children
is the form of symbol and not its meaning that forms the basis of the rule based
transformation (Anderson, 2003) in Cartesian cognition. The relation between sign
and signifier seems arbitrary and this creates a distance between the inner symbol
processing and the external world of meaning and action (Harnad, 2003). Rule based
transformation outlines specific rules for thinking. Having disconnected the form of
symbol from its meaning, Cartesianism rules out the possibility of content sensitive
processing, and so requires formal rules to govern the transformation from one
cognitive state to another (Clark, 2012). Physical Grounding is about how abstract
representations can acquire real-world meaning and centrally involves understanding
of how cognitive contents (however these are ultimately characterised, symbolically
or otherwise) must ultimately ground out in terms of the agent’s embodied
experience and physical characteristics (Gallese & Lakoff, 2005; Harnad, 2003;
Vogt, 2002).
Piaget (1955/2013) investigated how, and when, children develop an
understanding and knowledge of objects and properties. He asked children of various
ages to look for a toy hidden behind an obstacle. The toy was hidden in view of the
children, so they knew where it was hidden. Piaget found that children 7 months old
and under, however, did not look for the toy, as if it did not exist. Children 12
months and over retrieved the toy, having developed an understanding that although
they could not see it, it did exist. When Piaget changed the location of the toy after
the children had successfully retrieved it several times from the first location, he
found that children reached out for the toy in the first location rather than the second.
Piaget referred to this as A-not-B error, A and B referring to the first and the second
locations.
The older children had developed an understanding that objects persist even
when they move out of view and eyesight. However, the action of reaching for the
toy was unable to use that knowledge (Piaget, 1955/2013). Baillargeon & Graber
(1988) related the error to children’s competence and performance, suggesting that
they had not developed the competence necessary to access the acquired knowledge
to reach for the toy. Thelen (2008) rejected the performance and competence link,
and explained the A-not-B error using an Embodied dynamic systems model that
they had developed through a series of experiments with children. It is the dynamics
of perception and action over time that allows children to develop an understanding
Embodied intuitive interaction in children 31
of the reaching task. The A-not-B error is due to the fact that children are expected to
remember the hidden toy every time the location is changed. However, they tend to
remember the last location where it was hidden, even though they see the toy being
hidden in a different location. Thelen (2008, p. 4) explains: “The A-not-B error is not
about what infants have and don’t have as enduring concepts, traits, or deficits, but
what they are doing and have done”.
Another popular study in the field of Embodied cognition has been to
investigate how a baseball outfielder catches a fly ball. Saxberg (1987) suggested
that outfielders perceive the parameters that determine the flight of the fly ball, such
as initial direction, velocity and angle, along with other constants such as drag, air
density, and gravity. They then use these parameters as inputs to the internal
representations of inflight motion. Thus, the outfielders are then able to estimate the
trajectory of the fly ball. Cutting & Vishton (1995) challenged this point of view,
indicating that the parameters determined by the outfielder would be erroneous, as
the changes in optical projection size of the ball is not an appropriate resource to be
used to determine where it is going and how far away it is. Bingham (1995) divided
the whole process—from the point when the ball is hit to the point when it is caught
or dropped by the outfielder—into events that unfold over time. Underlying
dynamics then provide a better understanding of the changes in the events, such as
how the changes have occurred, and what forces have caused those changes.
Dynamics of inflight motion generate kinematic information that the outfielder uses
to reach a certain location at a certain speed.
Researchers have seen the advantages of studying Embodiment in animals
(Clark, 2005; Dautenhahn, 1996). The idea is to establish that perception-action
couplings (Section 1.1) produce complex adaptive behaviour in animals, and to then
develop explanations for similar solutions and behaviours in humankind.
Researchers, for example, have studied coordinated activities in groups of animals
(Barsalou, Niedenthal, Barbey, & Ruppert, 2003). Some animals move in groups to
defend against predators. The sustainment of these groups requires coordination and
management, and this is usually not centrally controlled. There is no obvious
intention to control or coordinate the group. However, the coordination emerges from
perception-action coupling rules specific to a context. Muro, Escobedo, Spector, &
Coppinger (2011) studied coordination in wolves hunting in packs. They found that
32 Embodied intuitive interaction in children
the wolves moved towards the prey until a minimum safe distance was reached, and
then moved away from any other wolves that were also close to the prey. The entire
coordination was neither planned nor instructed by anyone, but emerged from a
perception-action coupling strategy implemented by each of the wolves in a specific
context.
The traditional approach in cognition and Artificial Intelligence has been to
rely on representations in the mind and machines to process information captured
from the environment to make subsequent decisions. However, immediate actions in
the world without internal representations are evident in many real world complex
scenarios. For example, a typical mother shuttles her children to their various
activities at the right time, in the right order, meanwhile figuring out ways to do the
laundry and grocery shopping, and pick up the dry cleaning. Similar complexity is
evident in daily conversations with others, where we need to remember what has
been said, decide what to say next in light of the overall conversational goals, correct
misapprehensions, and reinterpret past interactions in light of the corrections and,
thereby, sometimes significantly alter the current state of the dialogue (Norman,
2013; Vera & Simon, 1993).
Internal representations, however, need not be completely rejected. While the
juggling mother needs representations and symbols, she cannot rely on symbols
alone. Her representations must be highly selective, related to the context, and
physically grounded. This strongly suggests that her inherent mental and physical
powers of representation should be linked to, and constrained by those which govern
her moving and acting in a dynamic environment. Thus, the execution of complex
tasks requires both reactive and representational powers.
2.1.1 Perspectives of Embodiment
The meaning of Embodiment is not as straightforward as suggested in some
literature, “states of the body modify the states of the mind” (Wilson & Golonka,
2013, p. 1). There is lack of clarity in the claims that have been made in literature on
Embodiment. For example, Dennett (1991) argued that literature on Embodied
cognition does not discuss the influence of narratives of the past which could
influence the intentions of the future. But from Strawson's (1999) view, memories
and experiences of an individual contribute to internal representations which is
rejected by theory of Embodied cognition. Then, there are researchers who believe
Embodied intuitive interaction in children 33
that it is the intentions to meet the future goals that drive individuals to use their
memories and narratives in their present to derive information from the environment
(Giummarra, Gibson, Georgiou-Karistianis, & Bradshaw, 2008; Rapp & Kurby,
2008). Moreover, there are multiple perspectives of Embodiment based on where
cognition originates from and how are cognitive processes distributed between brain,
body and environment.
Goldman & de Vignemont (2009) suggested a minimal interpretation of
Embodiment, referred as ‘Minimalist Embodiment’, where brain, body and the
environment participate in the cognitive processes, but brain is still considered to
control all cognitive processes. They suggested that all cognitive processes originate
in the brain and the body (without the brain) just supports and augments these
processes. Chiel & Beer (1997), Shapiro (2004) on the other hand suggested that
anatomy and movement of body structures shapes cognition before and after the
brain processes the information. Anatomy and movement of body structures result in
perceptual processes which could in turn result in body movements. It could thus be
appropriate to say that cognition originates in the body, the structure, composition,
and motor abilities of which determine what humans experience in the environment.
Johnson (2010) further added to the theory of cognition originating from the
body structures, suggesting that these body structures determine what people
experience, and how they understand the world. Reasoning with abstract concepts
requires mental simulations based on concrete motor-perceptual experiences
(Barsalou, 2010). Humans think, generate ideas, and act through these experiences.
The experiences offer structure to what people perceive, how they get around in the
world, how they relate to other people, and eventually result in conceptual systems
that define their everyday activities. Lakoff & Johnson (1980/2003) suggested that
the motor-perceptual experience makes the conceptual systems metaphorical in
nature. The metaphorical nature of the conceptual systems, however, is Embodied in
nature.
Language or linguistic systems are one such conceptual system that Lakoff &
Johnson (1980/2003) used to explain the Embodied metaphorical nature of
conceptual systems. According to them, semantics arose from the nature of the body,
in the form of Embodied metaphors. The most common types of metaphors are
structural metaphors, which represent abstract complex experiences with simple,
34 Embodied intuitive interaction in children
familiar, and known experiences, as seen in the phrase ‘defend an argument’.
Orientational metaphors are mostly spatial, as they arise from the orientations of the
body and its function in the physical environment. An example of such metaphors is
the concept of happiness and sadness, and the conceptual metaphor of ‘Feeling UP or
DOWN’. Happiness is associated with being physically at a high point, while
sadness is associated with being low down, at the bottom. An ontological metaphor
represents an abstract concept as a concrete physical element such as an object,
person, space or substance (Lakoff & Johnson, 1980). For example, in the metaphor
‘Joy filled the room’, ‘joy’ is an abstract emotion that is represented as a physical
object or substance that can fill a container. This allows people to link their abstract
experiences to physical entities that can then be categorised qualitatively or
quantitatively. The essence of metaphor is experiencing one kind of concept in terms
of another, and both can be completely unrelated.
Wundt (1922/1973) referred to gestures that transfer concepts from one domain
to another as ‘symbolic gestures’, and offered the use of spatial gestures to represent
temporal concepts as an example. Cienki & Müller (2008) described the
metaphorical nature of symbolic gestures through an example of the statue of Lenin
pointing toward the ‘bright future’ of communism. Pointing ahead in space is used to
indicate that an abstract object, ‘the future’, is situated ahead in time. Space is used
to represent time, and the pointing gesture that uses space to refer to time is
considered to be a metaphoric gesture.
Another perspective of Embodiment is that of functionalism, referred as
‘Embodied functionalism’, where cognition is considered to be originating in the
brain but then extending to the body and then to the environment (Clark, 2008, p.38).
The body acts as a vehicle for cognitive processes, suggesting that the body plays
only a functional role in Embodiment. Menary (2010) sees an overlap between
‘Embodied functionalism’ and ‘Minimalist Embodiment’ and suggests that cognitive
processes and brain extend beyond the body to include elements of environment in
which an individual is embedded and the interactions of an individual with the
environment. Menary (2010) refers to this interpretation of Embodied cognition as
‘Extended Cognition’.
The enactive perspective of Embodiment, similar to Embodied functionalism,
suggests that cognitive processes are distributed across brain, body and the
Embodied intuitive interaction in children 35
environment. However, cognitive processes originate in the bodily structures through
perceptual sensory systems in the body and are distributed to the brain and the
environment. The body structures include movements and actions and anatomy, but
also complex behavioural and emotive processes such as facial expressions, postures,
gestures, and so on (Noë, 2004). The enactive perspective has overlaps with the
perspective of Chiel & Beer (1997) and Shapiro (2004) which also suggests that
Embodiment originates in the bodily structures but the way the body structures are
defined in the Enactive perspective is the topic of ongoing research.
The perspective of Embodiment used in this study is based on Gibson's (1979)
theory that organisms are able to directly perceive things around them; thus,
perception, and by extension the environment, is a useful resource in cognition.
Thus, the perspective used in this study is that cognition originates from the
environment and through a continuous cycle of perception action loops (Section 1.1)
is distributed through the body and brain (Anderson, 2003; Kirsh, 2013a, 2013b;
Wilson & Golonka, 2013; Wilson, 2002). The brain is no longer considered the
centre of all behavioural and cognitive processes. It enhances and augments the
actions and activities of the body. The individual deciphers the environment,
interprets the actions to be performed on the body and the environment, which is
again followed by perception of the environment. This continuous loop of perception
action is characteristic of a dynamic anticipatory system. Cognition involves
processes where perception contributes to most part of these processes (Barrett,
2011). Once the perception action loop is initiated, it is difficult to distinguish when
perception stops and cognition starts and vice versa (Barrett, 2011, p. 55). Shapiro
(2011) suggested that the need for the objects and processes of traditional cognition
(concepts, internally represented competence, and knowledge) is replaced by
processes that rely on dynamic systems of these perception action couplings. Shapiro
(2011) referred to this as the ‘replacement hypothesis’.
Knowledge has been typically considered to be some form of internal
representational state (Rasmussen, 1985). However, Embodied Cognition theory
describes knowledge as an entity derived from an individual’s perceptual coupling
with the environment and the body. This perceptual coupling is achieved through
extension of mental processes beyond neurological processes into the parts of the
world that are of everyday use (Clark, 2001). This extension of mental processes is
36 Embodied intuitive interaction in children
seen in everyday activities such as navigating a city using a map. Although the brain
might have information about the city, people use tools such as maps (that are parts
of the world) to find their way, or at the least to check if they are on the right path.
These activities usually result in a decision such as the direction or route to follow.
The transitions between the mind and the tools, and ultimately to the decisions taken,
are seamless.
Payne (1991) examined how information flows from device to users during
their interaction with keyboard based displays and word processors. He found that
users often did not know the effects of the frequently used actions such as where will
the position of the cursor be. Thus, users need to acquire information from the
device’s display (which is the environment in this context) using their perceptual
sensory mechanisms.
Embodiment and Embodied cognition theory laid the foundation of this study
on Embodied intuitive interaction, literature on which was introduced in Section 2.4
and discussed in the context of children in Section 3.4. Cognitive aspects of
Embodiment are further discussed in Section 4.1. These cognitive aspects of
Embodiment are based on the perspective of Embodiment that perception action
loops originate from the environment as information on invariants is derived from it.
Design aspects of Embodiment were derived from the cognitive aspects which were
then used to study Embodied intuitive interaction in children playing with toys.
2.2 EMBODIMENT IN INTERACTION DESIGN
Embodiment is the property of being manifest in, and a part of the world, and
of being grounded and situated in everyday rituals and activities (Dourish, 2001).
This state creates an engaged interaction with the world. Dourish’s concept of
Embodiment has been universally interpreted as an approach that leverages an
individual’s body movements, in interaction with physical objects and spaces, to
control computing systems.
In HCI design, the rise of tangible, haptic, ubiquitous and pervasive computing
has blurred the boundaries between bodily interaction and digital information.
Researchers such as Dourish (2001), Ishii (2007), Shaer & Hornecker (2010),
Sharlin, Watson, Kitamura, Kishino, & Itoh (2004) have referred to these
manipulations as ‘Embodied and tangible interactions’. Interaction design has
Embodied intuitive interaction in children 37
explored Embodiment in terms of interaction modalities such as graspable user
interfaces (Fitzmaurice, Ishii, & Buxton, 1995), frameworks and taxonomies to
design Embodied interactive products such as the framework for reality based
interactions (Jacob et al., 2008) and paradigms to design TEIs (Hornecker & Buur,
2006; Shaer & Jacob, 2009).
Shaer & Jacob (2009) proposed a specification paradigm for designing
Tangible Embodied and Embedded Interfaces (TEIs) that allows designers to specify
behaviour of TEIs without getting lost in the implementation details. TEIs offer an
explicit way of exploiting Embodiment through digital technology. Marshall, Price,
& Rogers (2003) discussed the role of TEIs in learning, and suggested that learning
through TEIs happens through two types of interactions: expressive and exploratory
(Marshall et al., 2003). Expressive interactions focus on external representations of
activities (Mellar & Bliss, 1994) such as model making and prototyping using clay,
or building using LEGO Mindstorms. Exploratory interactions, on the other hand, are
used to ‘explore’ a model that someone else has created, either by practical
manipulation or theoretical reflection. Chalmers (2001) argued that Dourish's
treatment of Embodied and tangible interaction with TEIs neglects expressive
interactions, and overemphasises exploratory interactions.
Although Marshall (2003) associated expressive and exploratory interactions
with TEIs, it is the physicality and materiality of the objects that allow these
interactions. Technology is a facilitator of these interactions, and is used to embed
and process digital information within objects. Mellar & Bliss (1994) explain
expressive behaviour in the context of learning by making models that are not
necessarily embedded with technology. Advances in technology now allow
computing elements to be embedded in objects so that the technology itself simply
disappears behind a façade. This has resulted in new interfaces, known as ‘embedded
interfaces’ (Gervais et al., 2016). These interfaces retain the richness and situated-
ness of physical interaction, while simultaneously embedding computing in existing
environments.
Material interaction and engagement with physical objects involves the mind,
body, and perceptual systems acting together (Malafouris, 2013). Traditional design
requires deep knowledge about, and sensibility for materials. The designer sees the
potential of this knowledge and sensibility, and uses it in the act of design. Design
38 Embodied intuitive interaction in children
then becomes a negotiation between form and function, and between aesthetics and
utility. The incorporation of computing elements in products (TEIs) offers an
additional dimension for these negotiations. Attempts are being made to re-imagine
computing as another material in a design toolkit in line with other materials such as
paper and cardboard. The result is the activation of the existing properties of
materials to create opportunities for richer interactions and experiences.
People demonstrate intuitive spatial relationships with objects, as the skill of
manipulation comes naturally to them (Zigelbaum, Kumpf, Vazquez, & Ishii, 2008).
Users determine the inherent functionality of objects from their physical and spatial
qualities such as shape, weight, size, and colour. Physical analogies and cultural
standards pertaining to the objects develop a sense of familiarity with the spatial
qualities of the objects. People use this familiarity to map spatial qualities to new
functions and tasks, resulting in new spatial mappings (Sharlin et al., 2004).
Actions performed on objects should be closely coupled to perception space
(i.e. the view and weight of the object), as this allows one to direct attention at one
place and time (Zigelbaum et al., 2008). For example, the action space of a computer
mouse is separate to its perception space (display screen) (Beaudouin-Lafon, 2000).
This results in the users having to divide their attention between the mouse and the
screen. In the physical world, humans use tools such as their hands to perform
actions on the objects in the same space and time (Dreyfus & Dreyfus, 2000). They
use their sensory cues and the condition and motion of the tools to track the progress
of their activity. The close coupling between action and perception space can be
achieved by unifying the input and output devices, for example, in tablets (Ullmer &
Ishii, 2000; Zigelbaum et al., 2008).
Thus, the concept of Embodiment has found its way through into the field of
interaction design. The primary objective has been to allow people to interact with
computing systems but maintain the traditional and natural interaction modalities to
allow users to engage with the systems. This study has investigated Embodiment
from the perspective of intuitive interaction design. The study has first investigated
Embodiment through the lens of cognitive science to determine design aspects of
Embodiment (discussed in Section 4.2). Experiment 1 then compared a physical
Jenga (Hasbro & Scott, 2001) and a virtual Jenga (Natural Motions & Scott, 2011) to
investigate intuitive interaction in children. The extent each of the design aspects of
Embodied intuitive interaction in children 39
Embodiment facilitates intuitive interaction in children playing with physical Jenga
and virtual Jenga was determined (Chapter 6). Experiment 2 investigated a physical
product and Tangible Embodied and Embedded Interface (TEI) to determine the
variability of intuitive interaction with respect to the design aspects of Embodiment
(Chapter 7).
2.3 INTUITION AND INTUITIVE INTERACTION
2.3.1 Intuition
The word ‘intuition’ has been liberally used in everyday life. Terms such as
‘gut feeling’, ‘hunch’, ‘sixth sense’, ‘second sight’, and so forth are interchangeably
used with intuition. People use these terms when they are unable to explain their
decisions, experiences, and actions. This happens because they have encountered
something similar in the past. Intuitive thinking is fast, unconscious, and often
automatic (Kahneman & Klein, 2009; Klein, 2003).This differs from rational
thinking that is slow, conscious, and deliberate. Research has suggested that most of
our thinking occurs outside of our consciousness (Bastick, 2003; Dreyfus & Dreyfus,
2000). Intuitive thinking is the cognitive process of using information previously
perceived by the senses (Bastick, 2003). This sensory information is used to make
insights, recognitions, and judgements (Harteis & Billett, 2013).
Intuition is a non-unitary construct with two systems—rational thinking and
intuitive thinking systems—underlying human thinking and reasoning (Myers, 2007;
Schon, 1982). Both the systems process the stimuli from the environment, ultimately
resulting in a decisive outcome. However, the two systems differ in the way the
stimuli are processed. Rational thinking is slow, deliberate, and conscious, and
makes the person feel as if s/he is in control of the decision making process. It
follows set rules and guidelines (heuristics) in coming up with a decision. Intuitive
thinking, on the other hand is fast, automatic, and non-conscious. It is capable of
learning by associating new stimuli with known stimuli in terms of causality,
contiguity in time and place, and shared characteristics. Intuitive thinking is capable
of reading the environment quickly, enabling a quick response in return.
Intuitive thinking constantly feeds suggestions to rational thinking that can
override these suggestions; however, rational thinking is often unaware that intuitive
thinking is influencing it. Intuitive thinking operates well in instantaneous situations,
40 Embodied intuitive interaction in children
and is less effective in long term planning. When intuitive thinking is met with a
situation that it cannot handle, it automatically invokes rational thinking; for
example, when an individual is met with a dangerous obstacle while walking,
rational thinking is triggered in order to analyse the danger. Thus, the decision
making process involves a continuous handover of controls between the two thinking
and reasoning systems; however, neither of the systems is aware of these handovers.
Intuition is a cognitive process, and builds upon past experiences (Bastick,
2003; Klein, 2003). People use their senses and past experience to recognise patterns
in a given situation. They then link these patterns to a series of past incidents
(Bastick, 2003). These traces of past incidents are derived from an individual’s
implicit memory (Schore, 2010). This probably is the reason people are not
conscious of their intuitive thinking, or of deriving knowledge from their past
experiences. However, Klein’s study was limited to experienced people. He
suggested that experienced people make intuitive decisions in real situations by
adding up experiences from various incidents (Zsambok & Klein, 1997).
Wickens, Gordon, Liu, & Lee (1998) used Rasmussen's (1993) Skill, Rule and
Knowledge- (SRK) based model of decision making to explain the use of experience
in intuitive decision making. They referred to the latter as ‘naturalistic decision
making’. Skill, Rule and Knowledge are the three levels of cognitive activity that
people operate in during task performance and decision making. The nature of the
task, and the level of experience with the task, decides the level of operation.
Highly experienced people operate at the skill-based level. Their performance
and decision making is at the subconscious level, and is often an automatic response
to a particular situation. People familiar with the task, but who do not have enough
experience, operate at the rule-based level. They look for cues or rules that they
recognise from past experience to make a decision. When the task is novel, and
people do not have any rules or cues to rely on from their past experiences, they
resort to analytical processing, using conceptual information. Problems are defined,
solutions generated, and the best course of action is determined before making a
decision. Thus, people operating at skill- and rule-based levels, people who are
highly experienced in a given task, and people who are familiar with the task but do
not have enough experience, are making decisions intuitively (Wickens et al., 1998,
Embodied intuitive interaction in children 41
p.198). On the other hand, people new to a given task could resort to non-intuitive
decision making.
Another explanation of how people use their past experience in intuition was
provided by Gore & Sadler-Smith (2011). They conceptualised intuition into
processes and mechanisms of intuiting, and the outcomes of these processes. They
suggested that although most of the research focussed on the outcomes of intuition, it
is the process of intuiting that uses past experience in thinking, reasoning, planning,
and decision making. The processes of intuiting are considered to be either domain-
general or domain-specific, depending on the similarity of the domain in which a
task is undertaken to the domain from which knowledge is derived. Domain-general
processes operate automatically across domains on the basis of the complexity,
uncertainty, and level of risk associated with a triggering stimulus. Domain-general
processes of intuiting include applying heuristics under uncertain conditions,
acquiring and activating domain-relevant schemas, and infusing affect into decision
making (Dane & Pratt, 2007). Domain-specific processes, on the other hand, are
specific to particular domains. They are activated autonomously based on learning,
schemas, and frequently encountered domains in which the task is located.
Intuitive judgements in information processing are a result of the application of
heuristics (Kahneman & Klein, 2009, p238). Slovic, Finucane, Peters, & MacGregor
(2007) suggested that the types of heuristics that contribute to intuitive judgements
are affective in nature. They empirically studied the role of affect in assessing
dangers and merits of hazards such as nuclear power. They found that people relied
on affect under time-pressured conditions. People were able to efficiently evaluate
risks and benefits using their ‘gut feel’ reactions. Intuiting involves drawing on an
affect pool of positive and negative affective tags associated with conscious or
unconscious representations (Bastick, 2003; Finucane, Alhakami, Slovic, & Johnson,
2000).
Domain-relevant schemas are activated under time-pressured situations to carry
out intuitive judgements (Klein, 2003). Physiological and emotional signals from the
body are perceived as warning or attention signals in intuitive decision-making
(Dunn et al., 2010). Domain-relevant schemas comprise procedural and declarative
knowledge acquired through learning and practice. Development of domain-relevant
schemas requires individuals to undergo repetitive practice under constant expert
42 Embodied intuitive interaction in children
feedback in controlled environments. Affect infusion is a process in which
“affectively loaded information exerts an influence on, and becomes incorporated
into, cognitive and judgmental processes” (Forgas, 1995, p. 101).
Intuiting or intuitive processes are fast, unconscious, characterised by
unconscious processing, and based on past experience (Blackler et al., 2010). In
terms of correctness of intuition, Blackler et al. (2010) suggested that the correctness
of intuition depends on the acquisition of experience relevant to the task. This means
that domain-specific processes have more chances of resulting in correct intuitions
than domain-general processes. Intuition and intuitive processes have been studied in
the field of interaction design to develop instruments, continua and frameworks for
the design and development of intuitive products (Blackler, 2008; Fischer, Itoh, &
Inagaki, 2015; Hespanhol & Tomitsch, 2015; Hurtienne, 2007; Macaranas et al.,
2015; Mihajlov, Law, & Springett, 2015). These frameworks, continua, and
instruments (discussed in Section 2.3.2) were studied to understand Embodiment in
the context of intuitive interaction for this study.
2.3.2 Intuitive Interaction
The term ‘intuitive interaction’ has been inconsistently used in Human Product
Interaction to characterise a product and its use. Ullrich & Diefenbach (2010)
suggested that a device can be made intuitive to use by designing it so that its
operation can be learned simply through its observation. This is possible through the
appropriate use of deliberate clues in the product that trigger past experiences and
prior knowledge. Ullrich & Diefenbach’s perspective on intuitive interaction is that
‘intuitiveness’ is an attribute of a product that drives an automatic and unconscious
use in individuals.
Intuitiveness is used as an attribute to describe user interfaces, and as an
assessment criterion for technical systems or user interface requirements. While
systems, products, and interfaces are referred to as intuitive, they are inanimate and
cannot undergo the processes of intuiting described in Section 2.3.1. Cognitive
psychology, and decision making and planning research have associated intuition
with human beings who are animate elements. Systems, products, and interfaces can
be designed to support people to interact intuitively, and this is what is referred to as
‘intuitive interaction’ and ‘intuitive interfaces’ (Antle, Corness, & Droumeva, 2009).
Embodied intuitive interaction in children 43
Early intuitive interaction research conducted by two research groups in
Australia and Germany focused on the types of experiential knowledge accessed by
people during intuitive interaction. They also discussed how products could be
designed for maximum intuitive interaction, and designed continua and models of
intuitive interaction. In several experiments, Blackler (2008) studied people using
complex products, and showed that they used knowledge gained from their
experiences with using other products to intuitively interact with interfaces. Using
the Technology Familiarity (TF) Questionnaire to determine a TF score for each
participant, she linked their prior experience to familiarity. She concluded that people
with higher TF scores complete tasks faster and more efficiently than those with
lower scores. Prior experience and familiarity with the elements of the tasks help
people to develop rules to perform given tasks (Rasmussen, 1993). They rely on
these rules to complete operations on the product. McEwan, Blackler, Johnson, &
Wyeth (2014) and O’Brien (2010) adapted the TF questionnaire and TF score and
subsequently used them in their study.
O’Brien (2010) studied interactions of younger adults and older adults with
everyday technologies such as an alarm clock, a Kindle (Amazon & Foxconn, 2007),
and a Flip camcorder (Pure Digital Technologies & Cisco Systems, 2006). Each
participant’s familiarity with each feature of the technology was evaluated using the
TF questionnaire and TF score. O’Brien found that prior knowledge of similar
technologies helped both younger and older adults to successfully interact with the
new technologies. However, there was a difference in performance levels between
younger and older adults across different technologies. Younger adults performed
better with the Flip camcorder in comparison to older adults, who had similar
technical experience but less frequent and recent experience with digital camcorders.
On the other hand, half of the older adults with low technology experience completed
each task on the Kindle, despite their low levels of technical knowledge and low
familiarity with other similar technologies. O’Brien suggested that older adults with
high technology experience could be using their recent experience with video players
to interact with features of the Flip camcorder. Thus, the user’s prior experience with
the technology, as well as their prior knowledge of similar (but not identical) devices,
should be understood.
44 Embodied intuitive interaction in children
McEwan et al. (2014) used the Game Technology Familiarity (GTF)
questionnaire to measure previous gaming experience of participants in a research
study that investigated the potential of different types of Naturally Mapped Control
Interfaces (NMCIs) for intuitive interaction in video games. He observed participants
playing racing video games with three types of NMCIs: directional, incomplete
tangible, and realistic tangible. Four types of GTF scores were evaluated, one each
for familiarity with the three types of NMCIs, and one related to previous play with
racing games. McEwan et al. (2014) found that naturally mapped devices provide
great potential for intuitive interaction for people with less gaming experience using
NMCIs, and/or high familiarity with real life activity. High familiarity gamers
demonstrated intuitive interactions with naturally mapped, as well as less naturally
mapped, controllers.
Blackler (2008) developed a continuum of intuitive interaction. It is shown in
Figure 2, bottom as it relates to the Intuitive Use of User Interfaces (IUUI) research
group’s continuum, shown in Figure 2, top. It includes factors that can be used to
facilitate intuitive interaction, from the simplest and most ubiquitous (physical
affordances) to the more complex (metaphors from another domain). It is universally
accepted in research and industry that products must be intuitive to use. However,
there is a lack of consensus in industry and marketing on what is meant by
intuitiveness, and how it should be measured. On the other hand, researchers do
agree that intuitive interaction is a subconscious application of prior knowledge that
leads to effective interaction (Blackler & Hurtienne, 2007). Blackler (2008)
operationalised conscious reasoning through reportability and verbalisability. People
interacting intuitively did not verbalise to report their actions and reasoning. Blackler
(2008) also used expectations, degree of confidence, latency, and evidence of past
experience and prior knowledge as measures of intuitive interaction.
Hurtienne (2009) and Naumann & Hurtienne (2010) investigated the
subjective consequences of intuitive interaction. They suggested effectiveness of
interactions, mental efficiency of the users, and user satisfaction as measures of
intuitive interaction. They developed a questionnaire, QUESI (Questionnaire for the
subjective consequences of intuitive use), that covers all the above measures of
subjective consequences. The questionnaire consists of 14 items, grouped into five
subscales: subjective mental workload, perceived achievement of goals, perceived
Embodied intuitive interaction in children 45
effort of learning, familiarity, and perceived error rate. The score of each subscale is
computed by evaluating the mean of all the responses to the items of that subscale.
The total score of the questionnaire is equal to the mean evaluated across all five
subscales. Hurtienne (2009) and Naumann & Hurtienne (2010) used QUESI to test a
number of products for intuitive interaction, including a game console, a music
player, a website, a mobile phone, and an operating system. They found that products
that users were familiar with scored higher QUESI scores than those that users were
not familiar with. However, they also found that people who were unfamiliar with
the product scored high QUESI scores when they attempted the questionnaire
immediately after using the product.
The German-based Intuitive Use of User Interfaces (IUUI) Research Group
presented a 'continuum of knowledge in intuitive interaction' (shown in Figure 2,
top), with types of experiential knowledge accessed during intuitive interaction based
on their frequency of cognitive encoding and retrieval (Hurtienne & Blessing, 2007).
Figure 2 Comparison of Intuitive Interaction Continua, shown through blue arrows (adapted from Blackler & Hurtienne, 2007)
Blackler and Hurtienne (2007) compared the two continua, and agreed upon
the forms of knowledge that are more accessible to some people than others. In the
Intuitive Use of User Interfaces (IUUI) continuum, the most basic knowledge
possessed by most people is innate knowledge. In Blackler’s continuum of intuitive
interaction, the simplest form of intuitive interaction is the use of physical
affordances that use Embodied knowledge of the world established early in life. The
46 Embodied intuitive interaction in children
physical properties of the artefact decide their possible interactions, such as grasping
and pulling. According to the IUUI’s continuum of knowledge in intuitive
interaction, such activities use sensorimotor knowledge; that is, knowledge derived
from Embodied interactions with the physical world, acquired very early in
childhood. For example, children learn about gravity, and build up concepts of speed,
motion, and kinematics. Image schemas also sit at the sensorimotor level.
The next most accessible form of intuitive interaction in Blackler’s continuum
is population stereotype; this is in line with the culture and sensorimotor levels in the
IUUI’s continuum of knowledge. It includes knowledge that is possessed by most,
but that is limited by societal bounds such as different meanings for hand and body
gestures, and different interpretations of linguistic structures in the same language.
Population stereotypical representations—such as a clockwise movement signifying
an increase—derive largely from the experience of cultural conventions. When
artefacts are designed in conformation with the population stereotypes, decisions are
made faster, and interactions are more likely to be correct and precise.
Expert knowledge is placed at the highest level in the IUUI’s continuum of
knowledge. However, it is slowest in terms of encoding and retrieval of knowledge,
and very few people have it. It is the knowledge held only by those adept at a
particular skill (such as the knowledge a user might apply to using a software
package such as Excel).
Blackler, Popovic, & Mahar (2010) suggested using familiar features from the
same domain (for example, file menus in software packages) to design intuitive
products that require the use of cultural and expert domain knowledge. However, if
suitable familiar features from the same domain are unavailable, designers might
have to use familiar features from another domain (e.g. a ubiquitous power symbol in
computers and mobile phones). Familiar features tend to be perceived affordances,
virtual objects such as icon buttons that invite pushing or clicking because, based on
prior experience with similar products, a user has learned that this is what it is for
(Johnson & Russo, 1981; Norman, 2013). Perceived affordances was, therefore,
placed on the continuum as being equivalent to familiar features (Blackler, 2008).
Finally, if the technology or context of use is completely new, designers can
leverage metaphors to communicate the intended interaction or to explain a
completely new concept or function. Metaphors are grounded in experience (Lakoff
Embodied intuitive interaction in children 47
& Johnson, 1980/2003); that is, analogies are retrieved from past experiences.
Elements of a known situation are mapped to a new situation; for example, concepts
of desktops can be applied to palmtops (Holyoak, 1991; Lakoff & Johnson,
1980/2003). Metaphors can apply across the IUUI continuum from sensorimotor to
expertise (for example, image schemas use metaphorical extensions). Thus, metaphor
in Blackler’s continuum is linked to IUUI’s sensorimotor, culture, and expertise
levels (see Figure 2). Across the sensorimotor, culture, and expertise levels of
knowledge, the IUUI continuum also highlights knowledge about tools.
The IUUI continuum of knowledge has an inherent dimensionality. The
frequency of encoding and retrieval of knowledge increases from the top to the
bottom of the continuum. Then, the further one moves towards the top level of the
continuum, the higher the degree of specialisation of knowledge, and the smaller the
potential number of users with this knowledge. Similarly, Blackler’s continuum has a
progression in ubiquity from left to right: The extreme left of the continuum
represents the most ubiquitous interfaces. These interfaces have features that people
have Embodied experience of; as a result, they can understand how to use them. The
next most ubiquitous interfaces are those designed with population stereotypes in
mind; those with familiar features which users might or might not recognise
depending on their pattern of past experience; and, finally, those using metaphor.
Ullrich & Diefenbach (2010) expressed their concern that, as a selling point in
product usability, the term ‘intuitiveness’ is characterised by features such as ‘easy to
learn’, and ‘natural interaction’. While this tendency has exposed numerous
components of intuitive interaction, it is still not clear what ‘intuitive interaction’
actually means. Ullrich & Diefenbach agreed with Blackler & Hurtienne’s (2007)
reference to intuitive interaction as the subconscious application of prior knowledge,
as it seperates intuitive interaction from usability. They also agreed with the
measures of intuitive interaction suggested by Blackler (2008) and Naumann &
Hurtienne (2010). However, they expressed their concern that the measures would be
more pronounced in familiar products, and less in innovative and new products. To
counter these concerns, Ullrich & Diefenbach (2010) approached intuitive interaction
as processes originating from ‘gut feeling’, rather than from ‘reason’. Accordingly,
they suggested subjective and experiential components of intuitive interaction: gut
feeling and magical experience as components of intuitive interaction. While ‘gut
48 Embodied intuitive interaction in children
feeling’ refers to the process of decision making, ‘magical experience’ refers to the
result of intuitive decision making. In addition to these two components, they agreed
with two parameters of intuitive interaction suggested in the literature: effortlessness,
and verbalisability (Blackler, 2008; Naumann & Hurtienne, 2010).
Intuitive Interaction research has continued to build on these ideas over the past
few years (Blackler & Popovic, 2015). Specifically, it has focussed on applying the
measurements, concepts, and parameters of intuitive interaction to other application
domains. Hespanhol & Tomitsch (2015) investigated strategies to design intuitive
interactive public spaces, and these have been referred to as a third wave of HCI.
Interactive spaces are an emerging field, and Blackler (2008) and Hurtienne,
Klöckner, Diefenbach, Nass, & Maier (2015) pointed out that metaphors and image
schemas that relate to something that is familiar to users would need to be applied to
innovative and inclusive products and technology that users are not familiar with.
Hespanhol & Tomitsch (2015) suggested that intuitiveness in interactive urban
spaces is mostly derived from the interactive experience that is the feedback from the
interactive systems. It is the perceived affordances of the systems that determines the
level of intuitiveness in urban environments. Hespanhol & Tomitsch thus presented
feedback strategies for intuitive interaction in urban interactive spaces: (i) directional
feedback versus scattered feedback; (ii) immediate and concrete feedback versus
delayed and abstract feedback; and (iii) visual feedback versus audio feedback.
Previous research mainly focussed on familiarity, prior experience, and
knowledge as facilitators of intuitive interaction. However, a focus on designing
products with familiar features could inhibit innovation. Different user groups have
different levels of familiarity with products and features, especially with regard to
technology products. Hurtienne et al. (2015) suggested that image schemas and
primary metaphors could bridge the gaps between inclusive, intuitive, and innovative
design. Image schemas are recurring, dynamic patterns of perceptual motor
interactions that give coherence and structure to users’ experience (Johnson, 1987, p.
xiv). Use of gestures or/and body interactions were also suggested as ways to design
innovative products that provide touchless natural interactions (O’hara, Harper,
Mentis, Sellen, & Taylor, 2013). The effectiveness and efficiency of the intuitive
mappings of gestures and body movements in product interactions depends on the
sensorimotor knowledge derived from everyday experiences (Kiverstein, 2010).
Embodied intuitive interaction in children 49
Mihajlov et al. (2015) found that simple touch gestures, such as a drag gesture,
were easily learned and retrieved by older people who had no prior experience with
touch-based interactions, but are familiar with a drag gesture from their real world
experience (e.g. the dragging of objects). However, in contrast, older people found a
rotate gesture difficult to use as it was not something that they used in their everyday
lives.
Macaranas, Antle, & Riecke (2015) highlighted the problems and
inconsistencies associated with gestural interactions across different users. Some
gestural and body interactions could lack physical or perceptual affordances and,
without very detailed instructions, this could make it difficult for users to know what
gestures and actions are supported. Macaranas, Antle, & Riecke (2015) compared
three interaction models for intuitiveness, each with a different approach to mapping
body actions to system controls: (i) metamorphic mappings, (ii) isomorphic
mappings, and (iii) conventional mappings. While, they did not find any statistical
differences between the three in terms of intuitive behaviour, participants
demonstrated different attention ratings for the three mappings. Those using
isomorphic mappings gave more attention to completing the task than to using the
system. Macaranas, Antle, & Riecke (2015) offered guidelines to explain which
application should be associated with each of the mappings.
Over the past few years, intuitive interaction research has continued to build on
ideas and concepts around ways to facilitate intuitive interaction (Blackler &
Popovic, 2016), and has focussed on various application domains (Desai et al., 2015,
2016; Swann, Popovic, Thompson, Blackler, & Kraal, 2015) and on elderly
populations (Lawry et al., 2011; O’Brien, 2010). However, there is limited research
that investigates children’s intuitive interaction. Israel, Hurtienne, Pohlmeyer, Mohs,
Kindsmüller, et al. (2009) studied aspects of intuitive interaction and classified these
aspects into effects, features, enablers, and facilitators. This classification offered a
perspective on how intuitive interfaces could be designed. They suggested image
schemas and affordances (among others) as enablers of intuitive interaction, and
noted physicality, familiarity, and Embodiment as the major facilitators. However,
they did not consider children as users in their experiments.
Brandenburg & Sachse (2012) did include children in their study investigating
the role of prior knowledge on intuitive interaction. The objective of their study was
50 Embodied intuitive interaction in children
to compare the intuitive behaviour of people with different levels of prior experience.
Children in this study represented user groups who had sensorimotor knowledge, but
no prior experience (referred to as ‘naïve users’ in the study). They tested a multi-
touch interface with children, adults without specific operating knowledge of
interacting with multi-touch interfaces, and adults with prior experience in using
smartphones and tablets. They asked participants to manipulate three objects shown
on the interface by using three gestures: cut, rotate, and scale. The object had to be
dragged into the middle of the screen, and a start button pressed, before executing the
manipulations. Two time durations were measured at the press of the start button:
Time to First Click (TFC), and Total Task Time (TTT). TFC captured the delay from
pressing the start button to the start of the gesture execution, and TTT measured the
total time from pressing the start button to completion of the gesture execution.
Brandenburg & Sachse (2012) found that the participants on the sensorimotor
level on Intuitive Use of User Interfaces (IUUI) continuum of knowledge (children)
were slower in terms of TFC than the participants at the culture level on IUUI’s
continuum of knowledge without knowledge of the tool (i.e. adults without prior
experience); the latter were, in turn, slower than the participants with knowledge of
the tool (i.e. adults with prior experience). They concluded that the results conform
with Hurtienne & Blessing's (2007) continuum of knowledge (refer back to Figure 2,
top). Brandenburg & Sachse (2012) also found significant differences in the
participant’s task performance (TTT) of the three gestures. All participants were
significantly slower in executing the cut gesture, compared to the scale and rotate
gestures. Scale was the second slowest gesture to perform, and rotate was the
slowest. Adults were faster in the execution of the cut and scale gestures than
children. When the participants were asked to repeat the same activity of
manipulating objects on the screen, their TTT did not improve equally over the three
gestures. While the cut and scale gestures improved in the second trial, there was no
improvement for the rotate gesture. However, Brandenburg & Sachse (2012) could
not conclusively determine the reasons for the differences.
The lack of adequate research that focuses on the role of sensorimotor
knowledge in children’s intuitive interaction was the motivation behind this research
study. Thus, its objective was to advance the existing and ongoing research
(Diefenbach & Ullrich, 2015; Hurtienne, 2009; McEwan, Blackler, Johnson, &
Embodied intuitive interaction in children 51
Wyeth, 2014; Mihajlov, Law, & Springett, 2015; Naumann et al., 2007) on intuitive
interaction, with specific reference to parameters and findings that can facilitate
children’s Embodied intuitive interaction. The continua of intuitive interaction
proposed by Blackler & Hurtienne (2007) were used to explain the results of the
study in Chapter 6 and Chapter 7, the outcomes of the study resulting in an Enhanced
Framework for Intuitive Interaction (EFII) explained in Section 8.1.2.
2.4 EMBODIED INTUITIVE INTERACTION
Intuitive interaction is strongly rooted in users’ Embodied skills. The
continuum of intuitive interaction (Blackler, 2008) and the continuum of knowledge
(Naumann et al., 2009) both highlight the role of Embodiment in intuitive
interaction. The simplest form of intuitive interaction relies on the Embodied
interactions with the world that use the sensorimotor knowledge acquired early in
childhood. This Embodied sensorimotor knowledge is acquired in everyday activities
and, by subconscious assimilation, becomes part of an individual’s behaviour
(Taraborelli & Mossio, 2008).
Embodied intuitive interaction is considered to be natural, and the simplest
form of interaction (Djajadiningrat, Wensveen, Frens, & Overbeeke, 2004;
Hurtienne & Israel, 2007; Sharlin et al., 2004; Terrenghi, Kirk, Sellen, & Izadi, 2007;
Zuckerman, Arida, & Resnick, 2005). However, there are very few studies that
investigated the role of Embodiment in intuitive interaction. These studies focussed
on Embodied metaphors that use body movements (Antle, Corness, & Droumeva,
2009) and image schemas as facilitators of intuitive interaction (Hurtienne & Israel,
2007). Metaphors allow retrieval of analogies from past experiences, and the
mapping of this retrieved information into the use of the new feature. Effectiveness
of the metaphors depends on how successfully they can be discovered and translated
into an appropriate action (Bakker et al., 2011). Thus, metaphors should be designed
in such a way that users are able to relate them to familiar things within or outside
the immediate context (Blackler et al., 2010).
Hurtienne (2009) investigated metaphors that represent recurring dynamic
patterns of bodily interactions—referred to as ‘image schemas”—derived from a
user’s experiential sensorimotor knowledge. Image schemas are “sensorimotor and
subconscious forms of knowledge representation” (Israel, Hurtienne, Pohlmeyer,
52 Embodied intuitive interaction in children
Mohs, Kindsmüller, et al., 2009). For example, the UP-DOWN schema was derived
from everyday experiences such as throwing a ball up in the air and climbing the
stairs. These schemas are deeply embedded in the human subconscious. Through a
series of experiments, Hurtienne (2009) showed that designing in accordance with
metaphorical extensions of image schema facilitates intuitive use. For example,
moving a slider up on a control increases the volume, and moving it down decreases
it. Hurtienne (2009) identified around 40 schemas based on human experiences that
can facilitate intuitive interaction. The image schema method is relatively simple and
easy to implement in design. However, interpretation of metaphorical extensions can
be culturally sensitive, and depends on domain-specific prior knowledge.
There is limited study in the field of Embodied intuitive interaction, which has
mostly focussed on Embodied metaphors and image schemas. To further advance
research in the field of Embodied intuitive interaction, this research study
investigated Embodiment through the lens of Embodied cognition to determine
design aspects of Embodiment (Section 4.2) which are then investigated through
experiments with children playing with a physical, virtual and TEI toys.
2.5 SUMMARY
This chapter has discussed the literature on Embodiment and intuitive
interaction. Perspectives of Embodiment based on how cognitive processes are
distributed between brain, body and environment were discussed. The perspective
used in this study suggests that cognitive processes in Embodiment rely on
perception action couplings which are initiated by the stimulus from the
environment. Cognitive aspects of Embodiment based on this perspective are
discussed in Section 4.1. Embodiment has also found its way into interaction design,
and its role in interaction design was, therefore, also discussed. Limited research in
Embodied intuitive interaction was discussed.
Intuition, which forms the basis of intuitive interaction, was discussed as a
cognitive process that builds on past experiences and prior knowledge. It is fast,
automatic, and non-conscious. Intuitive processes were explained using the Skill,
Rule and Knowledge- (SRK) based model of decision making, and Gore & Sadler-
Smith's (2011) concept of intuiting. The intuitive interaction research literature was
discussed, from the early research conducted by two research groups in Australia and
Embodied intuitive interaction in children 53
Germany (Blackler & Hurtienne, 2007), to recent work on intuitive interaction in
gaming (McEwan et al., 2014), interactive public spaces (Hespanhol & Tomitsch,
2015), and gestural interactions in older people (Martin Mihajlov et al., 2015). The
continuum of intuitive interaction and continuum of knowledge in intuitive
interaction were discussed.
Sensorimotor knowledge and Embodied experiences play an important role in
automatic cognitive processes. The relationship between Embodiment and intuitive
interaction has been addressed in the literature in the form of claims and assertions;
to date, however, this relationship has not been empirically investigated.
Chapter 3 discusses children’s Embodied intuitive interaction—the focus and
scope of this research study.
Embodied intuitive interaction in children 55
Chapter 3: Children’s Embodied Intuitive Interaction
Chapter 2 discussed the literature surrounding Embodiment and intuitive
interaction. The role of Embodiment in interaction design was discussed, ultimately
introducing the concept of Embodied intuitive interaction, and discussing the
relationship between Embodiment and intuitive interaction. This discussion
highlighted the limited research in this area, although intuitive claims of
Embodiment are made in the literature. There is even less research that investigates
the role of Embodiment in children’s intuitive interaction.
This chapter thus discusses Embodiment in children (Section 3.1). Specifically,
it highlights: the importance of epistemic and pragmatic interactions in Embodied
interactions, and the role of material and physical explorations in children’s
interactions; directly manipulated interfaces for children (Section 3.2); intuitions in
children (Section 3.3), including the role of intuitions in their learning; Embodied
intuitive interaction in the context of children as user groups (Section 3.4); and the
limited research on Embodied intuitive interaction in children.
3.1 EMBODIMENT IN CHILDREN
There is abundant literature on Embodied cognition, and researchers have
successfully applied ideas from Embodied cognition in interaction design (some of
which were discussed in Section 2.2). Embodiment has departed from a traditional
belief that cognition is completely situated in the brain. On the contrary,
Embodiment believes in a distributed theory, that is, that cognition is shaped by an
individual’s physical interaction in the world. As discussed in Chapter 2,
Embodiment highlights the role of the human body, brain, and environment acting
together in complex physical, social, and cultural contexts to determine cognitive
structures.
However, the question that has motivated this research study is how we might
use ideas about Embodiment in interaction design for children. The concept of
Embodiment has found its way into HCI; however, it has been underutilised as it has
been limited only to action-based or body interactions, and has mainly focussed on
56 Embodied intuitive interaction in children
technology capabilities. This is even more the case when it comes to interaction
design for children. Research in interaction design for children has usually focussed
around play and learning and, in this context, the goal has usually been to provide
engaging experiences.
The role of action and environment in children’s development, learning, and
play is evidenced in the literature. Knowledge acquisition in children is a continuous
process of invention and re-invention as they interact with the surrounding world in
their developmental stages (Piaget, 1952). According to Piaget’s theory of Genetic
Epistemology, children acquire three types of knowledge through their actions and
activities in the world, and this acquisition is hierarchical in nature. Physical
knowledge, the first and most basic knowledge acquired by children, is the
knowledge about objects in the world that people derive through their perceptual
properties. This is followed by abstract knowledge, knowledge that is learnt and
invented by children. Finally, social-arbitrary knowledge pertains to a specific
culture, and is learnt from people within that cultural group. Acquisition of social-
arbitrary knowledge requires the prior acquisition of both physical and abstract
knowledge. Furthermore, no knowledge can be acquired without the prior acquisition
of physical knowledge.
Intelligence develops as cognitive structures are formed from patterns of
physical or mental actions. Piaget (1952) rejected the idea that development of
knowledge and intelligence in children is genetically programmed in the brain;
rather, he believed that cognitive structures are formed in stages as children grow in
age. Piaget described development in children as a linear progression through
discrete stages of reasoning that correspond roughly with children’s ages: sensory-
motor, pre-operation, concrete operation, and formal operation stages.
Genetic epistemology is often conceptualised through the lens of Embodiment
because it emphasises the emergence of cognitive abilities grounded in sensory-
motor abilities. It also emphasises the role of the physical world in the development
of intelligence and knowledge. However, the age/stage aspect of Piaget’s theory does
not adequately consider individual differences in the development of intelligence that
result from each individual’s unique interactions with the physical world. In contrast,
Embodiment shifts the focus away from development as a linear progression
Embodied intuitive interaction in children 57
culminating in the development of abstract reasoning abilities, and towards a situated
and integrated view of development.
Thelen & Smith (1996) discussed how walking depends on the child-
environment relationship and on other variables, such as limb weight and size. They
argued that the environment-dependent complexity of learning to walk rules out a
programmed linear progression in developmental stages in children. Children and
infants do not have a pre-set goal to learn to walk. The environment needs to provide
for the child’s learning. Variations in the environment ensure that people generate
different movements in different environmental situations (Bernstein, Latash, &
Turvey, 1996). Different circumstances in an environment result in different patterns
of muscle excitations for limb and body movements. Thelen and Smith (1996)
explained the role of variations in the environment in children’s walking, kicking,
and stepping. Limbs are dynamic systems whose properties (functional and physical)
depend on their interactions with body and environment. Steps are self-organising
motions that emerge from the history of changes to the environment system.
Alternate stepping (walking), then, emerges from the child-environment interactions.
The body and its movements play an important role in processing old ideas, as
well as generating new ideas. This results in better retention of knowledge. Cook,
Mitchell, & Goldin-Meadow (2008) observed children, encouraging them to make
gestures while learning a new arithmetical strategy. They found that children who
were told to gesture during learning retained 85% of their post-test gains four weeks
later, compared to only 33% of children who were told to only speak during learning.
Children’s increased use of technology in play and learning means that they are
spending less time interacting with the environment, performing activities that trigger
sensory-perceptual structures. These interactions and activities help develop
knowledge structures that aid children’s cognitive development (Livingstone, Marsh,
Plowman, Ottovordemgentschenfelde, & Fletcher-Watson, 2015). Thus, an
Embodied perspective to design could focus on the fact that children create meaning
through action (body movements and manipulation). This, in turn, could inform ways
to design children’s products, physical products, and TEIs that are situated in activity
and in the world.
Children’s Embodied cognitive processes are not different to adults’ Embodied
cognitive processes. Thus, the literature that was discussed in Chapter 2 also applies
58 Embodied intuitive interaction in children
to children. However, these processes develop differently in children, depending on
their social and physical environment and age. Nunnally & Lemond (1974)
suggested that children develop Embodied cognitive processes through explorations
of artefacts and the environment. This study considered that these explorations are
distinctly different to those in adults, and this is evident in the ways children
approach solving new challenges and problems (Kail, 2012).
Embodiment in children has formed the basis for this study on Embodied
intuitive interaction in children. Cognitive aspects of Embodiment are further
discussed in Section 4.1, and although not discussed in literature in the context of
children, these aspects do apply to children as well.
3.1.1 Children’s interactions
Children obtain and utilise information about persistent properties of the
environment (Gibson, 1988). Their use of object manipulation and environment in
development, play, and learning has been discussed in the literature (Kail, 2012; Ruff
& Saltarelli, 1993). Children learn and develop new skills and knowledge about the
environment through object manipulations and interactions with artefacts in the
environment that come their way (Jones & Lederman, 2006). McCall (1974), in one
of the first investigations of exploratory manipulation in human infants, suggested
that early exploration in children is an investigation of the raw sensory perceptual
feedback of objects. It does not require children to expend a significant amount of
energy to manipulate the objects in an environment. Visual and tactile inspection of
objects is accompanied by motor schemes such as shaking, banging, turning the
object over and over, and shifting it from hand to hand. Gibson (1988) suggested
that, in perception, these motor mechanisms are an essential part of seeking
information.
These manipulations with objects are not random; rather, they are driven by
intentions, curiosity, and intrinsic motivations (Bruner, Jolly, & Sylva, 1976). As
children grow older, their interactions become more relational and symbolic, and this
is evident in their imaginary play with objects. In turn, this suggests the emergence
of new cognitive abilities from these nonspecific manipulations. The acquisition of
new knowledge, age-related capacities, and changes in environment, all contribute to
the development of cognitive abilities.
Embodied intuitive interaction in children 59
Antle (2011) suggested that children simplify problem-solving tasks by
manipulating the environment (in ways such as physically turning jig-saw pieces to
see where their shape fits into the puzzle), and stated that children ‘think with the
hands’. Kirsh (2013) deconstructed such activities using concepts of pragmatic and
epistemic actions. Epistemic actions, such as search and planning, alter the world to
aid and augment the cognitive processes; for example, arranging furniture in a room
to decide on the layout, or adjusting alignments and placements of objects to obtain
an ideal fit to deflect balls on the targets in the game of Osmo (Tangible Play,
2014b). Pragmatic actions, on the other hand, are actions that alter the world when a
physical change is required for its own sake; for example, actually placing a puzzle
piece into the scene it is helping to create.
While epistemic actions could fail to assist in reaching a goal, they could
reveal completely unexpected information and shortcuts that would have been very
difficult to find by following a straightforward approach (Feinstein & Meshoulam,
2013). While such interactions might take longer, they are effective in offering
appropriate directions for children to reach a goal. Thus, an effective tool for
children’s learning and play should both facilitate goal-oriented pragmatic activity,
and provide a balanced amount of epistemic task space (Kirsh & Maglio, 1994).
3.1.2 Role of materiality in children’s interactions
Directly manipulated interfaces allow children to explore an artefact and, in the
process, they learn its use. This behaviour of exploring and learning comes naturally
to children. Montessori (2013) described the use of materials and activities to help
children develop their sensory capabilities. The materials put children in control of
their learning process, enabling them to learn through personal investigation and
exploration.
Children build and experiment with manipulative materials and, in the process,
develop a rich understanding of, and think about complex and abstract concepts.
However, Formal methods which involve manipulation of symbols are used to teach
complex concepts especially those related to dynamics and systems. As a result,
students are able to grasp and understand these concepts only when they grow older
and reach higher grades (year levels) at school, when they have developed more
mathematical expertise (Gopnik, Meltzoff, & Kuhl, 2009). Montessori (1914/2011)
introduced new ways of teaching children using materials and activities which tap on
60 Embodied intuitive interaction in children
children’s sensory capabilities. These materials allow children to explore and
investigate through physical and material manipulations and in the process be in
control of their own learning. Resnick et al. (1998) used the Montessori method to
develop traditional toys with computing elements embedded within them. Such
digital manipulatives enable children to explore a new set of concepts which would
otherwise be considered too advanced for them.
Children’s use of material and spatial properties in their interactions with toys
is one of the criteria used to investigate Embodiment in children playing with
physical, virtual and TEI toys ( Chapter 6 and Chapter 7 ). Spatial and material
properties of artefacts as a design aspect of Embodiment is discussed in Chapter 4.
3.2 DIRECTLY MANIPULATED INTERFACES FOR CHILDREN
The advantages presented by children’s material and physical interactions have
been the focus of research in TEIs for children, especially directly manipulated ones.
Researchers are interested in designing products that offer the advantages of physical
and material interaction. At the same time, however, the products need to incorporate
new innovations in technology.
Material and physical manipulations enable children to describe the actions
available in a physically shared space (Rogers, 2011). Fernaeus & Jacobsson (2009)
investigated the role of materiality in clothing in human culture. They concluded that
material properties of clothes allow people to present themselves to others through
their surface appearance. When people come in contact with the clothes, the
materiality of the clothes provide convenience, comfort, and warmth. Materiality also
serves a range of communicative functions; for example, it can suggest appropriate
group behaviour, group belonging, and expected interactions (Entwistle, 2015).
Taking inspiration from theatrical costumes, Fernaeus & Jacobsson (2009) suggested
that artefacts could be designed with material properties that conveyed information to
the user. This could then indicate to the user an artefact’s modes of operation and, in
turn, the behaviours and interactions that could be expected at a certain point in time
during interaction with the artefact. Fernaeus & Jacobsson (2009) used this concept
in developing a dinosaur toy that allowed children to decide its operation, using
accessories such as stickers, badges, bracelets, and decorative costumes. A dinosaur
Embodied intuitive interaction in children 61
toy, when decorated with a bracelet, was set in a play mode; when it wore pyjamas, it
went into sleep mode.
Seo, Arita, Chu, Quek, & Aldriedge (2015) developed Stampies to investigate
how young children associate meanings that allow playful physical interactions with
materiality. They were interested in understanding if the associations could be
extrapolated to digital interactions. Stampies consisted of physical objects made out
of different materials (wood, felt, silicone, and plastic), with a unique pattern of
conductive thread underneath. The Stampies were used with an iPad drawing
application that consisted of colour bands and graphic images of animals, fruits,
musical instruments, and clothing. These graphic images were both coloured and line
drawn. Children were asked to choose a Stampie to associate with each of the graphic
images on the ipad app. Then, they were asked to recall (on a blank canvas) what
each Stampie represented. Children were finally asked to create an artwork using the
chosen Stampies. This process was carried out separately with colours, coloured
images, and line-drawn images. The findings suggested that children associated
materials with meanings. Children associated certain materials with specific objects,
for example, wood with musical instruments. However, a material’s feel (whether
soft, hard, etc.) dominated as a guiding factor when reassessing the associations; for
example, children mostly preferred soft materials such as felt and silicone,
irrespective of their graphic image.
Children are increasingly using TEIs in gaming (Crowle, Boniface, Poussard,
& Asteriadis, 2014), patient rehabilitation (Vogiatzaki, Gravezas, & Solutions,
2013), collaborative coordination in time-critical situations (Fischer et al., 2014), and
in education (Gardner & Elliott, 2014). Efforts to develop TEIs for children are more
prevalent in education and (to some extent) gaming, than in any other field. In one of
the first examples of attempts to encourage social interactions in children while
playing digital games, Brederode, Markopoulos, Gielen, Vermeeren, & de Ridder
(2005) developed ‘pOwerball’. The objective was to bring together children, with
and without learning disabilities, to play a game. The game consisted of a tabletop
with graphic virtual elements which could be controlled through manipulation of
tangible objects on the tabletop. TEIs could pose challenges for children as they
could be interacting with multiple interfaces at the same time. These interfaces could
be physical or virtual, or both, thus requiring knowledge from physical and virtual
62 Embodied intuitive interaction in children
worlds. Knowledge from the physical world mostly relates to day to day experiences,
and are Embodied in everyday activities. On the other hand, knowledge from the
virtual world relates to experience gained from using virtual elements that are
design-dependent. The tendency in the past has been to look at TEIs through the lens
of pervasiveness (Fischer et al., 2014; Gardner & Elliott, 2014; Ricci, Piunti,
Tummolini, & Castelfranchi, 2015). Thus, the focus has been more on the technical,
rather than the human-centred, aspects of design.
Direct interaction with, and manipulation of objects helps children to ascribe
meaning to them. As technology becomes more accessible to children, new interfaces
and systems should be designed to take advantage of material and physical, as well
as technological, interactions. Tactile direct interactions were used as an interaction
modality in this study. Direct interactions with toys were coded for Embodiment and
intuitive interaction, discussed in Chapter 6 and Chapter 7
3.3 INTUITION IN CHILDREN
The interest in children’s intuition appears mainly in the fields of child
development and the learning sciences, fields that are mainly interested in children
educational needs. Schon (1982) suggested that children develop intuitive
understandings very early in childhood which enable them to give correct answers to
abstract questions. However, when they start attending school, teaching interrupts
their intuitive thinking through the introduction of models and procedures that
interfere with their intuitions; furthermore, as their intuitions are overridden, they
make mistakes. Later on, when children reach higher grades (year levels) at school,
they achieve a more developed, schooled intuitive understanding of various concepts
that enables them to answer correctly once again.
Choi (1993) argued that children are able to answer correctly again because
they are able to make higher order intuitive connections and understandings, and
with increased expertise levels. The development of expert skills leads to the
development of intuitions that Choi referred to as ‘Matured Intuitions’. However, this
takes years of learning, and Fischbein (1999) suggested that children either give up
on learning some of the concepts, or grow up with an incomplete and inaccurate
understanding of them.
Embodied intuitive interaction in children 63
Fischbein (1999), Noddings and Shore (1984), and Resnick (1986) proposed
that teaching models should support and enhance intuitive thinking in children, and
should not contradict the knowledge that is inherent in children when they start
school (Fischbein, 1999; Noddings & Shore, 1984; L. B. Resnick, 1986). Early
intuitions about numbers should form the basis of the elementary school mathematics
curriculum. These would then need to be extended to support secondary school
mathematics (Resnick, 1986). Resnick explained how these intuitions might function
in the child’s construction of new knowledge. Human beings acquire information and
convert it into well-structured, self-consistent, action-oriented representations of
reality, using language, logic, reasoning, and tools and instruments. This knowledge
through reasoning becomes an autonomous activity.
Intuitions are shaped by experiences, and they can be used as the basis for
understanding abstract mathematical concepts (Fischbein, 1987). Children develop
an understanding of space, geography, the cultural environment, and practices related
to professions from the daily experiences that shape their intuitions. Fischbein
further added that intuitive knowledge is immediate and self-evident, and that this
helps children to easily understand abstract concepts. Thus, teaching and learning
models should build on children’s intuitions.
Children readily relate to problems that have origins in the real world around
them. Zuckerman et al. (2005) explored the role of physical materials in building
children’s existing intuitions, and in developing their mathematical and scientific
intuitions. Clement (1994) suggested the use of physical materials to model
mathematical situations. Physical materials can be extremely useful in enabling
children to construct an essential intuitional foundation for mathematical thinking
and problem solving. For this level of intuition to develop, children should be
familiar with the physical materials from their everyday use in learning, or from
other domains (Blackler et al., 2010) in which they construct symbolic meanings and
operational definitions (O’Brien, 2010).
Disessa (1988) considered the role of familiarity in children’s intuitive
thinking. When children are confronted with unfamiliar questions, they are unable to
make connections with things around them, and are thus unable to use their intuition.
They make connections with their experiences and with objects around them based
on the clues that they pick up from the questions; they then choose appropriate
64 Embodied intuitive interaction in children
intuitions based on these connections (Disessa, 1988). To make these connections,
children should be familiar with clues, such as the symbols and language used in the
questions. This is very important, especially when children are presented with
abstract concepts and principles such as gravity and infinity.
To summarise, there is some evidence that intuitive learning in children results
in a less stressful learning experience, especially with the learning of abstract
concepts that deal with formally based types of certainty (Fischbein, 1987). Attempts
must be made to facilitate intuition-based learning (Fischbein, 1987). One of the
ways to do this is to allow children to learn in the real world and environment with
real objects (Clement, 1994), and allow them to apply their existing knowledge.
Intuition and intuitive processes have been studied in the field of interaction
design to develop instruments, continua and frameworks for the design and
development of intuitive products (Blackler, 2008; Hurtienne & Israel, 2007).
However, there are limited studies that investigate intuitive interaction in children
(Brandenburg & Sachse, 2012). This study has thus furthered the research on
intuitive interaction in the context of designing children’s products. Intuitive
processes in children were used to understand intuitive interaction and code these
processes in Experiment 1 and Experiment 2, discussed in Chapter 6 and Chapter 7.
3.4 CHILDREN’S EMBODIED INTUITIVE INTERACTION
Research suggests that Embodiment offers a natural or intuitive form of
interaction (e.g. Rosenbaum, Eastmond, & Mellis (2010). Antle et al. (2009a)
investigated these claims in the context of HCI. They suggested that claims of a
natural or intuitive form of interaction could be due to Embodiment being associated
with the human body and its participation in the environment. For example, humans
use their Embodied experiences to understand abstract concepts by using metaphors.
Antle et al. (2009b) developed a responsive auditory environment where Embodied
metaphors were used to map physical body movements, resulting in sound output
with percussive sounds. They used orientational and ontological metaphors to
represent parameters of music such as volume and pitch. They carried out empirical
experiments with adults (18–40 years old) and children (7–10 years old) interacting
with the auditory environment, with and without Embodied metaphors. Participants
performed specific physical movements to vary one of the sound parameters
Embodied intuitive interaction in children 65
(volume, tempo, pitch, and rhythm) to produce different sound sequences. They used
the following measures to determine whether the systems were intuitive to use: (i)
time taken to practice creating a sequence; (ii) accuracy of the final presentation of
the sound sequences; (iii) accuracy of the verbal explanation for each sequence task;
(iii) Intrinsic Motivation Inventory (IMI); subscales for enjoyment and interest; (iv)
perceived competence; and (v) individual statements related to ease of learning,
intuitiveness of learning, and amount of concentration required to learn. The results
revealed that the Embodied metaphor-based system was more intuitive than the
system without the Embodied metaphors. The experiments proved the effectiveness
of Embodied metaphors in intuitive interaction.
Completion of even a simple task involves a complex web of mental and
physical manipulations. Image schemas are translated into physical variables that are
then mapped into digital variables for interaction with technology. However, there
are multiple cross-mappings among these variables. As Embodied metaphors are
derived from Embodied experiences, the effectiveness of Embodied metaphors with
children could be questionable. This is because children’s Embodied experiences
could be more limited than those of adults, and are largely varied across
demographics and cultural backgrounds (Eriksen, 2001; Jenkins, 2014). In the Sound
Maker developed by Antle, Droumeva, & Corness (2008), children manipulated
pitch, volume, and tempo of sound through whole body interaction. However, some
children were unable to discover the mapping of the system within the set timeframe.
For this reason, Bakker, Van Den Hoven & Antle (2011) developed Moving Sounds
(MoSo) to study how tangibles can support children’s learning of abstract sound
concepts such as pitch, volume, and tempo. Bakker, Van Den Hoven, & Antle (2011)
identified Embodied metaphors that were used unconsciously by children aged 7-9
years to represent these musical sound concepts. Directly manipulated products were
developed to represent each of the identified Embodied metaphors. For example, a
puller artefact was used to represent the near-far Embodied metaphor, with ‘near’
representing low pitch, and ‘far’ representing high pitch.
The implementation of Embodied metaphors in the directly manipulated
products was evaluated in a user study with 50 children aged 7–9 years. The children
were asked to listen to a short sound sample in which the concept changed from one
extreme to another, for example, soft to loud volume. Each child was then given an
66 Embodied intuitive interaction in children
Embodied metaphor-based product to enact the change of sound. Children then
exchanged the products, and the exercise was repeated until all children had played
with all the products. The entire exercise was video recorded for analysis. Bakker,
Van Den Hoven, & Antle (2011) found that there were consistent patterns of
interactions with some products, while the interactions with others were inconsistent.
Almost all children used rotating artefacts to enact changing volume and the
Embodied schema slow-fast. On the other hand, children moved a stick with beads
attached to it in different ways to enact changing volume. None of the movements
resulted in low-high schema. At the same time, most of the children working with
pitch moved this artefact low-high. This means that some Embodied metaphors such
as low-high might be less appropriate for physical artefacts for change in volume, but
more appropriate for change in pitch.
Bakker, Van Den Hoven, & Antle (2011) pointed out that the effectiveness of
the Embodied metaphors in full body interaction systems such as the one described
in Antle et al. (2009b) depends on how successfully children are able to discover the
metaphors and translate them into an appropriate physical action. Directly
manipulated interfaces can exhibit features that children can easily discover and
activate.
Research in Embodied intuitive interaction has mainly focussed on
metaphorical interactions as facilitators of intuitive interaction. However,
Embodiment goes beyond body movements. Researchers in cognitive science have
identified aspects of Embodiment (as discussed in Section 4.1). However, the
relevance of these aspects to design has not been previously discussed, but is now
explored in this study (in Section 4.2). Physical products and TEIs are anticipated to
provide benefits to children in terms of usability, user experience, and learning and
development. However, there is limited empirical evidence to validate these claims.
Research in Embodied interaction for children has predominantly focussed on the
theoretical evaluation of tangibles (Antle, 2007); on the technology needed to embed
electronics in physical objects (Olson, Atrash Leong, Wilensky, & Horn, 2011); and
on the research methods to evaluate the effectiveness of tangibles for children
(Zaman et al., 2009). This research study focussed on the interactions with directly
manipulated interfaces to investigate children’s Embodied intuitive interaction.
Embodied intuitive interaction in children 67
There is limited study in the field of Embodied intuitive interaction (Antle et
al., 2009a; Hurtienne & Israel, 2007), which has mostly focussed on Embodied
metaphors and image schemas. Although limited studies on intuitive interaction in
children associate children with sensori-motor knowledge (Brandenburg & Sachse,
2012), there is a lack of research that focusses on Embodiment and intuitive
interaction in children. To further advance research in the field of Embodied intuitive
interaction, this research study investigated Embodiment through the lens of
Embodied cognition to determine design aspects of Embodiment (Section 4.2).
3.5 SUMMARY
This chapter has discussed the literature related to children’s Embodied
intuitive interaction, and highlighted the importance of their epistemic and pragmatic
activities. Epistemic interactions could prolong the time taken to reach a goal. At
times, they could even fail to assist in reaching a goal; however, they could also
result in unexpected discoveries that could make the journey to the goal easier.
Children find meaning in their interactions with material and physical
properties of objects. If the material and physical meanings derived through
perceptual sensory systems could be extrapolated to digital interactions, it could
result in interfaces that take advantage of both material and physical properties and
technological advances. Children unconsciously relate complex abstract concepts
with physical and material interactions. However, based on their past experience and
previous knowledge, they associate certain concepts with specific movements or
interactions. Thus, use of Embodied metaphors in tangible physical interactions for
intuitive use is suggested in research. However, considering the advantages offered
by material and physical properties of objects in interactions, there is scope for the
study of Embodied intuitive interaction in directly manipulated interfaces. This
research study has thus focussed on children’s Embodied intuitive interaction in the
context of directly manipulated interfaces.
Chapter 4 discusses cognitive aspects of Embodiment, based on the perspective
that suggests that the perception action loop originates from the environment in the
form of stimulus that is sensed by children’s perceptual sensory systems. Design
aspects of Embodiment derived from these cognitive aspects are discussed. These
68 Embodied intuitive interaction in children
design aspects were used in the empirical investigation of Embodied intuitive
interaction in children.
Embodied intuitive interaction in children 69
Chapter 4: Aspects of Embodiment
Chapter 2: discussed theories and frameworks pertaining to Embodiment
through the lens of cognitive science. Chapter 3 discussed Embodiment in the
context of children. The perspective of Embodiment used in this study is one that
suggests that cognitive processes are distributed between body, brain and
environment through continuous perception action loops and it is the environment
that initiates the process of perception and action loop and not the brain. In other
words, it is the environment that provides stimulus to trigger perceptual sensory
systems in children. This is the meaning of Embodiment that Anderson (2003),
Wilson & Golonka (2013) and Wilson (2002) have discussed in the context of
cognition. Thelen (2008) and Payne (1991) takes the same approach to describe
development of abilities and skills in humans and interactions with displays
respectively.
Section 4.1 discusses cognitive aspects of Embodiment. These aspects are
representative of perspective of Embodiment presented by Anderson (2003), Wilson
& Golonka (2013) , Wilson (2002) and to some extent Thelen (2008) in cognitive
science and perspectives from research in interaction design presented by Shaer &
Hornecker (2010) and Payne (1991). Section 4.2 presents design aspects of
Embodiment as a framework for Embodiment, derived from the cognitive aspects of
Embodiment. These design aspects were used to address the research sub-questions
in Experiment 1 and Experiment 2.
4.1 COGNITIVE ASPECTS OF EMBODIMENT
Physical grounding is the central, defining characteristic of Embodiment
(Anderson, 2003). Abstract representations acquire meaning from the real world
through physical grounding, and involve cognitive contents grounded in terms of the
agent’s Embodied experience and physical characteristics (Harnad, 2003; Lakoff &
Johnson, 1999; Vogt, 2002).
The traditional approach to cognition lacks the resources to ground its
representations (Harnad, 2003). Vogt (2002) further explains that Cartesianism,
Brainbound, and GOFAI (Section 2.1) rely entirely on human interpretation to give
70 Embodied intuitive interaction in children
meaning to symbols. Therefore, they implicitly require the human grounding
capacity to serve as an intermediary between its outputs and real-world activity.
Physical grounding of ‘chair’ requires specific physical skills and experiences related
to sitting and other related activities such as standing, walking, and running
(Anderson, 2003). Grounding requires people to have experienced an act of sitting,
and to have knowledge of chairs. This allows people to perceive the scene in front of
them, and to determine if it affords sitting. Experience, skill, and knowledge in using
chairs also enable people to understand other concepts such as ‘tables’. These
concepts are also related to semantics (Barsalou, Santos, Simmons, & Wilson, 2008)
and to context of use (Maguire, 2001). In other words, Embodiment and physical
grounding require a consideration of the social significance of objects and context.
The cognitive aspects of Embodiment that form the basis of physical grounding are:
Real-world and real-time aspect, Evolutionary aspect, Cognitive offloading, and its
Social aspect. Each of these aspects are discussed below.
4.1.1 Real-world and real-time aspect
People carry out cognitive activities in the real world, with real people and objects,
and in real spaces. The real-world activity is possible because cognition involves
perception and action (Clark, 2008; Eelen, Dewitte, & Warlop, 2013; Goodwin,
2000). Knowledge is situated in activity, and in social, cultural, and physical contexts
(Brown, Collins, & Duguid, 1989). However, this knowledge requires direct
interaction with the things that are to be cognitively processed (Clark, 2013).
Driving, communicating, and arranging puzzle pieces (i.e. trying to decide where
they fit into the whole picture) are some of the examples of such activities. Situated
activities are possible only in the real world and in real time, where information
perceived through the senses affects the way a particular task is executed; this
execution, in turn, affects the environment. Some situated activities might require
mental activities, such as the communication of past information, to perform situated
activities (Leakey, 2008); for example, the use of language in daily activities.
Children are considered as dynamic systems (Forrester, 1995; Thelen & Smith,
2006) as they build a dynamic history of activities in the real world. The real-world,
real-time nature of these activities gives momentum to behavioural acts, so that the
system is always impacted differently by its activities. Thelen & Smith (1996)
viewed development in children as a series of behavioural patterns evolving over
Embodied intuitive interaction in children 71
time, with time periods that are stable in terms of behaviour. Each behavioural act
occurs over time, showing courses of activation, peak, and decay. Every act changes
the system in some way or the other, such as properties, structural and so on. These
system changes build a history of acts over time. Thus, repeating the same behaviour
over time stabilises the system at some point resulting in habituation or learning. The
real-time, real-world nature of behaving/acting can result in different behavioural
outcomes under similar conditions. The outcomes depend on the immediate previous
history of the system.
Interaction in the real world is associated with response and feedback in real
time. This could result in individuals having to perform cognitive activities under
pressure. Certain activities such as flying an aircraft, and situations such as solving a
puzzle within a certain time limit, require fast responses to the environmental
stimulus. These responses need to evolve and adapt continuously to the ever-
changing environment. Responses in such situations cannot be derived from a mental
representation of the environment, as there is no time to build the representations of
the ever-changing environment. Cartesianism-, Brainbound-, and GOFAI-based
systems could either become stuck in a ‘confused state’, or generate an inaccurate
response. The systems might not have the representation for that particular situation
in the environment. Kirsh (2013) refers to this situation as the ‘representational
bottleneck’.
Embodied cognition, on the other hand, suggests that people generate situation-
appropriate responses on the fly to avoid representational-bottleneck. Individuals
tend to step back, observe, assess, plan, and then take action in the absence of time
pressure. However, they behave differently under time pressure, and this could lead
to a different outcome to the problem-solving task.
4.1.2 Evolutionary Aspect
Complex behaviour in organisms is linked to evolutionary theory (Hayles, 2010).
Most animals have significant built-in behavioural expertise, without having to
explicitly learn it from scratch. This expertise is developed as an organism evolves
over many years. Organisms possess specific physical features and behavioural
characteristics for coping with a specific environment (Anderson, 2005). As
organisms evolve, their physical and behavioural features undergo changes. The
cognitive adaptations then develop in light of the organism’s physical or structural
72 Embodied intuitive interaction in children
features and these, in turn, change in the context of reliable environmental features.
This is known as ‘emergent behaviour’ in organisms.
The central idea of the evolutionary aspect of Embodiment is the re-utilisation
of existing behaviours to solve new problems (Anderson, 2005). Clark (2005),
Turner (2013) and many others have used crickets as an example to explain the
evolutionary aspect of Embodiment. Cricket chirps are low in amplitude, and this
makes it hard to reach prospective mates. So, they build a specially shaped burrow
consisting of a hollow bulb underground, connected by a narrow constriction to a
flared tunnel opening into the air. The cricket chirps from the intersection of bulb and
horn, and adjusts the burrow until it experiences the right resonant frequency. Turner
(2013) suggested that the original purpose of the burrow must have been to protect
themselves from predators, rather than to attract mates, and that this behaviour is an
evolutionary development. Similar re-utilisation of physical structures and
behaviours should be expected in cognition (Clark, 2013).
Abstract reasoning capabilities can be traced back to perceptual and motor
inference in primitive creatures through metaphorical mappings across the relevant
domains (George Lakoff & Johnson, 1999, p4). They emphasise that the neural and
cognitive mechanisms that help organisms to perceive and move around also create
conceptual systems and modes of reasoning. Thus, to understand reasoning, one must
understand the details of the visual system, motor system, and the general
mechanism of the neural binding.
As evolved creatures, human beings have inherited their ancestors’ capacities
and systems for meeting their needs and coping with a given environment (Brooks,
1990). This tendency to emphasise the continuity between humans and animals, and
the willingness to see instances of intelligent behaviour in animals on evolutionary
grounds has been the motivation behind the study of Embodied Cognition (Gibbs,
2006).
4.1.3 Cognitive offloading
Actions that are used to offload cognitive processing onto the external world in order
to reduce the difficulty of the mental task at hand are called ‘epistemic actions’
(Kirsh & Maglio, 1994). “Examples of epistemic actions include looking at a
chessboard from different angles, organising the spatial layout of a hand of
Embodied intuitive interaction in children 73
cards…laying out our mechanical parts in the order required for correct assembly,
and so on.” (Clark, 2008, p. 511)
Problem-solving routines involve intensive computations and internal
representations on one side, and repeated environmental interactions on the other
(Clark, 2008). Clark explains this with an example of using a stick to retrieve a ball
stuck on a roof. People do not measure the distance of the ball or the trajectory of
approach necessary to retrieve the ball. Rather, they try a stick; if it does not work,
they either try another stick, or adjust the length of the first one. People might change
the angle at which the ball is approached, or the direction of approach. They might
re-position themselves to solve the complexity of the visualisation problem, and use
trial and error to decide the appropriate length of the stick.
In a puzzle-solving problem, the manipulation of the puzzle elements through
trial and error serve as tokens to reach a goal state. The manipulation of elements in
the puzzle does not offer any information about a solution to the problem, but it helps
in reaching the goal by revealing clues that could help in making decisions on the
actions to be taken. On the other hand, diagramming represents a different use of the
environment. Here, the cognitive system exploits the external resources to obtain a
solution or knowledge that could actually be used at a later stage. Nathan (2008)
explained the role of gestures in symbolic off-loading. Gestures help to convey
complex and abstract ideas either as a substitute to verbal interactions or as a support
structure to verbal interactions to communicate thoughts and ideas effectively.
Gestures offset the large cognitive demands in verbal interactions that require
communication of ideas and thoughts with clarity while overcoming barriers such as
linguistic, logistic, physical and so on (Hostetter & Alibali, 2008). This type of
symbolic off-loading could be applied to spatial tasks, such as arranging tokens to
represent armies on a map, and could also be applied to non-spatial tasks such as
mind maps to determine logical relations among categories.
Humans exploit the environment to reduce the cognitive load experienced
during ‘on the fly’ cognitive processing because of the limits on attention and
working memory. They use the environment to hold and manipulate information, and
to use it when required. ‘On the fly’ cognition is concerned with immediate input
from the local environment (Iverson & Thelen, 1999, p37). People switch to slower
cognitive processing, relying on representations to make more careful considerations,
74 Embodied intuitive interaction in children
such as making a mental check on something odd, or planning future behaviour
(Corr, 2008). Everyday activity such as reading, solving puzzles, and conversations
predominantly use on the fly cognition. However, when the usual flow is interrupted,
people unconsciously switch to slower processing.
Humans are able to handle the representational bottleneck when confronted
with real time cognitive demands. They either use internal representations that they
have acquired through prior learning and experience, or use the environment to
reduce cognitive load. They adapt to changing situations by using epistemic actions
to alter the environment, and reduce the cognitive load in the process (Kirsh, 2013a).
Cognitive processes could be distributed between internal (mind) and external
structures (material or environmental), however this requires a well organised
coordination between these two structures for effective distribution (Cole & Griffin,
1980). Norman (1993) suggested that artefacts improve human abilities in carrying
out tasks, such as the use of calculators to perform arithmetic calculations. Artefacts
can be internal such as task-specific rules used by an expert to perform a task, or
external material artefacts such as GPS used for navigation. Material artefacts are
used when an individual is unable to use internal artefacts. When a person is unable
to perform arithmetic calculations in their mind, he/she uses a calculator to perform
them.
4.1.4 Social Aspect
Knowledge lies not only within the individual, but also in the individual's social
environment (Hutchins, 2000). Activities in the real world are performed through
interactions between people and their tools and artefacts in the context of the task at
hand.
Cognitive processes can be distributed across the members of a social group
(Salomon, 1997). Intelligence can be explained by assembling groups of experts in
various configurations for the execution of tasks. Minsky (1988) suggested that
children learn new concepts through their interaction with others (adults and peers)
and artefacts. The experience of interaction with others enables children to create
functional systems in their absence and, in turn, to contribute towards the creation of
new functional systems in some other person. Minsky (1988) described this as a
transfer of functional skill from one society of mind to another, resulting in a
propagation of a certain pattern in a community.
Embodied intuitive interaction in children 75
Dourish (2004) discussed the participatory nature of Embodiment, and
suggested that the social world, along with the physical world, provides context and
meaning to an individual’s activity in the world. Embodied actions are an integral
part of any cooperative interaction (Robertson, 1997). Robertson proposed a
taxonomy of Embodied actions that emerged from observing people doing group
design work. The taxonomy identifies two divisions of Embodied actions, one in
which individual Embodied actions relate to actions on physical objects and spaces,
and another in which group activities are constituted by individual Embodied actions.
Thus, it can be seen from the above description of the cognitive aspects of
Embodiment that cognitive processes are result of elements in the environment,
dynamic processes which are part of the perceptual sensory system of children as
well as the environment and the processes involved in establishing connections
between the mind and the environment. The question then arises how design can
exploit these elements and processes to facilitate intuitive interaction. Section 4.2
explores design aspects derived from the cognitive aspects of Embodiment, which
were further examined in Experiment 1 and Experiment 2 for intuitive interaction,
discussed in Chapter 6 and Chapter 7 respectively.
4.2 DESIGN ASPECTS OF EMBODIMENT
Design aspects of Embodiment were derived from the cognitive aspects of
Embodiment (Section 4.1), which are perspectives of Embodiment from cognitive
science (Anderson, 2003; Thelen, 2008; Wilson & Golonka, 2013; Wilson, 2002)
and from perspectives of Embodiment in interaction design (Shaer & Hornecker,
2010). These aspects are not mutually exclusive, and are non-exhaustive.
4.2.1 Physical affordances
The concept of affordances was originally introduced by Gibson (1979/2014),
who described affordances as the properties of the environment relative to an
individual. The properties are interpreted by individuals through their sensory-
perceptual systems, enabling them to control their actions. For example, the
affordances of climbing a stair in a bi-pedal fashion has been described in terms of
the height of a stair riser in relation to a person’s leg length (Warren, 1984).
Individuals perceive what possibilities for action an environment offers in terms of
its properties, mediums, and compositions. Thus “…the affordances of the
76 Embodied intuitive interaction in children
environment are what it offers the animal, what it provides or furnishes, either for
good or ill …”(Gibson, 1986, p.127).
Gibson also emphasised that affordances depend on an individual’s ability to
perform actions in the environment. A chair, for example, affords sitting for adults;
however, for infants it is a walker or a support. Gibson’s affordances does not
depend on the ability of the user to perceive it, and it does not change if the needs
and goals of the user change. McGrenere & Ho (2000); Norman (1999) referred to
Gibson’s affordances as ‘real affordances’ or ‘physical affordances’.
People look for clues in an interface to determine the affordances that it offers
(Dotov et al., 2012). Physical affordances are associated with natural clues, for
example, the weight of a bag. Natural clues represent natural properties of artefacts
such as geometry, shape, or weight of an object. Objects also have properties or
qualities—such as colour, texture, composition, size, shape, mass, elasticity, rigidity,
and mobility—that form their affordances. These physical and material properties
constitute natural clues, and determine and constrain what can be done with the
objects. Physical affordances can be perceived through interactions with objects and
environments in the physical world. Physical affordances in the form of natural clues
embedded in everyday objects and tools, allow real world interactions analogous to
real world services and behaviours (Want, Fishkin, Gujar, & Harrison, 1999).
Gibson introduced affordances in reference to visual perception (Gibson,
1979/2014). However, in Gibson (1966), he explained the role of human perceptual
systems in affordances. He explained the role of senses in perception, especially
focussing on the fusion of information from all senses in perceiving the properties of
objects that would afford action. In recent years, research has focussed on designing
products that allow people to not only use their visual senses, but also their other
auditory, tactile, and olfactory senses (Franinović & Serafin, 2013). This indicates
that natural clues are not necessarily visual only, but can also be sonic, tactile, and
olfactory.
The role of affordances has been discussed in interaction design, specifically in
variations of TEIs (Shaer & Hornecker, 2010). Affordances of interface objects in
TEIs guide the user in how to interact with the computational systems. TEIs rely on
physical affordances to facilitate interactions with computational systems, exploiting
users’ tactile experiences where possible and deriving familiarity from other domains
Embodied intuitive interaction in children 77
in the form of metaphorical gestures. Shaer & Hornecker (2010) describe epistemic
actions of Kirsh & Maglio (1994) as an act of looking for clues to interpret
affordances. Constraints are created, new affordances revealed triggerin new actions
and old affordances hidden, this could reduce the complexity of activities.
4.2.2 Perceived affordances
Perceived affordances are based on prior experience with similar things
(Blackler et al., 2010) and, contrary to physical affordances, are learned conventions
(Linderoth, 2013; Norman, 2013). For example, people have been using a
QWERTY keyboard (that maps non-spatial characters in physical space) for decades.
The quality of interfaces that use these kinds of mappings depends on people’s level
of familiarity with the mapping, and on how long they have been using it (Zigelbaum
et al., 2008). (Gaver, 1991) clarified the differences between physical and perceived
affordances, and stated that the latter are ‘often more about conventions than about
reality’.
People use clues, natural and deliberate, to connect them to their past
experiences and prior knowledge. Natural clues such as spatial orientation and
material properties link to previous knowledge of artefacts and their use. For
example stacking blocks look like tall buildings which in turn results in interactions
with the stack that are reminiscent with properties of tall buildings. In the absence
of natural clues (physical affordances), people resort to deliberate clues that are
associated with perceived affordances. Deliberate clues are those that are deliberately
inserted by the designer for specific actions on the artefact. When these actions do
not comply with users’ past experience and knowledge of an artefact, their
interactions with that artefact are non-intuitive. These deliberate clues are symbolic,
and children use their internal representations built from their past experience and
knowledge to decode them.
A popular example discussed by Gaver (1991) is that of a scrollbar on a
computer screen. It not only tells users that they can navigate through the web page
by sliding the scrollbar, but also tells them how much they have read and how much
is still left to read. The scrollbar acts like a virtual book mark to the web page. Scroll
bars have evolved over a number of years, and continue to do so. People learn about
new changes to the scrollbars by acting on them (clicking on them); this, in turn,
results in another clue, such as a double-headed arrow in a word processor. Gaver
78 Embodied intuitive interaction in children
(1991) refers to such affordances as ‘sequential in time affordances’ where one clue,
when perceived and acted upon, results in another. Similarly, affordances can be
nested in space; that is, more than one clue for perception action is available at
different places.
Embodied representations are another kind of deliberate clues recommended
by Pezzulo (2011). Embodied representations are not symbolic or linguistic; rather,
they are re-enactments of motor processes in the mind (Pezzulo, 2011). These mental
simulations trigger the same neural processes of a goal-directed action and
perception, however, without any movement and external stimuli. The mental
simulations generate a motor understanding of objects and events in the environment,
and are produced by internal modelling and re-enactment of motor processes. These
mental simulations are triggered by watching something that is in action, such as
bouncing balls. Examples of Embodied representations in design are animated icons
for email in an interface for children developed by Uden & Dix (2000), and
simulations of functionalities of objects. Uden et al. (2000) studied visual and iconic
interfaces for children to facilitate children’s Internet searches. They investigated
animated icons for email, and compared them with icons that were symbolic and
metaphoric. They discovered that children found that it was easy to understand and
recognise the animated icons, compared to the symbolic and metaphoric ones.
4.2.3 Emergence
Emergence in interaction is inspired by the process of evolution of biological
structures. In this context, emergence is the use of existing tools and features in a
machine or an organism to adapt to changes in the environment. These changes could
be either over a short period of time, or over a longer period of evolution. The field
of Robotics saw the need to develop intelligent adaptive systems that were based on
aspects of biological systems (Pfeifer, Iida, & Bongard, 2005), rather than systems
with pre-programmed behaviour. Such adaptive systems are referred to as
‘Evolutionary robotics or ‘Emergent systems’. The final structure of the system is the
result of its history of interaction with the environment.
Emergence enables system adaptation and learning, and provides an indicator
of overall progress in complex systems (Allen & Strathern, 2003). Maier & Fadel
(2009) see emergence as a dynamic property of a system that changes its structure as
people interact with it; and this, in turn, changes the interactions with the system.
Embodied intuitive interaction in children 79
Allen & Strathern (2003), however, relate emergence to knowledge that evolves as
people interact with the system; and this, in turn, changes their interactions. Thelen
& Smith (2006), while explaining learning and development in children, suggested
that acquiring knowledge and change in behaviour are inter-related. Children develop
interpretive frameworks based on what they sense and perceive from the
environment. These interpretive frameworks decide their behavioural responses.
Over time, as they develop more knowledge and understanding, they develop more
effective ways to sense and perceive and, in turn, change their interpretive
frameworks. In the meantime, if the environment (physical and social) also changes,
the interpretive frameworks also need to be updated.
Emergence as a design aspect is used in many applications, such as learning
and development, as discussed above (Thelen & Smith, 2006); organisational
management (Senge, 1987); and design sustainability (Jonker & Harmsen, 2012).
The systems in all these applications are treated as dynamic systems with emergent
properties, with a feedback loop from the output of the system feeding back to the
input of the system (Forrester, 1995). The behaviour of a system cannot be
determined from an inspection of its parts as they are connected, and keep changing
their state. This change in state could be due to feedback loops, non-linear
relationships in the system, or behaviour paths, depending on the past history of
interactions. Feedback loops are the main reasons for a system’s emergent properties.
Depending on the relationship between the outputs and inputs in the loop, systems
could develop various behavioural complexities. Jonker & Harmsen (2012), Senge
(1987), and Thelen & Smith (2006) studied emergence in systems by investigating
the feedback loops to determine the behaviour of individual parts of the system; as a
result, they developed design solutions in teaching and learning, sustainability, and
management.
In summary, emergence as an Embodied feature represents the dynamic nature
of the environment, user behaviour, and artefact properties. This dynamism creates a
complex web of design interactions and negotiations, and could be taken into account
when designing interesting interactions and experiences
4.2.4 Scaffolding
Scaffolding is the use of the environment, physical objects, tools, processes,
and support mechanisms to carry out cognitive tasks by offloading some of the tasks
80 Embodied intuitive interaction in children
into epistemic actions (Kirsh & Maglio, 1994). Epistemic actions are actions taken in
an environment with the intent of gathering information or facilitating cognition.
Epistemic actions help people to learn things that will inform their future actions, and
to perform more complex tasks (Davis, 2015).
Scaffolding frees the mind from other tasks, resulting in effortless completion
of the primary task. The use of pen and paper to solve numerical problems, and the
use of space and physical objects in solving a puzzle are some of the examples of
scaffolding. Use of physical objects (Clement, 1994) and software tools (Reiser,
2004) are common scaffolds used in education. Physical materials can be extremely
useful in enabling children to construct an essential intuitional foundation for
mathematical thinking and problem solving. For example, children are taught
abstract mathematical concepts through the use of physical objects such as dices and
cubes. For this level of intuition to develop, the physical materials must be used as
the foundation upon which children construct symbolic meanings and operational
definitions. Epistemic actions on these materials and objects allow children to
offload cognitive tasks into activities.
There is scope for investigating ways to incorporate efficient scaffolds in
products and interfaces. A help menu in a software interface, and an instruction
manual of a product, are classic examples of scaffolds that have been in use in design
for many years. Loorbach, Karreman, & Steehouder (2013) proposed the
incorporation of motivational elements in an instructional manual of a cellular phone
for senior users (aged 60–70 years). According to their findings, the use of
verification steps and personal stories in an instructional manual improve: (1)
seniors’ confidence in being able to use the phone; (2) their motivation to work with
the phone; (3) their effectiveness and efficiency in performing tasks; and (4) their
satisfaction with the phone and the manual.
4.2.5 Cooperative activity
Advances in computing and technology have resulted in recognition of the
need for co-operative activity in achieving prescribed goals. Traditionally, physical
objects have been successfully used in facilitating co-operative work (Terrenghi et
al., 2007). Physical artefacts facilitate the well-articulated division of labour (Xiao,
2005). Different people involved in different activities, with the sole purpose of
achieving the same goal, are able to integrate their contribution, access the status
Embodied intuitive interaction in children 81
information, and delegate work to the rest of the team, all at the same time. This
happens without intruding upon each other’s activities. Air traffic controllers prefer
paper flight strips over autonomous digital systems to annotate flight information and
share it with fellow air traffic controllers. Interaction with physical objects is visible
to other team members, while interaction with virtual elements in digital systems
may not be visible to all team members. This is very important in high risk, high
pressure situations such as flying an aircraft. Verbal communications form an
essential part of co-operative activity. Africano et al. (2004) found that children
engaged in verbal discussions while playing with children’s artefacts such as
puppets, trading cards, and globes for 40% of the entire activity time, and that this
facilitated co-operative activity.
An underlying objective of TEIs in interaction design has been to facilitate co-
operative activity between people with same expertise and also between people with
varied expertise and skills (multidisciplinary teams) (Shaer & Hornecker, 2010).
Physical artefacts play an important role in starting a conversation among people in a
group. Previous experience and familiarity with the artefacts from everyday life
place less demands on the users to engage with the system. Physical artefacts have
multiple interaction points in terms of action (input to the system) and multiple
observable points in terms of reaction (output from the system). This allows people
in a group to simultaneously interact with the system as they all can act on the
artefact at the same time and the outputs from the system are observed by everyone
in the group. This results in group awareness and coordination (Shaer & Hornecker,
2010). However, careful design of TEIs is required for appropriate embodied
facilitation. Jordà, Geiger, Alonso, & Kaltenbrunner (2007) designed ReacTable with
a circular shape to encourage co-operative activity. Circular shape encouraged people
to move around the table and turn the table like a Lazy Susan (Vanity Fair, 1917) is
used distribute food to people.
4.3 RELATIONSHIP BETWEEN COGNITIVE AND DESIGN ASPECTS OF EMBODIMENT
The relationship between the cognitive and design aspects of Embodiment is
shown in Figure 3. The lightly shaded (in black font) rectangular boxes represent the
cognitive aspects of Embodiment, and the dark shaded (in white font) rectangular
boxes represent the design aspects. The double arrowed lines connecting two
82 Embodied intuitive interaction in children
rectangular boxes represent the relationship between the cognitive and design
aspects.
Figure 3 Design aspects of Embodiment derived from cognitive aspects
Physical affordances and perceived affordances are related to the real time
cognitive aspect. This is because the primary objective of affordances is to offer
appropriate clues to perform actions that are performed in real time. It is not normal
to interpret clues and perform actions in the future. Physical affordances are also
related to the real world because they are associated with use of natural clues to
interpret the actions to be performed on the world. These natural clues exist only in
the real world, with real products with natural properties. Perceived affordances, on
the other hand, are not necessarily always used in the real world. Perceived
affordances are often used in virtual worlds through deliberate clues to assist in the
interpretation of actions to be performed. The dotted line in Figure 3 represents these
dual possibilities of perceived affordances in terms of linkages with the real-world
aspect. This is also in line with Dennett's (1991) view that narratives of the past and
past experiences and memories could influence the intentions of an individual.
Physical affordances are related to cognitive offloading, as actions performed on
Embodied intuitive interaction in children 83
natural physical objects free the mind from cognitive processes and tasks. Activities
performed with physical and material elements assist the information that was solely
processed in the mind.
Scaffolding, which in literal sense means offering support to a structure, is
related to cognitive offloading. Emergence is linked to the evolutionary aspect of
design as it is associated with changes to the properties of artefacts and environments
and, in turn, to the behaviours and interactions of children over a period of time. In
other words, the properties of the artefacts, environments, behaviours, and
interactions of children in time are due to the dynamic features of the artefacts and
environments which they act upon.
The relationship between cooperative activity and the social cognitive aspect is
obvious. Products are increasingly required to support children’s cooperation with
others (e.g. having children and adults involved in an activity). Research in the field
of Computer Mediated Communications (CMC) and Computer Supported Co-
operative Work (CSCW) has been carried out over three decades. However, most of
this work focussed on remote collaborations, where people in a group are dispersed
geographically or spatially (e.g. the use of Massive Open Online Courses (MOOCs)
in teaching and learning (Howarth, D’Alessandro, Johnson, & White, 2016).
However, the success of this approach is debated in the literature (Padilla Rodriguez,
Armellini, & Caceres Villalba, 2016). Children work and play in groups, and
children’s products need to support this co-operative work. It is evident from the
above discussion that the co-operative aspect of Embodiment is in line with its social
aspect.
Thus, Embodiment represents aspects of human interaction in the real world. It
includes aspects of human evolution, relating our changes in abilities and our
understanding of the world to our epistemic actions. The design aspects of
Embodiment were further investigated and examined for intuitive interaction in
Experiment 1 (Chapter 6) and Experiment 2 (Chapter 7).
4.4 SUMMARY
This chapter discussed cognitive aspects of Embodiment. These aspects are
based on the perspective of Embodiment that all cognitive processes are result of a
continuous loop of perception and action that involves brain, body and environment
84 Embodied intuitive interaction in children
playing an important role in a complex mesh. These perception action loops originate
from the environment and not the brain (Anderson, 2003; Wilson & Golonka, 2013;
Wilson, 2002). It is the stimulus from the environment that triggers perceptual
sensory systems in children. Design aspects of Embodiment (Section 4.2) were
formed from the literature on Embodied cognition. The cognitive aspects of
Embodiment were discussed (Section 4.1) and the relationship between the design
and cognitive aspects of Embodiment discussed in Section 4.3.
The design aspects of Embodiment were further examined and verified in
Experiment 1 (Chapter 6) and Experiment 2 (Chapter 7). Chapter 5 discusses the
research design for this study, its data collection and analysis methods, and the
rationale for their choice.
Embodied intuitive interaction in children 85
Chapter 5: Research Design
This chapter outlines the research design and methodology adopted by this
study to achieve the aims and objectives stated in Chapter 1. Methods used for data
collection and analysis are explained in relation to the research questions, with a
focus on the rationale for the choice of these methods.
Section 5.1 discusses the study’s methodology and research design, briefly
describing the two experiments conducted; Section 5.2 details the participants in the
study; Section 5.3 discusses the data collection methods, and justifies their choice;
Section 5.4 outlines the various data analysis tools and methods and explains how the
results were integrated to report the findings and conclusions.
5.1 METHODOLOGY
The objective of this research was to investigate the role of Embodiment in
children’s intuitive interaction. Physical products, virtual interfaces, and TEIs were
studied for children’s Embodied intuitive interaction with them. The research
questions that formed the basis for the research design were:
What is the role of Embodiment in children’s intuitive interaction?
What are the aspects of Embodiment?
To what extent do the design aspects of Embodiment facilitate intuitive interaction
in children?
How can the design aspects of Embodiment facilitate children’s intuitive
interaction?
A mixed methods approach to research design was used. Mixed methods
research uses a combination of qualitative and quantitative approaches to data
collection, analysis, and interpretation of findings. Empirical research that uses
mixed methods either focusses on collecting and analysing two types of data
(qualitative and quantitative), or on integrating its qualitative and quantitative
findings (Tashakkori & Creswell, 2007; Zhang & Creswell, 2013). Thus, in an effort
to be inclusive of both qualitative and quantitative approaches, Tashakkori &
Creswell (2007) defined mixed methods research as “research in which the
86 Embodied intuitive interaction in children
investigator collects and analyses data, integrates the findings and draws inferences
using both qualitative and quantitative approaches” (p. 3).
Mixed methods research draws upon the strengths of each of the qualitative
and quantitative approaches and was particularly useful for this thesis as it offered
various perspectives on the study of the complex phenomena of Embodiment and
intuitive interaction. Integration of qualitative and quantitative findings at some stage
of the research process (i.e. at the data collection, analysis, or interpretative stage) is
a critical characteristic of mixed methods (Kroll & Neri, 2009).
In this study, qualitative data were collected through observations, co-
discovery methods, and retrospective interviews. Thematic analysis was performed
on the qualitative data, which was then used to generate quantitative measures. The
quantitative measures were then statistically analysed. The findings from the
thematic and statistical analyses were integrated in the interpretative stage of the
research process.
The research design consisted of two sequential experiments, the outcomes of
Experiment 1 informing the design of Experiment 2. Both the experiments involved
children as participants. Children have been involved in previous research studies at
different levels. Wang (2015) identified that traditional studies involving children,
especially in psychology, were mostly of the non-contact type. Data was collected
from administrative records and from people (such as parents, teachers, and carers)
who were in contact with children. In recent years, however, children have started
playing more of a participatory role in research. Druin (2002) and others have
considered children’s direct participation in the design process of technology
specifically designed for them. Liamputtong (2006) found that direct contact research
mostly involves self-completion questionnaires, interviews, and group discussions.
Furthermore, observations have been mostly limited to non-interactive data
gathering. In contrast, children were participants in this study, playing with toys with
another child (and, in the first study, with the researcher).
Ethics approval using the National Ethics Application Form (NEAF) was
sought from the Human Research Ethics Committee (HREC) at QUT prior to any
data collection (including the pilot studies). An approval from Education
Queensland was also obtained to conduct study at a local state school. The ethics
approval from HREC, participant information sheets, consent forms and approval
Embodied intuitive interaction in children 87
from Education Queensland is placed in Appendix (Appendices F-O). The structure
of the entire study, with details regarding each experiment, is shown in Figure 4.
Both the experiments were conducted in a similar way, with minor changes
made to Experiment 2, based on the findings from Experiment 1. Both experiments
involved pairs of children playing with real toys. Trial experiments with children of
friends and family were carried out prior to Experiment 1 to decide on: how children
should be paired; the selection of appropriate toys for the experiment; the
combination of data collection methods; and protocols for communicating with the
children. To cover the social aspect of Embodiment in the investigation, children
were observed playing a game with another child. The fact that children were playing
with another child whom they knew before the experiment also assisted in making
them feel comfortable during the study. This was important for the study to bring out
natural interactions and behaviours during the play. Experiments were audio and
video recorded for analysis.
5.1.1 Experiment 1
Experiment 1 was a between subjects study that investigated the aspects of
Embodiment that facilitate intuitive interactions with physical products and virtual
interfaces. The objective of the experiment was to determine the role of Embodiment
in children’s intuitive interaction with a directly manipulated interface and a virtual
interface. Their intuitive interaction with, and aspects of Embodiment that facilitated
their intuitive interaction with a physical product and an equivalent virtual interface,
were compared. A summary of Experiment 1 is presented in Table 2.
Observations that involved both children and the researcher in game play were
carried out in Experiment 1. Children who knew each other outside the experimental
setting and were around the same age (or from the same class or grade at school)
were paired for the experiment. Both the children played as a team against the
researcher. Observations were followed up with retrospective interviews, without any
concurrent protocols. Both observations and retrospective interviews were audio and
video recorded for analysis. Experiment 1 was carried out at a local state school in
Brisbane (Australia), and in the People and Systems Lab (PASP) at Queensland
University of Technology (QUT).
88 Embodied intuitive interaction in children
Figure 4 Research Design to investigate children’s Embodied intuitive interaction
Embodied intuitive interaction in children 89
Table 2 Summary of Experiment 1
Experiment 1 compared intuitive interaction and Embodiment between
physical product and virtual interface. A between subjects study was more
appropriate for this type of study as it allowed the experiment to be conducted with
two independent groups of participants without the carryover effects; that is,
participation in playing with one toy does not affect performance in playing with the
second toy. Practice or fatigue while playing with one toy, for example, could either
enhance or stifle performance in playing with the second toy. This results in a
confounding variable that can vary with the independent variable (Gray and Kinnear,
2012, p. 316).
5.1.2 Experiment 2
Experiment 2 was a within subject study that investigated ways in which
Embodiment can facilitate children’s intuitive interaction when playing with directly
manipulated interfaces. A summary of Experiment 2 is presented in Table 3.
With the exception of a few changes, the experiment was carried out in the
same way as Experiment 1. A Within subjects study design was used in Experiment
2, and each pair of children played with two directly manipulated interfaces—one
was a TEI, and the other a physical product. The TEI consisted of a physical space
and a virtual space, and involved direct manipulation and interaction with the
physical elements in the physical space. Data collected for each of the toys were
analysed separately (not compared with each other as in Experiment 1). A between
subjects study requires a large sample size. Recruitment of a large number of
participants, data collection and data analysis for this large sample is time
Experiment 1 Research Question Can Embodiment facilitate intuitive
interaction in children?To what extent do the design aspects of Embodiment facilitate intuitive interaction in children?
Setting QUT People and Systems Lab and Queensland State Schools
Experiment Design Between subjects Data Collection Methods Observations, Retrospective Interviews,
Co-discovery Data Analysis Thematic and statistical analysis Population School children Sample/Participants 108 (54 pairs) children aged 5–11 years
90 Embodied intuitive interaction in children
consuming. Within subject designs, on the other hand, require smaller sample size
for approximately same effect size and statistical power (Section 5.2.1 and Section
5.2.2). A smaller sample size means participant recruitment, data collection and
analysis comparatively takes less time. With significant amount of time spent on
Experiment 1, a within subjects design was used for Experiment 2.
Table 3 Summary of Experiment 2
Within-subject designs suffer from carry-over effects, Gray and Kinnear (2012)
suggested using counter-balancing to reduce the possibility of carry-over effects
confounding the effects of the game play. Thus, the order in which a child pair
played each of the games was varied from one observation to another, ensuring that
there were equal numbers of pairs playing each game first. This way, any carry-over
effects balanced out across the two game plays.
Since every child played with two toys, retrospective interviews were not
carried out after Experiment 2 in order to keep the experiment time reasonable for
the participants. Pilot studies performed prior to Experiment 2 revealed that children,
especially younger children, got tired and lost focus while playing for a lengthy
period. Furthermore, communication between the children during the observations
generated enough verbal data for analysis.
Again, children were paired if they knew each other outside the experimental
setting. However, unlike Experiment 1, they were not necessarily of the same age.
Children normally play with other children who may not be of same age such as
siblings, play mates outside the school environment such as at a sports club. Children
of same age were paired for Experiment 1 as it was a comparative study and the
Experiment 2
Research Question - Which design aspects of Embodiment facilitate children’s intuitive interaction - To what extent do the design aspects of Embodiment facilitate intuitive interaction in children?
Setting QUT People and Systems Lab
Experimental Design Within subject
Data Collection Methods Observations and co-discovery
Data Analysis Thematic and statistical analysis
Population School children
Sample/Participants 42 (21 pairs) children aged 5–12 years
Embodied intuitive interaction in children 91
difference in ages could have skewed the results. Intergenerational research
involving participants from different age groups has been successfully used in the
Social Sciences (Grenier, 2007), and in the design of technology products for
children (Guha, Druin, & Fails, 2013). However, that research often focussed on the
role of adults (over 50 years of age) as mentors to the children. In contrast, in order
to mimic natural play in children in Experiment 2, children were paired so that they
were not more than 4 years apart in age, and were paired with a sibling or a friend.
Experiment 2 was carried out at the PAS Lab, and participants were recruited
through advertisements in local state schools.
5.2 PARTICIPANTS
Children in the 5–12 age group were selected from a pool of volunteers from
local state schools in Brisbane (Australia), and through personal contacts. These
volunteers were recruited through school newsletters and email advertisements sent
out to email groups at QUT. Parents/guardians/carers and children filled out consent
forms prior to the experiment. Every participant was provided with a gift card or a
toy as a gesture of thanks for their participation. Children were asked to choose a toy
from a reward basket at the end of Experiment 1, and gift cards were given to the
parents in Experiment 2.
Children usually start school in Queensland, Australia at the age of 5-6 years
(Prep) and make a transition to middle school at the age of 12 years (Grade 6). This
transition brings changes in children’s behaviour (Heyns, 1987) and how they
approach a given problem and the way they process information cognitively (Rutter,
1985). Perry, Church, & Goldin-Meadow (1988) discussed how these transitional
stages bring a change in the acquisition of concepts. This explains the choice of the
age range 5 years to 12 years for the study and it was important to include the years
of the transitional stages in the study. Prep and Grade 6 represent a transitional
change in knowledge and concept acquisition in children in Queensland schools.
This explains the large age range (5 years – 12 years) in this study.
The number of children recruited for the study depended on the design of the
experiment (between-subjects in Experiment 1, and within-subjects in Experiment 2),
and statistical tests employed for quantitative analysis (parametric or non-parametric,
independent groups or matched pairs, etc.). A high statistical power meant that the
92 Embodied intuitive interaction in children
probability of making a Type II error, or concluding that there is no effect when there
is an effect present, is less. For Experiment 1 and Experiment 2 a power analysis was
used to determine the prospective statistical power of the experiment design and thus
the implied probability of making a Type II error. The probability of a statistically
significant result is high if the effect size (f2), sample size (N) and the significance
criterion (α) are high.
Optimal statistical power chosen for the study was 0.8. A statistical power of
0.8 means that the probability of rejecting a false null hypothesis is 0.8 or 80% or we
run a 20% chance of making a Type II error. Significance criterion, α = 0.05 has
been used in this study for all the analysis, including power analysis. G*Power, a
software for statistical power analysis (Faul, Erdfelder, Buchner, & Lang, 2009) was
used to estimate effect size and statistical power for Experiment 1 and Experiment 2.
5.2.1 Power analysis for Experiment 1
G*Power was configured for non-parametric between subject test for two
independent groups. Power analysis was run with the following parameters:
Initial Sample size (N1) for group1 – children playing with physical toy: 50
Initial Sample size (N1) for group2 – children playing with virtual toy: 50
α = 0.05
Estimated lowest effect size using G*Power from the mean values of the
dependent variables in Experiment 1 (discussed in Section 6.2.2), f2 = 0.7
Corresponding Power = 0.937
XY plot for computed power for sample size varying from N = 0 to 100 was
plotted to assist in the final selection of the sample size (N) (Figure 5). The statistical
power for an initial sample size of 50 for each of the independent groups, N1 = N2 =
50, was greater than the optimal power of 0.8, so the sample size was considered
appropriate for Experiment 1. The total sample size selected for Experiment 1 was
108, N1 = 56 and N2 = 52, contributing to a statistical power of 0.945.
5.2.2 Power analysis for Experiment 2
G*Power was configured for non-parametric within subject test for matched
pairs (pairs of children playing with both the toys). Power analysis was run with the
following parameters:
Embodied intuitive interaction in children 93
Initial Sample size (N): 50
α = 0.05
Estimated lowest effect size using G*Power from the mean values of the
dependent variables in Experiment 2 (discussed in Section 7.2.2), f2 = 0.52
Corresponding Power = 0.945
XY plot for computed power for sample size varying from N= 0 to N= 100 was
plotted to assist in the final selection of the sample size (N) (Figure 6).
Figure 5 XY plot of power for a range of sample size (N) value, 1 to 100
Figure 6 XY plot of power for a range of sample size (N) value, 1 to 100
The statistical power for an initial sample size of N=50 was greater than the
optimal power of 0.8, the sample size was considered appropriate for Experiment 2.
94 Embodied intuitive interaction in children
The total sample size selected for Experiment 2 was 42, contributing to a statistical
power of 0.925
5.3 DATA COLLECTION METHODS
Data were collected using observations, co-discovery, and retrospective
interviews.
5.3.1 Observations
The objective of the study was to study intuitive interactions in children,
interactions that are Embodied in their everyday life and activities. Physical artefacts
are mediators between the agent and their activity of manipulating these artefacts
(Nardi, 1996). This manipulation of artefacts brings out natural behaviour in the
agent (Engeström, Miettinen, & Punamäki, 1999). Thus, artefacts can be used to
study people’s complex behaviours (Popovic, 2003), as they result in human
activities that are representative of user goals. Thus, both experiments in this
research study involved performing activities with an everyday artefact.
Play as an activity comes naturally to children, and is part of their daily
activities. Thus, playing with toys with other children formed the basis of the
experiments to investigate children’s Embodied intuitive interaction. Children learn
how to use different features in a product by exploring and tinkering with it
(Blikstein, 2013). Adults, on the other hand, create a step by step strategy in their
minds, even before they begin a task or touch a product (Fisk, Rogers, Charness,
Czaja, & Sharit, 2009). Thus, children were given the holistic task of playing a game;
this allowed them to explore and invent new strategies of game play.
Children’s play was audio and video recorded for thematic and quantitative
analysis. Two video cameras were used to record the observations. One camera
captured the front view of the children playing with the toy. The second camera
either captured the screen view of the app on the tablet, or the side view of the play
with the physical toy (depending on whether there was a tablet involved in the play).
5.3.2 Co-discovery
Observations alone might not provide insight into the decision making process
involved in children’s intuitive thinking. Thus, observations were complemented
with verbal protocols to capture information that could not be seen externally.
Embodied intuitive interaction in children 95
Concurrent protocols (also known as ‘Think aloud’ and ‘Talk aloud’ verbal
protocols) have been successfully used in empirical studies of intuitive interaction as
they offer an immediate account of thoughts and actions either without, or with
minimal prompts (Blackler et al., 2010). Concurrent protocols, however, can be
difficult for children when they are interacting with systems that require a heavy
cognitive load (Höysniemi, Hämäläinen, & Turkki, 2003). Children’s cognitive
abilities are not fully developed until their late teens, and this influences their ability
to think and talk out loud. For example, they have a limited ability to think
simultaneously about more than one concept, such as playing with a toy and thinking
aloud at the same time.
Concurrent protocols make the conditions in an experiment unnatural, and are
difficult to facilitate in a field setting (such as schools) or in a collaborative context
(Hertzum, 2016). Trial experiments with children of friends and family revealed that
although instructed to think and talk aloud, they remained engrossed in the play.
They were not clear about what to do when they were instructed to think or talk
aloud. When prompted to verbalise their feelings and thoughts, they stopped playing
and talked. They also talked about things that were irrelevant to the game. When they
went back to play, their game strategies were different to those they employed before
the interruption. They took longer to finish the game, reacting slowly to various
stimuli (visual and haptic). They also spent a lot more time in distributed visual
behaviour, looking for something to talk about. This confirmed that concurrent
protocols affect children’s performance of spatial tasks (Gilhooly, Fioratou, &
Henretty, 2010). This means that it is difficult to use them in the context of play.
Considering the above problems with concurrent protocols, co-discovery was
used as a verbal protocol. Each child participant was paired with another to solicit in-
depth information and free-flowing discussion between them (Adebesin, De Villiers,
& Ssemugabi, 2009; Kemp & Van Gelderen, 1996). The verbalisation of ongoing
thought processes provided direct insights into the knowledge and methods used
during the play (Popovic, Kraal, Blackler, & Chamorro-Koc, 2012). This verbal
discussion between participants while performing a task is referred to as ‘co-
discovery’ (Lim, Ward, & Benbasat, 1997).
Co-discovery encouraged children to question each other and to engage in
deeper discussions and explanations; that is, to consider the why and how of the task.
96 Embodied intuitive interaction in children
It ensured that the researcher’s presence did not influence the children’s behaviour.
Co-discovery provided a platform for the children to discuss and question every
strategy in the game; this reflected not only what they were thinking, but also why
they were thinking it. This natural verbal communication between the children
provided rich verbal data that represented the internal cognitive processes that were
mapped into their interactions with the game system. They brought their own
experiences and knowledge into the discussion. The two sets of knowledge and
experiences provided a critical perspective on the understanding of the children’s
interactions.
The success of co-discovery depends largely on how participants are paired.
The level of expertise between participants is crucial for pairing them in co-
discovery (O’Malley, Draper, & Riley, 1984). Nielsen (1994) favoured pairing
participants with the same level of expertise in co-discovery so that expertise of any
one participant does not influence the outcomes of the study. Kahler, Kensing, &
Muller (2000), on the other hand, suggested pairing participants with different levels
of expertise, thus enabling one person to guide the interaction. However, pairing
children with different levels of experience could result in a child with higher levels
of experience in taking over the game play. Level of acquaintance is another
important element that plays an important role in pairing participants in co-discovery
(Als, Jensen, & Skov, 2005). Children often behave differently depending on how
well they know each other. Rather than expertise, level of acquaintance was more
important for this research study as it made children comfortable and encouraged
natural behaviour. Children who knew each other outside the research study - as
friends, classmates, or siblings - were paired for the experiments in this research.
Experiment 1 was mostly conducted at a local school in Brisbane, and thus
children from the same class or grade were paired together. This ensured that
children were acquainted with each other. This also made sure that children with the
same level of expertise were paired. The school where the data was collected
introduces the physical toy and the app to students in Prep. Pairing children from the
same grade (and who have been introduced to both the toys in Prep) ensured that
both the children had the same level of familiarity. Experiment 2, on the other hand,
was conducted at People and Systems Lab (PAS Lab). Children and their parents
were invited to participate in the study and were asked to bring along a friend or a
Embodied intuitive interaction in children 97
sibling whose age was not more or less than the child’s age by 4 years. This firstly
ensured that the children knew each other outside the study. Secondly, the
requirement of age difference not more than 4 years ensured that none of the children
in the pair were overly expert compared to their playmate.
Gorriz & Medina (2000) suggested that gender influences pairing in co-
discovery. Girls typically have different preferences than boys when it comes to
playing with toys. However, this has been debated by many who suggest that a
preference for specific type of toys depends on the influence of family members who
direct children towards specific types of toys and activities (Alexander, 2003).
Children also change their preferences depending on their exposure to specific toys.
Caldera, Huston, & O’Brien (1989) further suggested that toys and games can be
designed to be gender neutral and thus gender was not considered as criteria for
pairing children.
5.3.3 Retrospective Interviews
Retrospective interviews were carried out after the game play in Experiment 1.
Children were shown the video of their game play, and the verbal discussions
between them were recorded for analysis. However, they were neither instructed nor
prompted to think or talk aloud at any time. Co-discovery was used as a verbal
protocol in retrospective interviews, as it was in the observations.
Conscious events immediately fade in memory, and children might not
remember the details of game play during the interview (Popovic et al., 2012). They
were thus shown the audio and video recording of their play on a monitor screen
immediately after their play session. They talked about their play, and about why and
how they made their game decisions. They also asked their experiment partners
questions about the game. Researcher intervention in terms of prompts or questions
was required only when there was absolute silence; however, this was very rare.
Retrospective interviews were only carried out in Experiment 1. They were
video recorded, using two cameras. One video camera recorded the front view of the
children sitting together and discussing their play. The second video recorded the
view of the screen monitor on which they were shown the game play.
98 Embodied intuitive interaction in children
5.4 ANALYSIS
The audio and video recordings of observations and retrospective interviews
were analysed in three steps. The data was first coded for Embodiment and intuitive
interaction, using thematic analysis. Coded data was then exported to Excel for
quantitative analysis, using SPSS and STATA.
5.4.1 Thematic analysis
Thematic analysis was used to identify, analyse, and report patterns (referred to
as ‘themes’) within the data. A deductive approach to thematic analysis was used.
The analysis was carried out in Noldus Observer XT (Version 12) (Noldus, 1989),
where two videos from the observations and two videos from the retrospective
interviews (i.e. four videos) were all synchronised and analysed. This linking of the
verbal discussions (captured through co-discovery in observations) and the
retrospective interviews to the actions performed on the toys, allowed a better
understanding of the children’s interactions and behaviour.
Two groups of themes, and their corresponding sub-themes, identified based on
the research question and the literature review, are shown in Figure 7. Thematic
analysis was carried out in four stages to identify the interactions and behaviours that
were to be coded, and the heuristics for the themes. ‘Heuristics’ refers to strategies
derived from experience-based techniques for problem solving, learning, and
discovery (Pearl, 1984). The heuristics for this research study partly came from the
literature review, and partly from the data. This was important in applying the
heuristics to the context of the data. It was necessary to move back and forth
throughout the four stages of thematic analysis.
Data familiarisation: Direct involvement in the data collection process
provided some initial knowledge about the characteristics of the data. Videos of
observations and retrospective interviews were watched multiple times to become
familiar with them, and patterns of interest emerged as a result. Interesting
interactions and behaviours of the children relevant to the research questions, and
quotes from the verbal interactions between them were noted. This familiarisation
and notetaking was followed by a formal process of coding.
Initial coding stage: Initial codes for the children’s behaviour, and their
interactions with the toys and their partners, were generated in Observer XT. Extracts
Embodied intuitive interaction in children 99
of data were coded inclusively. The initial codes were in the form of comments,
quotes, and words relevant to the extract of data being coded. The heuristics for
coding interactions and themes began to emerge from the initial codes.
Formative stage: Initial codes that represented common themes were grouped.
Some initial codes formed the interactions; some formed the themes; and some did
not fit into either category. These were categorised as ‘others’. None of the initial
codes were discarded at this stage.
Review stage: In this review stage, the themes identified in the previous stage,
and the relationship between the themes and interactions were reviewed for
coherency, overlapping themes, missing behaviours or interactions, and correctness
of the codes. The heuristics were also finalised in this stage.
Figure 7 Coding scheme showing theme groups and the corresponding sub-themes
Definition stage: This was the final stage, where the coded data were reviewed
again, and the heuristics for the interactions and themes were finally clearly defined
in the context of the games played within each of the experiments.
100 Embodied intuitive interaction in children
The coding heuristics for Types of interaction and Aspects of Embodiment are
discussed below.
Coding Heuristics for Types of interaction
There were three themes within the theme group, types of interaction: intuitive,
non-intuitive, and partially intuitive. These themes were both mutually exclusive and
exhaustive.
Intuitive Interaction involves utilising knowledge gained through other
experience(s); it is fast, and generally unconscious (Blackler et al., 2010). The coding
heuristics employed to code for the intuitive interactions are based on this definition,
and are derived from methods developed by Blackler (2008). The main indicators of
intuitive interaction that were used to decide on the types of interaction during the
coding process are explained below. Two of these conditions needed to be met for a
behaviour to be coded as ‘intuitive’.
Unconscious Reasoning- Intuitive interaction is associated with processes that
yield little or no conscious experiences (Bastick, 1982; Blackler, 2008; Dane & Pratt,
2007; Myers, 2007). This means that if children demonstrated less reasoning while
playing with the toy, they could be interacting with it intuitively. Unconscious
reasoning is associated with mental states that are not verbally reportable (Chalmers,
1995). People are unable to access their internal mental states to verbally report
about them. Baars, Ramsøy, & Laureys (2003) studied the activity in the human
brain during conscious and unconscious mental states. They reported that
behaviourally conscious states demonstrated accurate reportability of attended
stimuli, orientation to space, time and self, visual images, inner speech, abstract
thoughts, and control of voluntary muscles. Unconscious states, on the other hand,
did not demonstrate any accurate reportability. In terms of brain activity, low
regional metabolism was observed in the frontoparietal cortex in unconscious mental
states. On the other hand, high regional metabolism was recorded in the
frontoparietal cortex in conscious mental states. The frontoparietal cortex is the
“…narrative interpreter of the speaking hemisphere…” (Baars et al., 2003, p. 673) of
the brain that is responsible for the reportability of the stimuli and events.
Embodied intuitive interaction in children 101
Children who were unable to accurately explain their game play—that is why
and how they decided on their game strategies—were considered to be in a
nonconscious mental state, processing information intuitively.
Degree of Certainty- Intuitive interaction is associated with high degrees of
certainty, confidence, and expectation with respect to the correct use of a feature
(Blackler, 2008; Hammond, 1993; Simmons & Nelson, 2006). Play is usually not
associated with correctness or incorrectness. Thus, when participants were certain
and confident about their decision and their interaction with the toy, the behaviour
could be coded as ‘intuitive’.
Certainty and confidence in children’s play was evident from their interaction
with the toys. Children would not hesitate, and their interactions would be
immediate. Certainty and confidence was evident from their facial expressions and
body movements. Ekman, Friesen, & Ellsworth (2013) provided guidelines to study
emotions and behaviours in people based on their facial expressions. They described
findings from experiments performed by Jones (1971), Ekman (2006) and Hulin &
Katz (1935). Of particular relevance are findings of Jones (1971) who investigated
facial expressions in children and tabulated behaviours and meanings for singular,
and combinations of expressions. Jones suggested that expressions such as biting
fingers, frowning, raising upper brows, and closing eyes represented uncertainty and
lack of confidence in carrying out a given task. Mahmoud & Robinson (2011)
studied gestures in children, and suggested that folding hands, rubbing hands
together, twitching fingers, and placing hands against their face are characteristics of
uncertain behaviour. Verbal conversations in co-discovery with a partner could also
provide indications of uncertain behaviour.
Fast decision making- Intuitive decision making is associated with fast
decisions (Plessner, Betsch, & Betsch, 2011). If a participant made a decision within
five seconds, the decision was considered to be fast, irrespective of whether it
resulted in successful play or not. However, a fast decision was coded as ‘intuitive
interaction’ only if the decision was taken with a degree of certainty and unconscious
reasoning.
Non-intuitive interactions are associated with decisions that are made
consciously. This means that if children were reasoning enough while playing with
the toy, they could be interacting with it non-intuitively. Conscious reasoning is
102 Embodied intuitive interaction in children
associated with mental states that are verbally reportable (Chalmers, 1995).
Behaviourally conscious states demonstrate accurate reportability of what trigged an
action, elements in the environment such as images, objects and their orientation in
space, mental thoughts, abstract concepts and control of their physical body (Baars et
al., 2003).
Non-intuitive interactions are associated with high degrees of uncertainty and
lack of confidence (Hammond, 1993; Simmons & Nelson, 2006). Thus, when
participants were uncertain and lacked confidence about their decision and their
interaction with the toy, the behaviour could be coded as ‘non-intuitive’. Evidence of
hesitation in their interactions with the toys was used to determine uncertain
behaviour. Ekman, Friesen, & Ellsworth's (2013) guidelines to determine level of
uncertainty and confidence from facial expressions and body movements were used
to determine non-intuitive interactions. Trial and error such as random playing with
physical objects, random tapping and swiping on touch screen interfaces is
associated with uncertainty and lack of confidence (Chin, Diehl, & Norman, 1988).
Thus, trial and error interactions with the toys were coded as non-intuitive
interactions.
Partial intuitive interactions: Children’s game play at times consisted of
interactions that showed signs of both intuitive and non-intuitive interactions.
Children could for example figure out the use of certain colour coded objects in the
game unconsciously and quickly but at the same time demonstrate uncertainty in the
use of some other colour codes. Such behaviour was coded as ‘partial intuitive
interaction’.
Coding Heuristics for aspects of Embodiment
The themes identified within the theme group Aspects of Embodiment were
physical affordances, perceived affordances, emergence, scaffolding, and co-
operative activity. These themes are the design aspects of Embodiment discussed in
Section 4.2. These themes were not mutually exclusive, and were non-exhaustive.
Physical affordances: posit that people directly perceive and act in the
environment through the ability of their perceptual sensory systems to detect
information about the environment. Physical affordance is a way of deriving
information from the environment when the body and environment are involved in a
Embodied intuitive interaction in children 103
perception action loop. People use natural clues from the environment such as spatial
and material properties of objects to derive this information from the environment.
These natural clues determine and constrain what can be done with the elements in
the environment. When children used natural clues from the objects in the
environment to play with the toys, the behaviour was coded as ‘physical
affordances’.
Perceived affordances: posit that people use information from their past
experience and prior knowledge to decide on actions to be performed on the
environment. This is in contradiction to physical affordances as the perceptual
sensory systems are completely removed from the process. While designing products
that tap on people’s past experience and prior knowledge, deliberate clues are
incorporated in the design of products. People use these deliberate clues to make
connections between the problem at hand and their knowledge and experience. As
discussed in Section 4.2, there are two types of deliberate clues, symbolic clues and
embodied representations. Symbolic clues are the conventional ones which have
been used in design for decades. They represent symbols in the form of language,
semiotics, aesthetics, cultural representations, etc. Embodied representations are
clues in the form of actions that offer meaning to the object or the design element.
When children used deliberate clues to play with the toys, the behaviour was coded
as ‘perceived affordances’.
Emergence: is a dynamic property of a system that changes its structure and/or
behaviour as people interact with it; and this, in turn, changes the interactions with
the system (Maier & Fadel, 2009). As the system and interactions adapt with each
other, people learn and develop knowledge about the dynamic system (Allen &
Strathern, 2003). Interactions and behaviours which led to changes to the structure of
the game or the toy resulting in changes to game strategies, were coded as
‘Emergence’.
Scaffolding: is the use of environment, physical objects, tools, processes, and
support mechanisms to perform cognitive tasks by offloading some of the tasks into
epistemic actions (Kirsh & Maglio, 1994). When children were using elements from
the environment such as physical objects, processes (such as imitation) and support
mechanisms (help menu for example), the behaviours were coded as ‘scaffolding’.
104 Embodied intuitive interaction in children
Co-operative activity: is the idea of working together to reach a particular
goal. This could be through use of physical objects or through verbal communication
between children. It is known fact that actions performed on the environment by a
person alone are different to actions performed by a group (Williams, Kabisch, &
Dourish, 2005). It is this behaviour of working together in an attempt to reach a goal
was coded as Co-operative activity.
5.4.2 Reliability analysis
A reliability analysis of the coded data was required to verify that the coding
was repeatable. The data were coded twice by the same researcher, with a break of
15 days between the two sessions. Reliability analysis was carried out on the two
versions of coded data in SPSS.
The strength of agreement between the two versions of coding was reported
through Intraclass Coefficients (ICC) for reliability which can range from 0 to 1. Koo
& Li's (2016) guidelines (Appendix A) were used to assess the strength of
agreement.
5.4.3 Quantitative analysis
All coded data were exported to Excel for analysis. While SPSS was primarily
used for statistical analysis, STATA was also used to determine sample size (that is,
the number of participants) for both experiments, and to assess the pre-requisites for
the regression analysis methods used in Experiment 2.
This research study used statistical methods to determine if differences
between groups existed, and to predict the variability in intuitive interaction resulting
from the design aspects of Embodiment. Descriptive statistics were first used to
summarise data within individual groups in such a way that patterns could emerge.
The results were reported in the form of tables and graphs (i.e. bar graph and box
plot).
The significance of the differences between the two groups was determined by
the use of inferential statistics. The quantitative data in Experiment 1 and Experiment
2 did not have the requisites for parametric inferential methods; thus, non-parametric
methods for significance testing were used. The Mann Whitney U test was used for
Experiment 1 and the Friedman test for Experiment 2. The non-parametric inferential
statistical test only tells if the differences between the groups are significant. Thus,
Embodied intuitive interaction in children 105
effect size (also called ‘Cohen’s d value’) was reported to tell the size of the
difference. Cohen’s (1992) guidelines (Appendix B) were used to determine the size
of the difference.
Multiple regression analysis was used in Experiment 2 to explain how much of
the variation in intuitive interaction could be explained by the aspects of
Embodiment (i.e. predictors): physical affordances, perceived affordances,
emergence, scaffolding, and cooperative activity. The objective was to investigate
the relationships between intuitive interaction and these aspects. Multiple regression
analysis requires observations to be independent, and data to be free of
multicollinearity (Draper & Smith, 2014). The within-subject study design of
Experiment 2, where each child played with two toys, resulted in observations that
were dependent on each other. This is largely a study design issue. Separate linear
regression models were created for data collected from children’s interactions with
the two directly manipulated interfaces.
Multiple regression analysis requires that the data should not show
multicollinearity (Draper & Smith, 2014), which occurs when two or more predictors
are highly correlated. Since there were more than three predictors, Variation Inflation
Factor (VIF) was considered more reliable than pairwise correlations to determine
multicollinearity in the data (Kutner, Nachtsheim, & Neter, 2004). VIF estimates
inflation in a regression coefficient for a predictor because of the linear dependence
on other predictors.
VIF values have a lower bound of 1, and no upper bound. A value of 1 means
there is no correlation between the predictors in regression analysis. VIF values
between 1 and 5 indicate little correlation between the independent
variables/predictors. VIF values between 5 and 10 mean there is moderate
correlation, and values greater than 10 mean there is a very high correlation between
the predictors.
Cohen et al. (2013, p. 419) suggested the use of ridge regressions to tackle
multicollinearity issues. Multicollinearity results in least square estimates that are
unbiased, and large variances that might be far from the true value. Ridge regression
adds a degree of bias to the regression estimates, thus reducing the standard errors
and resulting in reliable regression estimates. Separate ridge regression models were
created for data collected from children’s interactions with the two directly
106 Embodied intuitive interaction in children
manipulated interfaces used in Experiment 2. The regression coefficients, t-values,
and significance values were then compared with the results from the linear
regression results to determine the impact of multicollinearity on the results. STATA
statistical software was used to run Ridge regressions, as SPSS does not support it.
Ridge regressions generated robust standard errors, with no significant
differences between the linear regression and ridge regression results in terms of the
regression coefficients, t-values, and significance. As a result, it was concluded that
although the linear regression models showed multicollinearity, it had no appreciable
effects on the linear regression statistics. Thus, the statistics derived from the linear
regression model were used to discuss the results of the study (Chapter 7).
5.4.4 Integration of findings
Integration of the findings of this research was carried out on two levels: the
first at the experimental level, where findings from the qualitative and quantitative
analyses were integrated and reported; the second, where findings from Experiment 1
and Experiment 2 were integrated.
Triangulation was used to integrate qualitative and quantitative findings at the
experimental level. Erzberger & Kelle (2003) used the term ‘triangulation’ as a
metaphor to represent the process of integration, linking theoretical propositions and
the research findings. Considering the deductive nature of its analytical approach, the
triangulation representation of the integration process was deemed suitable for this
study. It has provided a greater understanding of the links between theory and
empirical findings, and aided the development of an additional theoretical
proposition. The triangulation involved assessing the logical relationships between
theoretical propositions and the findings.
The findings and theoretical propositions in Experiment 1 and Experiment 2
were collated to represent the overall findings in a Model for Embodied Intuitive
Interaction (MEII) (discussed in Section 8.2). The focus of this study was on the
design of interactions, rather than the design of products. Thus, the outcome is MEII:
an interaction model that will facilitate the design of products with Embodied
intuitive interactions. The findings from Experiment 1 and Experiment 2 were
integrated into the continuum of intuitive interaction (Blackler, 2008) resulting in an
Enhanced Framework for intuitive interaction (EFII) (discussed in Section 8.1.2).
Embodied intuitive interaction in children 107
Buur & Andreasen (1989) described design models as representations of the
properties of objects that designers create. These representations provide insights into
the properties of the product that is being designed. Interaction models also
represent ways in which interactions with certain properties such as intuitiveness
(Blackler & Popovic, 2016), Embodiment (Dourish, 2001), thoughtfulness (Löwgren
& Stolterman, 2004), and seductiveness can result in the design of playful and
effective user experiences (Anderson, 2011). Interaction models take various forms
such as mathematical formulae, verbal descriptions, sketches, block diagrams, and
functional models, depending on the expertise of the designer and the level of
abstraction of the design process. An interaction model was developed in this
research study and the model for Embodied intuitive interaction (MEII) is presented
in the form of a block diagram (and discussed in Section 8.2).
5.5 SUMMARY
Chapter 5 has described the research design and methods used to collect and
analyse the data. It has also discussed the rationale behind the use of the methods and
techniques used for data collection and analysis for the research topic and user group
under investigation.
Two empirical experiments were conducted. Experiment 1 investigated
intuitive interaction with a physical product and an equivalent virtual interface, and
studied aspects of Embodiment in both of these artefacts. Experiment 2 studied the
impact of each of the aspects of Embodiment on intuitive interaction with a physical
product and a TEI. The experiments were conducted at a local Brisbane (Australia)
state school, and in QUT’s PAS Lab.
Qualitative data in the form of audio and video recordings of observations and
retrospective interviews were collected. The data was analysed using qualitative
thematic analysis, followed by quantitative statistical analysis. The findings were
triangulated with the theoretical proposition (derived from the literature review). The
chapter has also explained how the study’s findings and outcomes were integrated
into MEII, an interaction model (discussed in detail in Chapter 7).
Chapter 6 now describes Experiment 1, and is then followed by a description of
Experiment 2 in Chapter 7. The research methodology unique to each experiment is
108 Embodied intuitive interaction in children
briefly discussed, and the results and findings from the experiments are explained
and discussed.
Embodied intuitive interaction in children 109
Chapter 6: Embodied Intuitive Interaction: Physical and Virtual
A comparative empirical study was conducted to firstly investigate researchers’
previously un-tested beliefs and claims that physical Jenga (Hasbro & Scott, 2001;
Scott, 2006) is more intuitive than virtual Jenga (app) (Natural Motions & Scott,
2011). The two systems, one physical product (physical Jenga) and an equivalent
virtual interface (Jenga app), were investigated for aspects of Embodiment that
facilitate children’s intuitive interaction. The two interfaces represent the extreme
ends of the physical-virtual continuum (Figure 1). The observational and
retrospective interview data were analysed for intuitive interaction and Embodiment.
This chapter addresses the following research sub-question:
- Can Embodiment facilitate intuitive interaction in children?
This chapter begins with a description of the research methodology and the two
toys that were used in the study (Section 6.1). The selection and recruitment of
participants is also briefly explained in this section. Section 6.2 then describes the
thematic and quantitative analysis of the data collected. Results from the analysis are
presented in Section 6.2.2 with the results discussed in Section 6.3.
6.1 METHODOLOGY
A between-subjects study was conducted in order to investigate differences in
intuitive interaction with a physical Jenga and a virtual Jenga, and to determine the
aspects of Embodiment that facilitate children’s intuitive interaction. Participants
were divided into two groups: one group played with the physical product—a
physical version of the game of Jenga; the other with a virtual interface—Jenga app.
The allocation of children to a particular group was random to make sure that each
child had an equal chance of being assigned to either group. The objective was to
observe children playing with a toy and an equivalent, similar looking app, with the
same set of game rules applied in each case. Type of Toy was the independent
categorical variable for the experiment, with two levels—physical and virtual—that
were not intrinsically ordered.
110 Embodied intuitive interaction in children
6.1.1 Game Description
A physical version of the Natural Motions’ Jenga toy was chosen as the
physical product, to be compared with its equivalent virtual version in the form of an
app ( Figure 8). Both games had similar rules. The physical Jenga toy consisted of
54 wooden blocks. The game was set up by stacking all blocks in 18 layers of three,
placed next to each other lengthwise, and perpendicular to the layer below.
The virtual interface was an Android app on a tablet that had exactly the same
game setup of 18 layers of blocks. The Jenga app uses real-time 3D physics
simulation, and recreates the behaviour of the physical Jenga toy. Each block is
affected by the surrounding blocks just as it would be in the real world, and recreates
the same strategic depth as the physical toy. It can be played in both single and
multiplayer options; however, the multiplayer option was chosen for this study.
Figure 8 Physical Jenga toy (left), and virtual Jenga app (right)
Blocks can be removed from the stack by swiping and/or tapping on them.
Once a block is removed from the stack, it is automatically taken to the top of the
stack for placement, without any player intervention. A tap on the block places the
block on the stack. The actions required to remove and stack a block are shown in
Figure 9.
Embodied intuitive interaction in children 111
Swipe block sidewards
Swipe block outwards
Tap on the block
Block positioned on top of the
stack for placement
Tap on the block initiates
placement
Block is dragged into the
position on the stack
Figure 9 Swiping and tapping to remove the block (above); placing the block on the top of the stack (below)
The lack of materiality in the app is compensated for through deliberate clues
such as warning signs in the form of white, pink, and red coding on the blocks when
touched or tapped, as shown in Figure 10 - (a) A white-coded block suggests that it is
safe to remove the block; (b) a pink-coded block suggests that it is slightly unsafe to
remove the block; and (c) a red-coded block suggests that it is highly dangerous to
remove the block. The stack is rotated by swiping left or right anywhere on the
screen, but not on the stack itself.
(a) (b) (c)
Figure 10 Colour-coded warnings of the danger of crashing the stack
112 Embodied intuitive interaction in children
6.1.2 Participants
One hundred and eight children in the 5–12 year age group were randomly
selected from a pool of volunteers recruited through a local primary school. 56
children (28 pairs) played with physical Jenga, and 52 children (26 pairs) played with
virtual Jenga.
Jenga being a popular game, children could be more familiar to physical Jenga
than the virtual Jenga. Blackler et al.'s (2010) investigation into intuitive interaction
showed that more familiar interface features are used more intuitively. This means
that familiarity of physical Jenga with children could affect intuitive interaction
results in the Experiment. However, familiarity of physical Jenga in children
represents experience and familiarity of children to physical objects, physical
interactions and material properties. Thus, the familiarity and popularity of physical
Jenga could be attributed to intuitiveness of the toy. The objective of Experiment 1
was to study what makes certain interfaces more intuitive for children’s interaction.
Children played together in a team which ensured that their natural play was
not affected by the possible intimidation of being with children and a researcher
whom they did not know. Parents were not present during the study, as children
behave differently when around adults, especially parents and teachers (Gardner,
2000).
6.1.3 Setting and Procedure
Observations and retrospective interviews and co-discovery verbal protocols
were used to collect the data. Two children were paired to play against the
researcher. All three (two children and the researcher) took turns to play. Each turn
involved taking one block out from any layer of the stack, with the exception of the
layer just below an incomplete top layer above it, and placing it on the topmost layer.
The game ended when the stack fell completely, or if any block fell from a stack.
The team responsible for the stack falling lost the game.
Experiments were conducted in a classroom at a local primary school and
QUT’s PAS Lab. Children were welcomed into the room and made comfortable and
at ease by playing some warm up games such as Blokus (Mattel, Sekkoïa, &
Tavitian, 2000). Children were then introduced to the toy/app. Studies on intuitive
interaction are based on the premise that intuitive interaction relies on prior
Embodied intuitive interaction in children 113
experience and knowledge (Blackler et al., 2010). Thus, telling participants how the
products and interfaces work could skew the application of past experience and
knowledge of the participants. Thus, children were not explained anything about how
toys and games worked. The children were told that the objective of the game is to
stack the blocks one above the other without letting the stack to fall over. The blocks
in each layer of the stack are placed perpendicular to the blocks in the lower layer.
The children were told that they would be playing together in a team against the
researcher, and everyone takes turn one after another. The winning team would win a
prize from the ‘goodie box’. The ‘goodie box’ (funded by Creative Industries Higher
Degree Research grants and the Design Research Society) was a box filled with
children’s toys and games. The same instructions were given to both the groups of
children, those playing with the physical toy and those playing with the virtual app.
Children were asked to set up the game wherever they wanted to do so. Some
children opted to set up the physical game on the table, and others opted to play on
the floor (Figure 11). All children opted to play with the virtual app on the table
(Figure 12). This facilitated creation of a natural setting for their playtime.
The entire game play was video recorded for analysis, using two digital video
cameras. For the experiments with the physical toy, one camera was placed in front
of the children and the other on the side, to capture their interactions and facial
expressions from all possible angles during their play (Figure 11). For the
experiments with the virtual interface, one camera was placed in front of the children
to capture facial expressions, and the other behind them to focus on their hands as
they interacted with the tablet (Figure 12).
The game play was followed with a retrospective interview where the children
were shown the video of their game play and asked to talk about it. Two cameras
were used to record the retrospective interviews. One camera captured the screen that
the children were watching together and the second camera captured the face of the
children. They did this, and conversed with each other as they watched. Co-discovery
verbal conversations captured during the game play and retrospective interviews, and
the video recordings, were then used to code the interactions of the children with the
systems. The entire session lasted for 45 minutes to one hour.
114 Embodied intuitive interaction in children
Figure 11 Children playing with the physical toy
Figure 12 Children playing with the virtual interface
6.2 ANALYSIS
The video recordings of the game play and retrospective interviews were
analysed using a two-step process, as described in Section 5.4. A qualitative thematic
analysis in the form of coding was followed by a quantitative analysis of the coded
data.
Audio-visual recordings of the game play and retrospective interviews were
coded concurrently in Noldus Observer XT software (Version 12.5). All the four
videos (that is, front and back camera recordings of the observations and the two
recordings of the retrospective interviews) were synchronised in Observer to the
same event marker in all four videos (Figure 13). An event marker was set during the
observations with a prompt of ‘let us start’ which was also captured during the replay
of the observations in the retrospective interviews.
Embodied intuitive interaction in children 115
Figure 13 Coding environment in Observer XT
The entire game play with Jenga (both the toy and the app) was divided into
three main behaviours and interactions: Decide, Remove and Stack. Decide
represented behaviour of the children where they were deciding which block to
remove and how to remove from the stack. Remove represented actual removal of the
block and Stack represented placing the removed block on the stack. Each of these
behaviour streams were parsed into meaningful actions which were then coded for
themes within two theme groups: Type of interaction and Aspects of Embodiment.
The coding heuristics for Type of interaction and Aspects of Embodiment are given in
Appendix D and Appendix E. Rigour in the determination of heuristics in the context
of Experiment 1 was ensured through the detailed observations of all four recordings
(front and back cameras during game play, and retrospective interviews).
The first level of codes for each of the theme groups, referred to as ‘themes’,
corresponding to Types of interaction were: Intuitive, Non-intuitive, and Partially-
intuitive; while the themes relating to Aspects of Embodiment were: Physical
affordances, Perceived affordances, Emergence, Scaffolding, and Co-operative
activity. The data were coded for each of the children separately. The interactions
116 Embodied intuitive interaction in children
and behaviour of the children with the physical and virtual Jenga were coded for
Types of interaction and Aspects of Embodiment.
Data were coded with caution: every observation was checked twice and, at
times, thrice. All coding was done by one researcher and, to avoid observer bias, data
were coded twice, with a break of 15 days between the two coding sessions.
Reliability analysis was carried out in SPSS to determine the intra-rater reliability.
ICC estimates and their 95% confident intervals were calculated using SPSS
statistical package version 23 (SPSS Statistics, 2016) based on a single measurement,
absolute-agreement, 2-way mixed-effects model. The ICC results are as follows:
ICC 3,1 0.89, 95%ConfidentIntervalof0.83 0.95
Based on the guidelines from (Koo & Li, 2016) (Appendix A), it can be
concluded from the ICC results that the intra-rater reliability was ‘good’ to
‘excellent’.
All the coded data were then exported to Excel, where the numbers of codes or
themes were totalled for each child. The totalled codes were used as dependent
variables (DVs): Number of intuitive interactions, Number of physical affordances,
Number of perceived affordances, Number of emergence events, Number of
scaffolding and Number of co-operative activity. The absolute and relative time of
each of the coded events were also reported by Observer in the exported data. The
time spent by each child deciding which block to remove and how to remove (Decide
behaviour) was evaluated from the exported time information. This time was also
totalled for each child for the entire episode of play and used as the DV, Latency to
decide.
6.2.1 Thematic Qualitative analysis
Types of interaction – intuitive, non-intuitive and partially intuitive
Intuitive interaction involves utilising knowledge gained through other
experience(s), is fast, and generally unconscious (Blackler, Popovic, & Mahar,
2010). The coding heuristics employed to code for the intuitive interactions are
derivations of methods developed by (Blackler, 2008)
Children were considered to be reasoning unconsciously when they could not
explain why they chose a certain block or how they removed and/or stacked the
block. One of the participants, when asked how he chose the block for removal, said,
Embodied intuitive interaction in children 117
“I don’t know. I just did it”
Another participant chose a block to remove after tapping at the blocks looking
for a loose block. This participant when asked the same question, said,
“I removed this block because it will balance the stack”
Although the participant did explain why he chose the block, the verbalisation
did not match his action.
Intuitive interaction is associated with high degrees of certainty, confidence
and expectation with respect to correct use of a feature (Blackler, 2008). There is no
correct or incorrect way of playing with a toy. Thus, when participants were certain
and confident that the stack would not fall because of their choice of block for
removal or during the removal and stacking of the block, the behaviour was Intuitive
Interaction. One participant, while describing how he removed the block, said
“I know the stack will not fall because it is balanced.”
The above conversation between children not only shows that the child was
certain and confident of her decision but also was reasoning unconsciously because
she was unable to verbalise the actual reason and then gave a metaphorical reason,
being in balance is equivalent to not falling.
Intuitive interaction is associated with faster decision making. Blackler, (2008)
coded correct use of a feature with not more than 5 seconds of hesitation as Intuitive
Interaction. Since play is not associated with correct use, latency was measured as
time taken to decide irrespective of whether that decision results in a win or a loss.
When a participant made a decision within 5 seconds (Blackler, 2008) and when the
decision was made with a degree of certainty and unconscious reasoning, the
behaviour was coded as Intuitive Interaction.
Partial intuitive interaction was coded when children showed signs of Intuitive
Interaction as well as Non-Intuitive Interaction. One participant clearly verbalised his
behaviour (Non-Intuitive Interaction) but was certain and confident about his
decision (Intuitive Interaction),
“I picked a loose block so that it easily comes out…It will not fall for sure.”
118 Embodied intuitive interaction in children
Aspects of Embodiment
Physical Affordances - Children used natural cues such as visual cues from
the spatial orientation of the blocks and the stack and haptic or tactile clues from the
material properties of the blocks to decide which block to remove. Children checked
whether the blocks were loose enough to be removed from the stack easily by
tapping on the blocks. Side blocks were removed from the stack by sliding the block
out of the stack. The middle blocks were removed by either pushing or pulling the
block out of the stack. Children placed the block either in the middle or the side on
the top of the stack depending on the spatial structure of the stack. They used
contextual cues such as tapping on the blocks to determine a suitable block to
remove from the stack.
Perceived Affordances –Children used their past experience with physical
blocks and their knowledge about physical and material properties such as mass,
rigidity, mobility, etc. Children often preferred to remove a middle block as they had
learned from their past experiences with everyday artefacts that as long as the side
blocks are kept intact the stack will remain balanced and will not fall. One of the
children explained his decision to remove only middle blocks as
“…I want to create windows in tall buildings...Tall buildings don’t fall, do
they?”
One of the children while explaining why he/she kept the blocks at the lower
end of the stack intact, said,
“…The base has to be balanced just like a chair is balanced on 4 legs…..”
When children could not directly use their experience from physical world,
they used population stereotypes from everyday life to interpret deliberate clues in
the Jenga app, for example they interpreted red colour codes on the blocks in the
virtual toy as highly risky to remove the block and white colour codes as safe to
remove the block. Children used their past experience and knowledge using digital
technology such as tablets. Children swiped at the tablet screen to remove the blocks
and to turn the stack around.
Children used deliberate clues such as colour codes, warning signs and
symbols, arrows to bring users’ attention to a feature etc. to make connections with
their prior knowledge and experience.
Embodied intuitive interaction in children 119
Emergence involves use of features in a product that result in change in
strategies to interact with the product and the interface. Children would often start
the game with the removal of middle blocks from the stack as they had learnt from
their past experience that it would keep the stack balanced. As the spatial structure of
the stack evolved over a number of turns, the children started removing the side
blocks. Similar changes in game strategy were also observed during a turn, such as
when children sensed that a block was too stiff to remove, and they started removing
another block.
Scaffolding was evident when children used different types of scaffolds in the
form of processes and support mechanisms while playing. The most effective
scaffolds used by children are people around them (Sharifnia et al., 2015). Imitating
strategies is one such scaffolding process; children learned different strategies from
other players, such as push and pull technique to remove the block from the stack
and turning the stack around by swiping left-right on the tablet screen.
Children offered physical support by holding the stack while the other child
removed the block. Children demonstrated use of physical space to assist them to
decide which block to remove and the actual removal of the block. Children
inspected the stack from all possible angles and moved around the tangible stack to
decide and remove the block from the stack in a tangible toy. This allowed them to
gain visual cues from the stack, which resulted in successful removal of a block from
the stack.
Co-operative activity between children was evident when they worked
together physically and verbally to develop strategies for the game. Children co-
operated through verbal encounters to develop highly organised game strategies.
Children discussed their strategies while playing with the toy. Some discussions were
brief and simple such as,
“Just keep removing the middle blocks…”
While others were more complex,
“Let’s remove all the side blocks from the top, keep the blocks at the bottom
intact…”
Actions such as pointing or touching at the blocks/stack, pulling a block out
from the stack on the other player’s turn and then either handing it to the other player
120 Embodied intuitive interaction in children
or stacking the block on the top of the stack, etc. are also examples of co-operative
activity demonstrated by children in the game play.
6.2.2 Quantitative analysis
The coded data with the totalled codes/themes were analysed quantitatively in
SPSS. The physical Jenga and the virtual Jenga were compared for intuitive
interaction, and for aspects of Embodiment that facilitated intuitive interaction in
both the physical and virtual toy. The variables (dependent and independent) chosen
for quantitative analysis are shown in Table 4.
Table 4 Dependent for measures of intuitive interaction, successfulness and aspects of Embodiment
The DVs Number of intuitive interactions, Number of physical affordances,
Number of perceived affordances, Number of emergence events, Number of
scaffolding and Number of co-operative activity were taken directly from the totalled
codes/themes in Excel. As the game progressed, the stack grew taller as more blocks
were removed and added to the stack. The successfulness of the game was measured
by noting the number of layers added over and above the 18 layers that the children
and the researcher started playing the game with. The DV Number of layers added
was noted for each child during the experiment, and double checked during video
analysis. The time taken to decide which block to remove and how to remove from
the stack, totalled for the entire episode of the game for each child was used as DV
Latency to decide.
Independent Variable Dependent Variables
Type of Toy – Physical and Virtual Measures of Intuitive Interaction
Number of intuitive interactions
Latency to decide
Measure of Successfulness
Number of layers added
Measures of aspects of Embodiment
Number of Physical affordances
Number of Perceived affordances
Number of Emergence events
Number of Scaffolding
Number of Co-operative activity
Embodied intuitive interaction in children 121
Intuitive interaction in physical and virtual interfaces
The descriptive statistics for the Number of intuitive interactions, Number of
layers added and Latency to decide corresponding to the Independent Variable (IV) -
Type of Toy: Physical Jenga and Virtual Jenga are presented in Figure 14 and Table
5.
Figure 14 Comparison of Number of Intuitive Interactions; Number of Layers Added; and Latency to decide for physical and virtual Jenga
Table 5 Descriptive statistics for Number of intuitive interactions, Number of layers added and
Latency to decide corresponding to Type of Toy: Physical and Virtual Jenga
Descriptive (→) Statistics DVs (↓)
Type of Toy Physical Jenga Virtual Jenga Mean Median Standard
Deviation Mean Median Standard
Deviation Number of intuitive interactions
26.59 24 13.84 10.16 9.5 7.09
Number of layers added 8.90 7.5 4.46 2.94 2.5 2.43 Latency to decide (secs) 10.83 7.47 13.14 13.83 10.12 13.16
122 Embodied intuitive interaction in children
A Mann-Whitney U test was run to determine if the differences in Number of
Intuitive Interactions, Number of Layers Added, and Latency to decide between
physical Jenga and the virtual Jenga were statistically significant. The Mann-
Whitney U test works by ranking each score of the dependent variables, irrespective
of the group it is in (physical Jenga or virtual Jenga), according to its value, with the
smallest rank assigned to the smallest value. The ranks obtained for each of the
groups are then averaged. This results in a mean rank for each of the groups -
physical Jenga and virtual Jenga (shown in Table 6).
The null hypothesis of the Mann-Whitney U test is that the mean rank is the
same for both the groups, physical Jenga and virtual Jenga. However, if one group
tends to have higher values than the other group, that group's scores will have been
assigned higher ranks and will have a higher mean rank (and vice versa for the group
with lower scores). It is this difference in mean rank that is tested by the Mann-
Whitney U test for statistical significance. This approach was used to determine
whether the differences in the DVs for physical Jenga and virtual Jenga are
statistically significant. The mean ranks obtained for the physical Jenga and the
virtual Jenga are shown in Table 6.
Table 6 Mean Rank Values of Number of Intuitive Interactions, Number of Layers Added, and Latency
to decide for each type of toy
The Mann Whitney U Test statistic of the dependent variables is presented in
Table 7.
Table 7 Mann-Whitney U Test Statistic of Number of Intuitive Interactions, Number of Layers Added,
and Latency to decide for each type of toy
DVs (→) Type of Toy (↓)
Mean Rank Values Number of Intuitive Interactions
Number of Layers Added
Latency to decide
Physical 74.26 76.52 49.82 Virtual 33.22 30.79 59.54
Number of Intuitive Interactions
Number of Layers Added
Latency to decide
Mann-Whitney U 2562.5 2689 1194 Standardised test statistic (Z) 6.809 7.613 -1.611 Asymp. Sig. (2-tailed) (p) 0 0 0.107 Effect Size (d) 0.66 0.73 0.16
Embodied intuitive interaction in children 123
Putting all the results together (both descriptive and inferential), Number of
Intuitive Interactions for the physical Jenga (mean = 26.59, median = 24, std.
deviation = 13.84) was statistically significantly higher than for the virtual Jenga
(mean = 10.16, median = 9.5, std. deviation = 7.09), U = 2562.5, p ≈ 0 (p<0.05),
d=0.66.
Number of Layers Added for the physical Jenga (mean = 8.90, median = 7.5,
std. deviation = 4.46) was statistically significantly higher than for the virtual Jenga
(mean = 2.94, median = 2.5, std. deviation = 2.43), U = 2689, p ≈ 0 (p<0.05), d=0.73.
Effect size (d) indicates that the differences in Number of Intuitive Interactions and
Number of Layers Added between the physical Jenga and the virtual Jenga are strong.
Latency to decide was not statistically significantly different between children
playing with the physical Jenga (mean = 10.83, median = 7.47, std. deviation =
13.14) and those playing with the virtual Jenga (mean = 13.83, median = 10.12, std.
deviation = 13.16), U = 1194, p = 0.090 (p>0.05), d=0.107. Effect size (d) = 0.107,
indicates that the difference in Latency to decide between the two groups is trivial.
The above results suggest that physical Jenga possess properties and features
that make them more intuitive than the virtual Jenga. The latency parameter is
insignificant in both the systems. This could be due to the context of play where
children are involved in pragmatic actions to reach the ultimate goal specific to a
game, and also in epistemic actions that allow them to offload tasks from their mind
to actions.
Aspects of Embodiment in physical toy and virtual interface
The two toys, physical Jenga and virtual Jenga, were then analysed for use of
aspects of Embodiment to determine the extent to which each of the aspects of
Embodiment contribute to intuitive interaction. The descriptive statistics for the use
of aspects of Embodiment corresponding to the Independent Variable (IV) - Type of
Toy: physical Jenga and virtual Jenga is presented in Table 8 and Figure 15.
124 Embodied intuitive interaction in children
Table 8 Descriptive statistics for Number of uses of aspects of Embodiment for physical and virtual
Jenga
Figure 15 Box plot of use of aspects of Embodiment in physical and virtual toys
The physical Jenga demonstrated the highest uses of physical affordances
(mean = 22.09, median = 20.0, std. dev. = 12.28), followed by perceived affordances
(mean = 6.97, median = 6.0, std. dev. = 5.29). This was followed by use of
Descriptive (→) Statistics DVs (↓)
Type of Toy Physical Jenga Virtual Jenga Mean Median Standard
Deviation Mean Median Standard
Deviation
physical affordances
22.09 20.0 12.28 5.79 4.0 4.98
perceived affordances
6.97 6.0 5.29 7.94 7.5 6.39
Emergence 4.25 3.0 3.20 0.75 0.0 1.31 Scaffolding 3.161 2.0 2.90 3.16 3.0 2.04 Co-operative activity
5.64 4.0 4.76 1.90 1.0 2.01
Embodied intuitive interaction in children 125
cooperative activity (mean = 5.64, median = 4.0, std. dev. = 4.76), emergence (mean
= 4.25, median = 3.0, std. dev. = 3.20), and scaffolding (mean = 3.161, median = 2.0,
std. dev. = 2.90). Physical affordances were the prime contributor to intuitive
interaction with the physical Jenga.
The virtual Jenga demonstrated the highest uses of perceived affordances
(mean = 7.94, median = 7.5, std. dev. = 6.39), followed by physical affordances
(mean = 5.79, median = 4.0, std. dev. = 4.98). This was followed by uses of
scaffolding (mean = 3.16, median = 3.0, std. dev. = 2.01), cooperative activity (mean
= 1.904, median = 1.0, std. dev. = 2.01) and emergence (mean = 0.750, median = 0.0,
std. dev. = 1.31). Perceived affordances were the prime contributor to interaction
with the virtual Jenga.
A related Samples Friedman test was run to determine if the differences in uses
of aspects of Embodiment in each of the toys, were statistically significant. Related
samples Friedman test works in a similar way to Mann Whitney U test except that it
is used to determine significance within samples (within physical Jenga and virtual
Jenga). The mean ranks obtained for the uses of each aspect are shown in Figure 16.
The use of aspects of Embodiment were statistically significantly different for
interactions with the physical Jenga, χ2(4)= 119.5, p <0.05. Use of aspects of
Embodiment were statistically significantly different for interactions with the virtual
Jenga, χ2(4)= 108.77, p <0.05.
Pairwise comparisons were performed for statistical significance, with a
Bonferroni correction for multiple comparisons. The results for the physical Jenga
are shown in Figure 16 (left), and for the virtual Jenga in Figure 16 (right). The mean
rank values for each of the aspects of Embodiment for interactions with the toys are
presented. The black thick lines represent statistically significant differences between
pairs, while the grey thin lines represent statistically insignificant differences.
Pairwise comparisons for the physical toy indicate that the difference between
physical affordances and other aspects is statistically significant. On the other hand,
pairwise comparisons for the virtual toy indicate that the difference between physical
affordances and other aspects is statistically significant, with the exception of
perceived affordances. Perceived affordances also demonstrated statistically
significant differences with the other aspects, with the exception of physical
affordances.
126 Embodied intuitive interaction in children
Figure 16 Pair-wise comparisons of aspects of Embodiment for interactions with physical toy (left) and virtual toy (right) with mean ranks for each aspect
6.3 DISCUSSION
The findings suggest that the physical toy was more intuitive, and supported
more successful game play (in terms of layers added) than the virtual app. This study
also investigated the aspects of Embodiment that make physical Jenga more intuitive
than virtual Jenga. The results showed that the intuitive use of aspects of
Embodiment was more evident in the physical Jenga than the virtual Jenga. Physical
affordance was the main facilitator of intuitive interaction in the physical Jenga,
followed by perceived affordances, co-operative activity, scaffolding, and
emergence. Perceived affordances were the main facilitator of intuitive interaction in
the virtual toy. However, there was no significant difference in intuitive use of
perceived affordances between physical Jenga and virtual Jenga.
Children used physical affordances and their experiential knowledge of playing
with blocks, and other similar toys and games, to play with the physical toy. They
also used their experiential knowledge from everyday life such as seeing buildings
with windows, or knowing that a broader base keeps the stack more stable. Spatial
orientation of the stack and the blocks, and material properties of the blocks (such as
their amount of stiffness) provided assistance in playing the game. Children used
visual, material, spatial and contextual natural clues from the blocks and the stack in
their decision making before removing blocks from the stack. The spatial layout of
the blocks in the physical toy offered the visual natural clues to tap on the block.
However, when they sensed that the block was not loose, some used two hands to
Embodied intuitive interaction in children 127
remove the block so that the stack did not fall down, while others looked for another
loose block and continued tapping at the blocks.
Children predominantly used perceived affordances to play with the virtual
Jenga, using their previous experience from the digital world and the physical world
to play with it. The app offered visual cues in the form of the three-dimensional
spatial orientation of the block on the tablet. The app also mimicked the wobbliness
of a physical stack in the real world. The game designers represented the materiality
of the physical Jenga toy in the app by contextual cues in the form of white, red, and
pink colour codes on the blocks, and crash signs to warn players of the danger of
stack collapse. However, the children did not often detect these contextual perceived
affordances. Some were able to learn them as they played through a number of turns.
However, most toppled the stack before they had any opportunity to notice the cues,
or gave up playing the game. Thus, although attempts were made to incorporate the
physical affordances of physical blocks into the app in the form of deliberate visual
clues, children did not have the relevant knowledge (i.e. sensorimotor or
experiential) to use those features. Thus, there were higher numbers of uses of
physical affordances in intuitive interaction of the physical toy than the virtual app.
However, there was no significant difference between the toys in terms of the use of
perceived affordances for intuitive interaction.
Children were interacting with two interfaces while playing with the virtual
app. Their experiential knowledge of the tablet sometimes contradicted the
experiential knowledge associated with the stack in the app. They had learnt in the
physical world that the blocks had to be pushed or pulled to remove them from the
stack. However, their experience in the virtual world had taught them that the only
way to interact with a tablet (or any other touch screen interface) is to swipe, tap, and
hold onto the screen. Thus, some of the features of the app, such as rotating the stack
by swiping it left and right, were not discovered.
Children often work in teams and groups at school. Peer support mechanisms
are often used in a classroom setting, where children offer support to others in the
class and, at the same time, gain support from others (Berk & Winsler, 1995;
Hammond & Carpendale, 2015). Children offered physical support by holding the
physical stack in the physical toy together while the other child removed and stacked
the block. Some children unknowingly imitated each other’s game strategies while
128 Embodied intuitive interaction in children
playing with the physical toy, while there was limited evidence of such processes
being used with the virtual app. High levels of physical support, imitation of game
strategies, and use of the space around them to interact with the physical toy, explain
the higher numbers of intuitive uses of scaffolds with the physical toy than with the
virtual toy.
Children played with the physical toy together, even though they were asked to
take turns to play. They discussed strategies and provided verbal suggestions to each
other while playing. The material and spatial properties offered by the physical toy
allowed the display of real time information about the object (Sellen & Harper,
2002), and provided a mechanism for efficient co-ordination of information between
the players (Vyas et al., 2012). Children intervened in the other player’s game, for
example, to change the placement of the block on the top of the stack so that the
stack remained balanced, or to complete the removal of the block from the stack. On
the other hand, when playing with the virtual app, they waited for the other child to
finish interacting, and avoided intervening in their play. This was the result of their
perception that two people or more cannot interact with a tablet at the same time.
Thus, the physical toy demonstrated higher numbers of intuitive uses of the co-
operative activity aspect than the virtual app.
The effective use of visual, material and contextual natural clues in the
physical toy resulted in changes to its spatial and material properties (stiffness and
looseness of the blocks in the stack, for example); this, at times, resulted in changes
in strategies, previously explained as ‘emergence’. For example, removal of a block
from the stack resulted in changes in system behaviour (e.g. a wobbly stack and stiff
and loose blocks), and changes in system configuration. This resulted in emergent
interaction as the players had to change their strategy to remove the blocks from the
stack.
The lower the amount of prescribed behaviour built into the system, the higher
the degree of emergence (Pfeifer et al. 2005). The simplicity of the blocks, which
have a loosely defined behaviour, allowed emergence. On the other hand, the virtual
app, with its pre-defined, pre-programmed behaviour, inhibited emergent
interactions. Thus, the intuitive use of the emergence aspect of Embodiment was
more evident in the physical toy than in the virtual app.
Embodied intuitive interaction in children 129
Intuitive use is fast and, thus, it was expected that children would take longer to
decide which block to remove when playing with the virtual interface compared with
the physical toy (Blackler, 2008). However, there was a trivial difference in Latency
to Decide for both toys. Children were neither pushed to commit to a strategy, nor
were they given a time limit to finish the game. They explored the environment and
socially interacted with the other players. One child, while deciding which block to
remove from the stack, talked about a uniform free day at school:
Child 1: “….why are you wearing yellow socks?”
Child 2: “…we can wear any colour of clothes but not uniform.”
Child 1: “…but why are you wearing yellow?”
Children demonstrated social and exploratory behaviour with both physical toy
and virtual app, and this affected the Latency to Decide dependent variable. This
suggests that no time measure is relevant when investigating children’s play, unless
they have been told to finish as quickly as possible.
6.4 SUMMARY
This chapter has presented a comparative study of physical Jenga and virtual
Jenga for intuitive interaction, and the use of aspects of Embodiment in intuitive
interaction with these toys. The results showed that physical Jenga demonstrates
more intuitive interactions than virtual Jenga, confirming previously un-tested claims
of researchers that a physical product is more intuitive than its virtual counterpart
(Hornecker & Buur, 2006; IKindsmuller, et al., 2009; Mihajlov et al., 2015; Olson et
al., 2011; Sapounidis et al., 2015; Seo & Lee, 2013). Children showed high levels of
uses of physical affordances, emergence, scaffolding, and co-operative activity in
interaction with the physical toy in comparison to the virtual toy. They showed the
equal likelihood of using perceived affordances in interaction with the physical and
the virtual toy.
The interaction with virtual systems relies solely on experiential learning
during the use of the product, or from the prior use of similar products. However,
children can either abandon their use of the product, or fail to meet given goals
before they have detected and learnt the deliberate clues embedded in the design.
130 Embodied intuitive interaction in children
Thus, a design that is solely based on experiential knowledge and perceived
affordance does not ensure effective interaction.
Chapter 7 now describes the second experiment that was carried out to
determine how aspects of Embodiment can facilitate children’s intuitive interaction.
Two systems that allow direct manipulation of physical elements, a physical toy and
a TEI with physical and virtual elements, were investigated for intuitive interaction.
They were then analysed for aspects of Embodiment that have a significant impact
on children’s intuitive interaction.
Embodied intuitive interaction in children 131
Chapter 7: Primary Predictors of Embodied Intuitive Interaction
Experiment 1 described a comparative study of a directly manipulated physical
Jenga and virtual Jenga for intuitive interaction and, more specifically, for aspects of
Embodiment that facilitate intuitive interaction in these systems. Experiment 1
confirmed previously un-tested claims in the literature that physical products are
more intuitive than their virtual counterparts. The study further explained the role of
aspects of Embodiment in intuitive interaction with physical Jenga and virtual Jenga.
Children showed higher levels of the use of physical affordances, emergence,
scaffolding, and co-operative activity in intuitive interaction with physical Jenga than
with virtual Jenga. The use of perceived affordances is equally likely in intuitive
interaction with physical Jenga and virtual Jenga. The findings suggest that the
presence of these aspects of Embodiment in the design of a system could ensure
children’s intuitive interaction.
Given that the directly manipulated physical products have sufficient aspects of
Embodiment to allow children to interact with them intuitively, Experiment 2
analysed two directly manipulated interfaces separately (not compared)—a physical
product, and a Tangible Embodied Embedded Interface (TEI)—for aspects of
Embodiment that have a significant impact on children’s intuitive interaction. The
objective was to study variability of intuitive interaction with respect to the aspects
of Embodiment in a physical product and a product with both physical and virtual
elements, so that the physical-virtual continuum is sufficiently covered in the study.
Experiment 2 answers the following research sub-questions:
- Which design aspects of Embodiment facilitate children’s intuitive
interaction
- To what extent do the design aspects of Embodiment facilitate intuitive
interaction in children?
The two products chosen for this study were Monkey Blocks (Popular
Playthings, 2014) and Osmo (Tangible Play, 2014b). Monkey Blocks is a physical
toy consisting of physical blocks. Osmo is a TEI with a physical and a virtual space
132 Embodied intuitive interaction in children
that are distinct and separate. Both the toys have a common interaction mechanism
by way of direct manipulation of physical objects. Choice of these two toys was
influenced by factors such as accessibility to toys, especially a TEI, cost of TEI and
time taken to play with the toys in an experimental setup (children, especially
younger children, get tired playing for a long time).
This chapter describes the methodology of the data collection, and its
subsequent analysis. This is then followed with the results and discussion.
7.1 METHODOLOGY
A within-subjects observational study was carried out with 42 children, aged
5–12 years, at PAS Lab, QUT (Australia). Children were paired with their friends or
siblings. Parents and children were invited to participate in the study during the
school holidays, and were asked to bring a friend or a sibling to play with. The
parents were however asked not to be present during the study. If they insisted on
being around or present for the study, they were requested to sit on the other side of a
two-way mirror. The children and their parents were known to the researchers
through personal contacts, and through their participation in Experiment 1.
Twenty-one pairs of children were observed playing with two toys, Osmo, a
TEI, and Monkey Blocks, a physical product. Monkey Blocks represents the extreme
left of the physical-virtual continuum (Figure 1), while Osmo represents the middle
of the continuum. Both the toys involved direct tactile interaction and manipulation
with physical objects to achieve game-specific goals. Children and parents were first
welcomed into the lab, made comfortable, and then asked to sign the consent forms.
Children were instructed to start playing with one of the toys.
While playing with Osmo, children were instructed to start playing from Level
1, and to progress to other levels after they had completed the previous one. While
playing Monkey Blocks, children were instructed to arrange the blocks by looking at
a black and white pictorial arrangement of their choice on a sheet of paper (Figure
21), and to work together as a team. They were given a maximum 30 minutes to play
each game. However, they were allowed to stop playing before 30 minutes if they
wished so. They played both the games one after another. The order in which a child
pair played each of the games was varied from one observation to the other, ensuring
that there were an equal number of pairs playing each game first. This was to ensure
Embodied intuitive interaction in children 133
that the carry-over effects balanced out across the two game plays (Gray & Kinnear,
2012) .
Both the game plays were video recorded for analysis. Two digital video
cameras were used to record the play. One camera was placed in front of the
children, and the other on the side to capture their interaction and facial expressions
during play.
7.1.1 Osmo Game description
Osmo, a TEI game from Tangible Play (Figure 17) , allows physical play with
a virtual system (Ipad). It comes with a reflector, a stand that is attached to an
Ipad,and games that can be downloaded as apps from iTunes.
Figure 17 Osmo setup and Newton app
The game used for the study is called ‘Newton’ (Tangible Play, 2014a) which
was downloaded from iTunes (Apple, 2001) on the iPad (Apple, 2015). The Newton
game consists of 60 levels, each level involving challenges that require manipulation
of objects and drawings placed in front of the screen to guide free falling balls onto
various targets such as spheres on the screen (Figure 18). The challenges require
simple strategies to start with and as children progress through the levels, they are
134 Embodied intuitive interaction in children
presented with challenges with bouncing balls, accelerating platforms, teleporters,
and fans.
Figure 18 Children playing Osmo. The view of the tablet screen (on the left) shows Newton game in action and the view of children manipulating objects and drawing in the physical space (on the right)
Children were provided with the following objects to play the game: An A3
paper sheet placed in front of the tablet, two pencils, two pens, two erasers, a cheese
block toy, a ruler, five straws, two lollipops, cardboard strips in both random and
regular shapes (triangle, square, rectangle, and trapezium), a plastic spoon, and two
chop sticks. The direct physical interaction in the game is entirely with the objects in
the physical space, and in the context of achieving the goal of directing balls onto the
targets. The Ipad display screen provides feedback on the manipulation, in relation to
the targets on the screen.
Tangible Embodied Embedded Interfaces (TEIs) were defined and discussed in
Section 1.1. There are several variations of TEIs depending on how the physical and
virtual spaces (in other words, perception and action spaces) are configured. Physical
and virtual spaces could be overlapping (Ullmer & Ishii, 2000) or they could be
distinct and separate (Zuckerman et al., 2016). Osmo and Newton can be considered
as TEIs as they offer a physical and a virtual space for interaction, with integration of
electronics in the form of a tablet (computing element) and a camera. The virtual and
physical spaces are distinct and separate which is in line with the approach taken to
design and develop TEIs by researchers such as Gervais et al. (2016), Mistry & Maes
(2009), Yu et al. (2016), Zuckerman et al. (2016).
Children were asked to play with Osmo and Newton app for as many levels as
they could in a maximum 30 minutes.
Embodied intuitive interaction in children 135
7.1.2 Monkey Blocks game description
Monkey Blocks is a logical deduction game, consisting of 12 blocks divided
into three types (4 blocks each) depending on the position of the weights inside the
blocks (Figure 19). Orange blocks are weighted at one end of the block, green blocks
are weighted in the middle, and blue blocks have no weights inside. The game comes
with 12 blocks (4 blocks of each of the three types), and 6 monkeys.
Figure 19 Three types of blocks in Monkey Blocks: orange blocks with weights at one of the ends, green blocks with weights in the middle, blue blocks with no weights.
The object of the game is to stack blocks and monkeys in an arrangement (e.g.
Figure 20) so that they remain balanced, without falling over.
Figure 20 Blocks and monkeys in arrangements
Children were given sheets with 18 arrangements in black and white to use as a
reference. Black and white images were given to prevent children from just copying
the arrangements using the colour codes, but forcing them to rely on physical and
136 Embodied intuitive interaction in children
material properties to complete the arrangement. They were asked to build as many
arrangements as they could in maximum 30 minutes. Figure 21 shows an example of
an arrangement in black and white and its equivalent coloured arrangement. All the
18 arrangements given to children in the study and their equivalent coloured images
are in Appendix P.
Figure 21 Example of an arrangement: black and white image given to children (left), equivalent coloured image (right)
7.2 ANALYSIS
The video recordings of the game play were analysed using thematic analysis
(as described in Section 5.4.1) The qualitative thematic analysis (in the form of
coding) was followed by a quantitative analysis of the coded data. Audio-visual
recordings of children playing each of the games were coded as two separate
observations in Noldus Observer XT software (Version 12.5) for each pair of
children. Two videos of the observations, one from the front camera and the other
from the back camera, were synchronised in Observer to the start of the game play.
An event marker was set during the observations with a prompt of ‘let us start’.
The entire game play with Osmo and Monkey Blocks was coded separately for
each child as different observations. The entire episode of game play was
continuously sampled for meaningful interactions which were then coded for themes
within two theme groups: Type of interaction (intuitive, non-intuitive and partially
intuitive) and Aspects of Embodiment (physical affordances, perceived affordances,
emergence, scaffolding and co-operative activity). The coding heuristics for the
theme groups are given in Appendix D and Appendix E. The themes corresponding
Embodied intuitive interaction in children 137
to Types of interaction were: Intuitive, Non-intuitive, and Partially-intuitive. The
themes corresponding to Aspects of Embodiment were: Physical affordances,
Perceived affordances, Emergence, Scaffolding, and Co-operative activity. Types of
interaction was configured as mutually exclusive and exhaustive, so that every
behaviour was required to be scored for one of the themes from the theme group.
Aspects of Embodiment was, on the other hand, configured as non-mutually exclusive
and non-exhaustive; in other words, it was not necessary to score every behaviour for
Aspects of Embodiment.
Interactions of each of the children were coded separately. Rigour in the
determination of heuristics in the context of Experiment 2 was incorporated through
detailed observations of the video recordings of game play (using both front and back
cameras). Data were coded with caution: every observation was checked twice and,
at times, thrice. All coding was done by one researcher and, to avoid observer bias,
data were coded twice, with a break of 15 days between the two coding sessions.
Reliability analysis was carried out in SPSS to determine the intra-rater reliability.
ICC estimates and their 95% confident intervals were calculated using SPSS
statistical package version 23 (SPSS Statistics, 2016) based on a single measurement,
absolute-agreement, 2-way mixed-effects model. The ICC results are as follows:
ICC 3,1 0.925, 95%ConfidentIntervalof0.82 0.99
Based on the guidelines from (Koo & Li, 2016) (Appendix A), it can be
concluded from the ICC results that the intra-rater reliability was ‘good’ to
‘excellent’.
All the coded data were then exported to Excel, where the number of codes or
themes were totalled for each child. The totalled codes were used as DVs: Number of
intuitive interactions, Number of non-intuitive interactions, Number of partially
intuitive interactions, Number of physical affordances, Number of perceived
affordances, Number of emergence events, Number of scaffolding and Number of co-
operative activity.
7.2.1 Thematic Qualitative analysis
Types of interaction – intuitive, non-intuitive and partially intuitive
Intuitive interaction involves utilising knowledge gained through other
experience(s), is fast (Salk, 1983), and generally unconscious (Bastick, 2003). The
138 Embodied intuitive interaction in children
coding heuristics employed to code for intuitive interaction are derivations of
methods developed by Blackler et al. (2010). Intuitive interaction does not involve
conscious reasoning (Bastick, 2003) but involves actions and decisions which cannot
be explained or verbalised (Blackler et al., 2010). Children were reasoning
unconsciously when they could not explain how they guided the balls onto the targets
on Osmo or how they arranged different blocks in Monkey Blocks so that they remain
balanced. When children could not verbalise their decisions and actions and were
unable to explain the actual reasons, they were interacting intuitively.
While playing Osmo, one of the participants said:
“It is easy, don’t you understand this?”
One of the children explained the decision to draw lines and curves instead of
using objects to guide the balls onto the targets,
“I like to draw, drawing is easy. Do you know I got an award at the assembly
for art?”
Explaining the strategy to guide orange and green balls onto respective
coloured targets,
“After the green, we will do the orange”.
But the children guided the balls on the targets together, both targets at the
same time.
While playing with Monkey Blocks, one of the children explained the strategy
to stack the blocks in an arrangement as follows,
“…it will keep the blocks balanced.”
“I don’t know.”
Thus, it is evident that there was lack of accurate reportability and verbalisation
in the description of strategies that children used to play the game.
Intuitive interaction is associated with high degrees of certainty, confidence
and expectation with respect to correct use of a feature (Bastick, 2003; Hammond,
1993; Woolhouse and Bayne, 2000). When participants were certain and confident
about their strategy to guide the balls onto the target while playing the game of
Osmo, in contrast to trying out various options, the behaviour was coded as intuitive
interaction. One participant, while playing the game Osmo, said to the other child,
Embodied intuitive interaction in children 139
“I know. I know. I got this.”
While playing with Monkey Blocks, children demonstrated certainty and
confidence in their strategic decisions while playing,
“…the stack WILL NOT fall like that..”
The above statements not only show that the children were certain and
confident of their decisions but also were reasoning unconsciously because they did
not verbalise the actual method that they were going to follow.
Non-intuitive interaction is associated with conscious reasoning, uncertainty,
lack of confidence and unclear expectations with respect to the interaction with the
system (Blackler, 2008). Children were reasoning the game strategies consciously,
when they could not figure out the clues in the toys to decide on their next actions to
be performed on the toys.
While playing Osmo, some children could not figure out how to play a level in
the game which asked them to spin fans. They could not interpret the deliberate clue,
a ‘SPIN’ prompt with an arrow pointing towards fan(s).
“Alright, we need to think about this logically”.
While playing with Monkey Blocks, one of the participants while trying to
balance the orange block on its edge (with no weight) expressed concern as the block
kept falling off:
“We need to figure this out slowly….hold this block…I will balance it with
another heavy block…so it does not fall…”
Some children expressed uncertainty, lack of confidence and unclear
expectations while playing with Osmo and Monkey Blocks. While playing Osmo,
some children looking at the ‘SPIN’ prompt, expressed their lack of understanding
and expectations from the level,
“What does this mean?”
One of the participants was unable to determine the boundaries of the game to
place the objects in the physical space to stop the balls from escaping the screen.
“But where does it [ball] go?”
140 Embodied intuitive interaction in children
Children when presented with bowls on the screen that function as teleporters
in the game of Osmo, they could not figure out how it works until they saw the balls
falling into one of the bowls, and balls emerging from another bowl.
“What is the point of this level?”
While playing the game of Monkey Blocks, children were faced with situations
where they had to balance the edge of the block. Some children tried to use the blue
(no weight) and the green blocks (weight in the middle), the block kept falling.
“This is hard.”, Patting the forehead indicating confusion.
“How are we going to do this?”
Partially intuitive interactions involved interactions that showed signs of
intuitive as well as non-intuitive interaction. A child while playing Osmo noticed that
the balls were escaping towards the left of the screen instead of being guided towards
the target. He picked up a straw and placed it on the left so that the balls did not
escape anymore. He told the other child,
“Hang on. I have got this! Let’s put this [here] so that balls don’t run away”
The child clearly verbalised his behaviour (non-intuitive interaction) but was
certain and confident about his decision (intuitive interaction). Most children quickly
figured out that the green block is the heaviest and the blue block the lightest in
Monkey Blocks. However, when asked about the orange block, there were mixed
responses,
“It is the same weight as the green block. No, wait a minute; it is lighter
than green block but heavier than blue block. Hmm, I am not sure.”
Children quickly and unconsciously figured out the differences between the
green block and the blue block (intuitive), but demonstrated internal conflict when it
came to the orange block (non-intuitive).
Aspects of Embodiment
Physical affordances in Osmo were evident when children used spatial
orientation of objects and drawings relative to the balls and targets on the screen to
decide on the optimum angular position to guide the balls onto the targets. Choice of
certain objects for certain alignments was also representative of use of physical
Embodied intuitive interaction in children 141
affordances of the virtual elements and the objects. In Monkey Blocks, children used
material and spatial properties of the blocks to stack them in an arrangement.
Children discussing the weights of the blocks in Monkey Blocks,
“This is heavy, I can feel it”
C1: "Oh, this is heavy."
C2 (feeling the blue in her hand): “All are not heavy.”
It is evident from the above that children were using natural clues in the form
of visual (such as spatial orientation) and material properties that children could
perceive to make decisions for their actions.
Perceived affordances is the use of familiarity, past experience and
knowledge about features of physical objects such as mass, elasticity, rigidity,
mobility, etc. and about digital technology. Deliberate clues in the toys helped
children connect them to the prior knowledge and previous experience.
In the game of Osmo, as the objects are moved around in the physical space, it
results in balls being bounced around in the virtual space, hitting targets and objects
mirrored into the virtual space. This is linked to the game of tennis. Such deliberate
clues in the form of actions and movements that link to previous knowledge of
children are Embodied representations.
“…This is like tennis...”
Some children likened the guiding of balls onto the targets to guiding water
through tunnels. These children drew curves in the physical space, referring them as
tunnels, to guide balls onto targets such as spheres, bowls and fans. The embodied
representation of balls hitting the targets and objective of the game to use the
physical space to guide the balls onto the targets helped children to link the context
to the game of tennis.
Referring to the simulation of teleportation, one child explained that he had
read about it in the book, ‘Charlie and the Chocolate Factory’, another one referred
to the ‘Charlie and the Chocolate Factory’ movie. Embodied representations of balls
actually emerging from a bowl when balls were guided to another bowl helped
children connect the level of the game to their experience of watching the movie or
reading the book.
142 Embodied intuitive interaction in children
The game Monkey Blocks is incorporated with deliberate clues in the form of
coloured blocks to allow players to decipher the weights in each of the blocks.
Children. However, children used these colour codes to group them while stacking
the blocks but only after they had figured out the difference in weights using natural
clues (physical affordances). The spatial orientation (natural clues) of the stack of
blocks reminded children of buildings. This combined with the knowledge that the
blocks are different in weights, children used their previous knowledge of buildings
such as heavy weights at the bottom and light weights on the top to keep the blocks
in balance.
“….just like buildings…”,
“Heavy bottoms are good…keeps everything in place.”
It is evident from the above that children were using their past experience and
knowledge to relate to things outside the domain of the game, such as stories, tennis,
buildings, tunnels and so on. They were using natural and deliberate clues to make
these connections.
Emergence represents dynamic nature of the system. Children changed their
strategies and techniques in response to contextual clues. In Osmo, children placed
an object in front of the screen along the edge of the tablet screen to block the balls
from escaping from the sides. They erased parts of drawings to guide the balls onto
the target. They started using edges of the sheet of paper to guide the balls onto the
target. Children extended shorter objects by adding another object when they realised
that the object was too short to create the required angular deflection to guide the
balls onto the target. Children were seen partitioning the guns the guns for the two
coloured balls by placing an object between them so that the coloured balls do not
get mixed up. This allowed the children to deal with each of the coloured balls
separately and individually.
Emergence in Monkey Blocks was prominently due to children changing their
strategies and techniques in response to constraints placed on orientation and
placement of the blocks due to weights in them. The emergence is evident from the
fact that children sometimes ended up creating the same arrangement as in the black
and white images (see Figure 21) but with different combination of blocks. Some
children placed another block on the top of a block placed horizontally to balance it
on a single edge.
Embodied intuitive interaction in children 143
Scaffolding in using support mechanisms to assist in the game play. In Osmo,
children used hands in V formation to either guide the balls or block the balls from
deflecting away from the targets. While one child was busy aligning the objects, the
other child blocked the escaping from the sides of the tablet screen. Children imitated
strategies of the other child such as use of hands.
In Monkey Blocks, scaffolding was evident when children imitated strategies of
the other child to complete half of a symmetrical arrangement. The children were not
told or prompted about the symmetry, the children automatically detected it even
without discussing between them. Children laid out blocks on the table before
stacking them.
Co-operative activity was evident when children discussed strategies and
worked together physically to play the game. Children were seen strategizing
verbally while playing with Osmo,
“After the green, we will do orange..”
Children worked together to guide the balls onto the targets such as aligning
objects to guide free falling balls onto the targets. One child drawing lines and curves
while the other one erasing it.
While playing with Monkey Blocks, one child held the horizontal block while
another child placed a heavier block on it. Some children completed a part of the
arrangement working in collaboration with the other child.
7.2.2 Quantitative analysis
The coded data with the totalled codes/themes were analysed quantitatively in
SPSS and STATA tools for statistics. The two toys were analysed separately for
intuitive interaction and aspects of Embodiment (i.e. they were not compared with
each other).
Types of interactions
The first part of the analysis investigated types of interactions- intuitive
interaction in the two directly manipulated interfaces—the TEI Osmo, and the
physical toy Monkey Blocks. The boxplots for number of intuitive, non-intuitive, and
partially-intuitive interactions demonstrated by the children with the two interfaces
are shown in Figure 22 and Figure 23.
144 Embodied intuitive interaction in children
The TEI game, Osmo, demonstrated the highest number of intuitive
interactions (mean = 30.10, median = 27, standard deviation = 19.74), followed by
non-intuitive interactions (mean = 4.38, median = 3.50, standard deviation = 3.52).
Partially-intuitive interactions were the least demonstrated (mean = 1.62, median = 1,
standard deviation = 1.27).
Monkey Blocks demonstrated a high number of intuitive interactions (mean =
7.88, median = 7, standard deviation = 4.83), while the number of non-intuitive
interactions (mean = 1.57, median = 1, standard deviation = 1.02) and number of
partially-intuitive interactions (mean = 0.62, median = 0.0, standard deviation = 1.10)
were very low.
Figure 22 Number of intuitive, non-intuitive, and partially-intuitive interactions for the TEI Osmo
Embodied intuitive interaction in children 145
Figure 23 Intuitive, non-intuitive, and partially-intuitive interactions in the physical toy, Monkey Blocks
A related Samples Friedman test was run to determine if there were any
statistically significant differences between the numbers of intuitive interactions,
non-intuitive interactions, and partially-intuitive interactions in children playing with
each game.
The related Samples Friedman test works by ranking scores of dependent
variables (number of intuitive interactions, number of non-intuitive interactions and
number of partially intuitive interactions), all combined, according to its value, with
the smallest rank assigned to the smallest value. The ranks obtained for each of the
DVs are then averaged. This results in a mean rank for each of the DVs. The null
hypothesis of the Friedman test is that the mean rank is the same for all the DVs.
However, if one of the DVs tends to have higher values than the other DVs, it will
have been assigned higher ranks and will have a higher mean rank (and vice versa for
the DV with lower scores). It is this difference in mean rank that is tested by the
Friedman test for statistical significance. This approach was used to determine
whether the differences in the DVs are statistically significant.
The mean ranks obtained for each of the dependent variables, numbers of
intuitive, non-intuitive and partially-intuitive interactions are presented in Table 9.
146 Embodied intuitive interaction in children
Table 9 Mean Rank Values of Number of Intuitive Interactions, Non-Intuitive Interactions and
Partially Intuitive Interactions
The mean rank values of intuitive, non-intuitive, and partially-intuitive
interactions were not same for both Osmo and Monkey Blocks. Thus, the differences
number of intuitive interactions, number of non-intuitive interactions and number of
partially intuitive interactions are statistically significant for Osmo, χ2(2)= 54.544, p
< 0.05 and Monkey Blocks, χ2(2)= 75, p <0.05.It is therefore evident from the results
above that both Osmo and Monkey Blocks demonstrate high numbers of intuitive
interactions.
Primary predictors of intuitive interaction
Having established that both Osmo and Monkey Blocks are substantially
intuitive, the second part of the analysis investigated primary predictors (in terms of
aspects of Embodiment) of intuitive use. Multiple regression analysis was used to
determine the proportion of variation in intuitive interaction that could be explained
by aspects of Embodiment.
A multiple regression was run to explain how much of the variation in intuitive
interaction with each toy could be explained by aspects of Embodiment. All predictor
variables (physical affordances, perceived affordances, emergence, scaffolding, and
co-operative activity) were added to the regression models; in other words, no
variables were dropped from the models. The partial regression coefficients and
Variation Inflation Factor (VIF) values are given in Table 10. The VIF values
obtained in regression analysis carried out for Osmo and Monkey Blocks were mostly
between 1 and 5, indicating very little multicollinearity between the predictors.
Physical affordances exhibited moderate multicollinearity, with a VIF value between
5 and 10.
Type of interactions
Mean Rank Values Osmo Monkey Blocks
Number of Intuitive Interactions
2.81 2.95
Number of non-intuitive interactions
1.94 1.52
Number of partially intuitive interactions
1.25 1.52
Embodied intuitive interaction in children 147
Table 10 Regression coefficients and VIF values for the MRS system
The aspects of Embodiment statistically significantly explained 52.4% of the
variability in intuitive interaction with the Osmo game, F(5,36) = 27.28, p<0.05; and
51.5% of the variability in intuitive interaction with Monkey Blocks, F(5,36) = 26.04,
p<0.05. Comparing the relative contributions of the aspects of Embodiment to the
intuitive interaction with the two systems, it was found that physical affordances
explained 36.06% of the variability in the intuitive interaction with the Osmo game.
The results for the contribution of all aspects of Embodiment are shown in Figure 24.
Similarly, in the case of Monkey Blocks, physical affordances explained 42.42% of
the variability in intuitive interaction. The contribution of all aspects of Embodiment
is shown in Figure 25.
Toy Predictor variables -aspects of Embodiment
Partial Regression Coefficients
Variation Inflation Factor (VIF)
Osmo Physical affordances 0.3603 8.925 Perceived affordances 0.3918 1.527 Emergence 0.2549 3.670 Scaffolding 0.1602 4.948 Co-operative activity 0.6918 3.418
Monkey Blocks
Physical affordances 0.4242 6.614 Perceived affordances 0.1347 1.808 Emergence 0.2155 3.750 Scaffolding 0.1232 2.853 Co-operative activity 0.099 5.585
148 Embodied intuitive interaction in children
Figure 24 Relative contributions of aspects of Embodiment to intuitive interaction with TEI game
Figure 25 Relative contributions of aspects of Embodiment to intuitive interaction with Monkey Blocks
Due to multicollinearity, the contributions of aspects of Embodiment to
intuitive interaction do not total 100%. The existence of multicollinearity and the
Embodied intuitive interaction in children 149
VIF values indicate that a certain level of correlation exists between the predictors,
especially between physical affordances and other predictors. A Spearman's rank-
order correlation was run to assess the relationship between the predictors (Table 11).
Table 11 Correlations between aspects of Embodiment (predictors)
For Osmo, there was a strong positive correlation between physical affordances
and emergence, rs(40) = 0.867, p < .01; physical affordances and scaffolding, rs(40) =
0.952, p < .01; and physical affordances and cooperative activity, rs(40) = 0.728, p <
.01. There was a moderate positive correlation between physical affordances and
perceived affordances, rs(40) = 0.506, p < .05.
For Monkey Blocks, there was a strong positive correlation between physical
affordances and emergence, rs(40) = 0.755, p < 0.01; physical affordances and
scaffolding, rs(40) = 0.801, p < 0.01; and physical affordances and cooperative
activity, rs(40) = 0.885, p < 0.01. There was a weak positive correlation between
physical affordances and perceived affordances, rs(40) = 0.375, p < 0.01.
There was a weak positive correlation between perceived affordances and other
aspects of Embodiment. There was a strong correlation between emergence and
Toy Predictors of intuitive interaction
Physical affordances
Perceived affordances
Emergence Scaffolding Co-operative activity
Osmo Physical affordances
0.506 0.867 0.952 0.728
Perceived affordances
0.506 0.388 0.461 0.403
Emergence 0.867 0.388 0.825 0.564
Scaffolding 0.952 0.461 0.825 0.731
Co-operative activity
0.728 0.403 0.564 0.731
Monkey Blocks
Physical affordances
0.375 0.755 0.801 0.885
Perceived affordances
0.375 0.665 0.310 0.293
Emergence 0.755 0.665 0.777 0.715
Scaffolding 0.801 0.321 0.777 0.794
Co-operative activity
0.885 0.293 0.715 0.794
150 Embodied intuitive interaction in children
scaffolding, emergence and co-operative activity, and scaffolding and co-operative
activity.
7.3 DISCUSSION
The TEI, Osmo, consisted of both physical and virtual elements, while the
physical toy, Monkey Blocks, consisted of physical elements only. The results in
Section 7.2.2 indicate that both games demonstrated more intuitive interactions than
non-intuitive and partially-intuitive interactions.
Results suggest that physical affordances were the primary predictor of
intuitive interaction in both games. Perceived affordances were the second highest
predictor of intuitive interaction with the Osmo game, but was the lowest contributor
to variability in intuitive interaction with Monkey Blocks. Emergence, scaffolding,
and co-operative activity were the next highest contributors to variability in
children’s intuitive interaction when playing with both games, and showed high
correlation with physical affordances.
Children used physical, material, and spatial properties of physical and virtual
elements in the systems to interact intuitively with them. They used the spatial
orientation of the targets and the balls on the screen (virtual elements), and the
physical and material properties of the objects in front of the screen, to obtain the
optimum angular position of the objects to guide the balls onto the targets. With
Monkey Blocks, the children used the spatial and material properties of the blocks to
stack them in an arrangement.
The materiality of the blocks played a crucial role in children’s play with
Monkey Blocks. They figured out that the orange and green blocks were heavier than
the blue blocks by feeling their weights. Most children, however, could not figure out
the difference between the orange and green blocks. While the two blocks weighed
the same, a weight was embedded at one end of the orange block, and another at the
centre of the green block. The children’s perception of the weights of the blocks,
therefore, depended on the way they held them in their hands. They were observed
holding the blocks in their two hands, and then holding their ends to compare the
weights of the blocks. Some children figured out the heavier end of the orange block;
some believed that the orange block was heavier than the green block; some believed
that the green block was heavier than the orange; and some believed that there was
Embodied intuitive interaction in children 151
no difference between the two. None of the children said that the two blocks weighed
the same. Children developed their strategies to complete the arrangement according
to their own interpretation of the weights.
Perceived affordances were the next prime contributor to variability in
children’s intuitive interaction with the Osmo game. In the absence of real
affordances in the virtual elements, they resorted to interpreting deliberate clues in
the system. They used their past experience and knowledge to interpret the deliberate
clues in the virtual elements. For example, some children had personally experienced
similar scenarios of matching objects to same-coloured targets while playing other
games.
Most of the deliberate clues in the Osmo game were simulations of
functionality of elements in action that were known to children from their past
experience and knowledge. Simulations of the functionality of a teleporter, spinning
fans, and targets disappearing when hit by the balls, are examples of simulated
actions. The perceived affordances was based on the experience and knowledge of
how things (such as teleporters) work. Children were able to identify the
functionality of the teleporter when they noticed that the balls guided to a bowl
resulted in balls emerging from another bowl. Some children had never heard of
teleporters; however, they were still able to detect the functionality of the deliberate
clues simply by seeing it in action. Pezzulo (2011) referred to such deliberate clues
as ‘Embodied representations’. Children perceived these deliberate clues by using
their past experience and knowledge.
The simulations of the virtual elements were often initiated through physical,
direct manipulation of objects. Children were using spatial orientation of the virtual
elements—such as balls and targets on the screen and the spatial and physical
properties of the objects—to decide on the manipulations of the objects. This meant
that while the manipulations of the physical objects were determined by the physical
and spatial properties of the objects, they also required interpretation of the deliberate
clues in the virtual elements. Gaver (1991) referred to such affordances as ‘sequential
in time affordances’. This explains the moderate correlation between physical
affordances and perceived affordances in the Osmo game.
Perceived affordances was the lowest contributor to the variability in intuitive
interaction with the Monkey Blocks. Children mostly relied on the natural clues in the
152 Embodied intuitive interaction in children
form of physical and material properties of the blocks to create an arrangement.
Although some children could not detect the difference in weights and location of
weights in the blocks, they were still able to form an arrangement, using the physical
affordances of the blocks. Those who were able to detect the difference in weights in
the blocks, and the location of the weights inside the blocks, were using their past
experience playing with blocks and their knowledge of weights. Children arranged
lighter blocks on the top of the heavy blocks. When a block had to be balanced
horizontally on one end, some children placed a heavier block on top of the end of
the horizontal block. One child compared this to the functionality of a paper weight.
Some children said that they learned about heavy and light weights at school,
but they could not explain exactly what and when. Studying the attributes of heavy
and light objects is covered in the early numeracy and mathematical understanding
section of the Queensland education curriculum (Queensland Studies Authority,
2006). Some children used the orange block as the horizontal block, and balanced it
by placing its heavier end on the vertical block. The knowledge required to interact
intuitively with Monkey Blocks, a system with only physical elements, was based on
facts pertaining to weights that they had learnt at school. These facts were not
revealed through physical manipulation of the blocks at the start of the game at the
least. This explains the weak correlation between physical affordances and perceived
affordances in the Monkey Block game.
There was a strong correlation between physical affordances and other aspects
of Embodiment, namely, emergence, scaffolding, and co-operative activity for both
the games. The environment is the crucial influencing factor in developing this
relationship, as children use elements in the environment to play together. They
manipulated objects to effect change in the layout and structure of the elements in the
physical and virtual spaces (physical space only in the case of Monkey Blocks). This
resulted in change in affordances offered by elements in both spaces. This, in turn,
changed the manipulations of the objects. Some of the changes in affordances
resulted in manipulations of the objects that were not anticipated by the game
designers. Thus, emergence was the next most important contributor to variability in
children’s intuitive interaction.
Scaffolding and co-operative activity were the next most important
contributors to variability in intuitive interaction. Children used their hands, and the
Embodied intuitive interaction in children 153
space and people around them to support their play. They used symmetry of elements
in the game play to imitate another child’s strategies and play. The imitation was a
confident decision, rather than a vague mimic of another child’s actions. Children
played together, co-operating with each other, or getting out of the way if they were
impeding another child’s play. They developed strategies focussed on reaching a set
goal together. There was a high positive correlation between physical affordances
and scaffolding, and between physical affordances and co-operative activity. The
physical and material properties of elements in the system allowed children to
support each other’s game play, and to co-operate and collaborate.
In conclusion, physical affordances was shown to be the prominent facilitator
of children’s intuitive interaction with TEIs and physical products. This, in turn, has
an impact on emergence, scaffolding, and co-operative activity due to the correlation
of these aspects with physical affordances. These aspects are important because they
could have an impact on people’s intuitive experiences; this topic, however, remains
an area for future research. Products could be designed for intuitive interactions but
that depends on the how the deliberate clues are used in design. This statement is
supported by the fact that Embodied representations in deliberate clues have emerged
as a prominent factor in the use of perceived affordances in intuitive interactions with
TEIs with virtual elements. One of the ways of incorporating Embodied
representations is through the use of sequential affordances.
7.3.1 Implications for Design
Based on the above discussion, the following guidelines were developed to
assist in designing Embodied intuitive products for children:
Physical affordances should be used in design to offer natural clues for intuitive
interaction - Natural clues allow children to use their sensorimotor knowledge and
Embodied experiences for intuitive interaction. This study found that intuitive
interaction in both physical products and TEIs is primarily due to physical
affordances. It is the direct interaction and manipulation of the physical elements that
facilitates intuitive interaction. In physical products, the feedback from the system is
in high conformance with the action performed on the system. Thus, TEIs, where
physical and virtual elements co-exist, could be designed for intuitive interaction by
facilitating interaction with the system through interactions with the physical
elements that allow children to take advantage of their affordances. The virtual
154 Embodied intuitive interaction in children
system in the TEI (if present) could be used to provide feedback on the manipulation
of, and interaction with the physical elements.
Where physical affordances cannot be used in design, perceived
affordances should be used to offer deliberate clues for intuitive
interaction – Clues, both deliberate and natural clues allow children to use
their experiential knowledge for intuitive interaction. To facilitate this
interaction, virtual elements in the interface should be designed with
children’s past experience and prior knowledge in mind. However, this is
easier said than done, as children accumulate different experiences and
knowledge. To allow children to interpret the symbolic clues correctly (to
avoid any false affordances), scaffolding could be provided. For example,
in Osmo, children found it difficult to determine how to spin a virtual fan,
even though symbolic clues in the form of a ‘SPIN’ prompt were provided
on the virtual interface. Children were only able to figure out how to spin
the fan when the balls started to fall on it, and it began to spin. Embodied
representations such as animated clues (Uden & Dix, 2000), that is, clues
that depict or represent the action to be performed, are preferred over
symbolic clues, as children can use both their sensorimotor and
experiential knowledge to interpret and understand these deliberate clues.
Interactions in physical and virtual spaces in TEIs should be in the
same dimensions - This guideline mostly applies to TEIs, where the
coupling of the physical and virtual poses the biggest challenge (physical
products do not suffer from coupling issues). One of the ways to resolve
coupling issues is to segregate the physical and virtual spaces so that
interactions in both spaces are in the same dimensions. For example, if the
interactions in the virtual space are in two dimensions (such as on a tablet
screen), limiting the interactions and manipulations in the physical space
to two dimensions (such as by moving objects on a horizontal plane such
as a table) helps children to traverse the coupling between the physical and
virtual spaces.
Use both physical and perceived affordances (sequential in time,
or/and nested in space) to couple physical and virtual spaces in TEIs -
Again, this guideline mostly applies to TEIs. Actions performed in the
Embodied intuitive interaction in children 155
physical space (in response to physical affordances of the physical
elements) result in clues for actions in the virtual space. These are
‘sequential in time’ affordances. Affordances could also be nested in
space; that is, where both physical and perceived affordances are present at
the same time but in different locations (one in physical space and the
other in virtual space, for example), and are linked to the same objective or
action that they afford.
The size of physical and virtual interaction spaces should be calibrated
for boundary limits - Separating the virtual and physical space in a TEI
has its own limitations. The size of each space in TEIs is usually not the
same. This could result in physical manipulations and interactions in the
physical space going beyond the limits of the virtual space. One way to
resolve this is to assign boundaries to the physical space in relation to the
virtual space. For example, in Osmo, a simple calibration of the physical
space with respect to the virtual space, and drawing lines on the table to
limit manipulations within these boundaries, would solve the problem.
Alternatively, a feedback from the system to move back within the
boundary limits could help train people to determine the boundaries.
7.4 SUMMARY
Having established (in Chapter 6) that physical products have aspects of
Embodiment which allow children to intuitively interact with them, this chapter has
presented a within-subject study of two systems: one physical and the other a TEI,
both of them requiring interaction with physical objects to achieve game objectives
and goals. Experiment 2 determined which of the design aspects of Embodiment
discussed in Section 4.2, have a significant impact on children’s intuitive
interactions. The TEI game Osmo had virtual elements as well as the physical
elements, while the physical game, Monkey Blocks consisted of physical elements
only.
The results show that physical affordances was the primary predictor of
intuitive interaction with both systems. Perceived affordances was the next most
significant predictor of intuitive interaction with the Osmo game, but contributed
least to variability in intuitive interaction with Monkey Blocks. The prominent use of
156 Embodied intuitive interaction in children
perceived affordances in Osmo was due to the use of Embodied representations
incorporated in the design through sequential affordances. On the other hand,
intuitive interaction with Monkey Blocks was mostly due to natural clues in the
system. Children used their past experience and knowledge to play with both Osmo
and Monkey Blocks.
Emergence, scaffolding, and co-operative activity were the next most
significant contributors to variability in children’s intuitive interaction with both the
systems, and showed a high correlation with physical affordances. These
relationships establish the scope for future research in the use of these aspects to
facilitate intuitive experiences.
Chapter 8 now discusses the design implications of the results of Experiment 1
and Experiment 2. The findings from Experiment 1 and Experiment 2 were
integrated into Blackler's (2008) continuum of intuitive interaction. The Model of
Embodied Intuitive Interaction (MEII), an interaction model that provides a reference
for the design and development of Embodied intuitive products for children is also
discussed.
Embodied intuitive interaction in children 157
Chapter 8: Discussion
This research study has investigated children’s Embodied intuitive interaction.
Physical products, virtual interface, and a TEI were used to study children’s
Embodiment and intuitive interaction. Experiment 1 (Chapter 6) compared a physical
Jenga and a virtual interface for Embodiment and intuitive interaction. The results
suggest that the children interacted more intuitively with the physical Jenga than the
virtual interface. In terms of Embodiment in children’s interaction with the two
products, physical affordances, due to the spatial and material properties of the
physical artefact, contributed to intuitive interaction with the physical Jenga. The
intuitive interaction with the virtual interface, on the other hand, was mostly due to
perceived affordances.
Experiment 2 ( Chapter 7) investigated aspects of Embodiment that contribute
to variability in children’s intuitive interaction with physical product and a TEI. The
findings suggest that physical affordances was the primary contributor to intuitive
interaction with both the products. Perceived affordances was the second most
contributor in Osmo while it was the least contributor in Monkey Blocks. Embodied
aspects could thus facilitate intuitive interaction and efforts should be made to
incorporate these aspects in design.
This chapter discusses the design implications of the results of the two
experiments (in Section 8.2). The results are discussed in relation to the continuum of
intuitive interaction, and the findings are integrated into this continuum. Section 8.2
discusses a model for children’s Embodied intuitive interaction—MEII. This model
provides a reference for designers and researchers to develop Embodied intuitive
products for children.
8.1 DESIGN IMPLICATIONS
This study has found that physical products have properties that make them
intuitive to use (Desai et al., 2015). It further found that there are more aspects of
Embodiment used in children’s intuitive interaction with physical products than with
virtual interfaces. Physical affordances is the primary contributor to children’s
intuitive interaction with physical products, while they used perceived affordances to
158 Embodied intuitive interaction in children
intuitively interact with the virtual interface. Other aspects of Embodiment—
emergence, scaffolding, and co-operative activity—showed correlations with
physical affordances. These results can be explained in relation to the continuum of
intuitive interaction (Section 8.1.2).
8.1.1 Children’s intuitive interactions with physical and virtual interfaces
Physical products are associated with low domain transfer distance, the
distance between the application domain and the origin of prior knowledge
(Diefenbach & Ullrich, 2015). In physical products, the origin of children’s prior
knowledge, and the application of that knowledge, both relate to the same physical
domain with spatial and material characteristics. Low transfer distance results in less
verbalisation, and effortless use of the interface (Diefenbach & Ullrich, 2015). Less
verbalisation (Swaak & De Jong, 2001) and effortless use (Alter, Oppenheimer,
Epley, & Eyre, 2007), in turn, are associated with intuitive thinking. Thus, low
transfer distance in physical products results in intuitive interactions in children.
Maier and Fadel (2009) suggested that the relationship between artefacts
contributes to affordances, as the interaction between the artefacts offers various
possibilities for action. Maier et al. (2009) referred to this relationship as ‘Artefact-
Artefact-Affordances (AAA)’. Although Maier et al. described AAA as a
relationship between artefacts that are identical (for example, a chair stacks on
another chair), the concept can be extended to artefacts that are not identical, such as
the mix of physical objects and virtual elements in Osmo. Physical Jenga blocks
consist of identical blocks. The spatial and material properties of the blocks and the
stack provided the children with various possibilities for action, as did the other
blocks. The spatial layout of the stack offers possibilities for other blocks to be
placed in certain positions ( Figure 26). In Monkey Blocks, the blocks are colour-
coded as per the location of weights inside them, and the children feel the material
difference. In this case, the material and spatial properties of the blocks determine the
possible actions. Children feel the weights of the blocks, and place them horizontally
on the vertical stack. Then they slide the blocks at different angles until they feel that
the block will remain stable (Figure 26, top).
Embodied intuitive interaction in children 159
Figure 26 Artefact-Artefact Affordances: stack-ability and slide-ability of blocks in Monkey Blocks (above); pull-ability and slide-ability of blocks in Jenga (below)
There are three types of artefact-artefact interactions in TEIs - physical to
physical, physical to virtual, and virtual to virtual. In Osmo (Figure 17), the geometry
of physical objects (such as pencils and erasers) allows other similar objects to be
aligned with them to deflect the virtual balls. A copy of the physical manipulations is
created in the virtual space, and facilitates physical-virtual interaction in the system.
The physical manipulation of objects (physical-to-physical interaction) then results in
changes to the virtual elements (virtual-to-virtual interactions). The virtual balls, for
example, are deflected to hit virtual targets.
Children’s actions on physical objects, and the objects’ responses, are spatially
compatible; therefore, the manipulation of these physical objects and their responses
are easily anticipated. This results in high degrees of conformance (Beaudouin-
Lafon, 2004)—the extent of similarity between a user action on a domain object and
the resulting response. Sliding a block in the physical Jenga toy to the left, moves the
block to the left. The response to children’s action in virtual interfaces depends on
the design of the interface, the technology (e.g. touchscreen), and the system
configuration. Children are thus less able to reliably anticipate the response from a
virtual system. The high degree of conformance in physical products, on the other
160 Embodied intuitive interaction in children
hand, results in anticipated responses to manipulations of the blocks; this, in turn,
results in a higher number of intuitive interactions.
Virtual interfaces are instrumental interactive systems, where a computing
element is treated as a tool that acts as a mediator between users and virtual elements
(Beaudouin-Lafon, 2004). In the virtual app, children’s interaction with the stack of
blocks is through the touch screen interface of the tablet. The tablet acts as a
mediator between the children and the virtual stack of blocks. Children are
effectively interacting with two systems simultaneously—the tablet, which has its
own system of operation and perceived affordances (e.g. swiping) and the game,
which has perceived affordances transferred from other games and elsewhere (e.g.
colour codes based on population stereotypes). This is likely to result in high degrees
of indirection, and a high transfer distance. Physical products, on the other hand, do
not require any mediators; rather, they involve direct interaction with the system and
its elements.
The actions offered by the mediator, and the actions offered by the virtual
elements, represent a relationship between the artefact and the children. This is
referred to as Artefact-Children-Affordances (ACA) (Maier & Fadel, 2009). Often,
these actions are contradictory, resulting in unsuccessful or incorrect interactions.
For example, tapping on the blocks in the Jenga app, and swiping left and right on
the tablet screen, were contradictory actions resulting in the stack falling. This
suggests that the elements of a system should either invoke consistent actions, or the
app should be designed to take this inconsistency into account. The blocks and the
stack in the virtual Jenga game offer the possibilty of actions such as push, pull, and
stack on the other blocks (AAA); however, children were found swiping their fingers
on the tablet, and tapping on the blocks (ACA). It was only the tap action in the app
(ACA) that was consistent with the push action with the physical Jenga blocks
(AAA). This means that the actions that elements offer to other elements in the
system (AAA) should be consistent with the actions that they offer to the children
(ACA).
8.1.2 Relationship with the continuum of intuitive interaction
Building upon past work (Blackler & Hurtienne, 2007) that compared and
contrasted the two separate continua of intuitive interaction (as discussed in Section
2.3.2), the results from Experiment 1 and Experiment 2 have been used to present an
Embodied intuitive interaction in children 161
Enhanced Framework for Intuitive Interaction (EFII) (Blackler et al., 2018). Figure
27 is adapted from EFII, where children’s Embodied intuitive interaction with
physical products, virtual interfaces, and TEIs are illustrated in relation to Blackler's
(2008) continuum of intuitive interaction and the German-based Intuitive Use of
User Interfaces (IUUI) research group’s continuum of knowledge in intuitive
interaction. The aspects of Embodiment discussed in this research study are
highlighted in red in Figure 27, which presents their relationship with the two
continua. The latter are both shown in grey. The Enhanced Framework for Intuitive
Interaction (EFII) highlights parallels and connections between the different
dimensions of intuitive interaction shown on the right-hand side - pathways to
intuitive use, interface types, characteristics of features, and the origin of the
knowledge that enables intuitive use.
Figure 27 Research results incorporated into an Enhanced Framework of Intuitive Interaction (EFII) (adapted from Blackler et al., 2018)
Blackler’s (2008) continuum of intuitive interaction (Figure 2) is re-
conceptualised as ‘pathways to intuitive use’ and shown alongside is the ‘origin of
knowledge that enables intuitive use’ (Figure 27). The level of physicality and level
of virtuality determines the pathways to intuitive use, characteristics of features and
type of knowledge leveraged. Physical products are on the extreme left of EFII,
162 Embodied intuitive interaction in children
while virtual interfaces are on the extreme right. TEIs are between these extreme
ends, and represent products with both physical and virtual elements. TEIs towards
the left extreme represent TEIs with more interactions with the physical, while TEIs
towards the right comprise more interactions with the virtual.
Children mostly relied on physical affordances to interact with the products
that were more physical (such as grasping, holding, taping, and sliding the physical
blocks). They mapped the physical and material properties of the objects (such as
looseness, shape, weight) onto decisions and actions (for example, to remove, stack,
and align the objects). Experiment 1 found some use of perceived affordances with
the physical toy; however, most of the overall uses and intuitive interactions were
facilitated by physical affordances. Children can, however, also leverage cultural
conventions and metaphors (e.g. turning a wheel, using a racket to play tennis,
balancing blocks one above the other to create a stack).
In contrast, children mostly used perceived affordances to interact with
interfaces towards the right of the EFII which, depending on the system and its
design, are more virtual. Virtual elements within these interfaces do not have real
physical affordances. Children thus rely on their past experience and knowledge
acquired from playing other games, both physical and virtual. Children were not only
interacting with the virtual elements within the virtual interface, but also with the
mediating element (e.g. the tablet in Experiment 1). They used cultural conventions
associated with tablets (such as swiping left-right on the screen), and conventions
associated with physical blocks and stacks (such as tapping on the blocks).
Children’s intuitive interaction with virtual interfaces is thus associated with
population stereotypes (e.g. the colour codes in the app) and perceived affordances
on the EFII (Figure 27).
Physical products (left of the EFII in Figure 27) are associated with low
domain transfer distance and high degree of conformance. As interfaces become
more virtual, transfer distance could increase and the degree of conformance could
decrease, depending on the way physical and virtual elements are configured within
the system. Familiar features and metaphors are used as the pathways to intuitive
interaction in interfaces as they become more virtual. Presence of natural clues
become less as interfaces become more virtual and are replaced with deliberate clues
which rely on familiarity and previous knowledge of the users to determine the
Embodied intuitive interaction in children 163
actions afforded by these clues. This is generally the case with more complex
systems that cannot be simply mapped at the sensorimotor level (such as accounting
software or menu-driven role playing games). In these cases, the level of previous
domain experience is more relevant in determining the intuitive potential of the
system. Again, however, metaphors could be used as a bridge to more ubiquitous
knowledge.
In relation to directly manipulated TEIs (shown in Figure 27), children
primarily used physical affordances in intuitive interaction with the physical
elements in TEIs, and perceived affordances to interact with the virtual elements of
the system. Thus, they could access any part of the continuum to interact intuitively
with TEIs, depending on the design and configuration of the physical and virtual
elements in the system. Emergence, scaffolding, and co-operative activity are
represented on the extreme left of the EFII, as Experiment 2 showed that they are
strongly correlated to physical affordances. The EFII thus suggests that children’s
Embodied intuitive interaction is mostly related to the left of the framework, and is
mostly associated with sensorimotor knowledge.
Children use their Embodied experiences and sensorimotor knowledge to
intuitively interact with physical products towards the left of the EFII. This
experience and knowledge is available everywhere, and at any time, to assist them to
interpret the natural clues to possible action on the physical elements that physical
affordances offer them. This explains the high ubiquity of previous experience for
the left end of the EFII, for more physical products. Population stereotypes are the
next most ubiquitous in terms of availability of experience and knowledge to assist
children to interact intuitively with the interfaces. Cultural experiences and
knowledge within a group of children are readily accessible and available. Ubiquity
of previous experience decreases towards the right end of the EFII, as children might
or might not recognise the familiar features, depending on their pattern of previous
experience. Children might not have the knowledge to interpret the deliberate clues
that perceived affordances must offer in order to interact intuitively with the
interface. Therefore, ubiquity is important for children as they have less overall
experience to draw on.
Metaphor is an exception here, as image schemas and Embodied metaphors
(for example ‘up’ to increase, ‘progress’ to travel along a path) could be more
164 Embodied intuitive interaction in children
ubiquitous as they could be derived from children’s sensorimotor knowledge and
Embodied experiences. Hurtienne, Klöckner, et al. (2015) showed that image
schemas can be more ubiquitous because they use metaphorical extensions of basic
mental concepts and, hence, can be intuitive as well as inclusive. Image schemas are
arguably universal, so sit on the far left. Metaphor has been slightly detached from
the other parts of Blackler & Hurtienne's (2007) original continuum (Figure 2). This
is because it is not always a simple continuation of the other concepts and, in fact,
could be applied in other ways than originally assumed by Blackler & Hurtienne
(2007). The extension of the metaphor block beneath the continuum is intended to
demonstrate that metaphor can, in fact, be applied through both physical and
perceived affordances.
Children could easily interpret some metaphors if they are derived from
everyday experiences, while other metaphors could require them to develop expertise
and understanding to be able to use them intuitively. For example, the verbal and
symbolic reference to all that is good as ‘up’ or ‘high’, such as describing good
evaluations as ‘high marks’, or a ‘thumbs up’ gesture (Meier & Robinson, 2004); and
the use of a letter box icon for email on an interface developed for children (Uden &
Dix, 2000). However, metaphors such as a cracked wine glass to convey fragility
require children to develop an understanding through practice and learning. Uden &
Dix (2000) found that children were unable to interpret icons with images of objects
that they have not encountered before (such as a typewriter), or objects that they do
not associate with the activity (such as a fountain pen). Such metaphors could be less
ubiquitous in experience and knowledge.
8.2 MODEL FOR EMBODIED INTUITIVE INTERACTION (MEII) FOR CHILDREN
One of the outcomes of this research study is the model for Embodied Intuitive
Interaction (MEII), an interaction model to guide the design of children’s products
that facilitate Embodied intuitive interactions. Buur & Andreasen (1989) described
design models as representations of the properties of products that designers create.
These representations provide insights into the properties of the product that is being
designed (such as insights into products’ usability, sustainability, ergonomics and so
on). Interaction models represent ways in which interactions with certain
properties—such as intuitiveness (Blackler, 2008); Embodiment; thoughtfulness
Embodied intuitive interaction in children 165
(Löwgren & Stolterman, 2004); seductiveness resulting in playful, fun, and effective
user experiences (Anderson, 2011)—can be designed. Interaction models take
various forms such as mathematical formulae, verbal descriptions, sketches, block
diagrams, and functional models, depending on the expertise of the designer and the
abstraction level of the design process. The model for children’s Embodied intuitive
interaction is presented in the form of two block diagrams, one for perception (Figure
29), and the second for action (Figure 30). The derivation of these models is
discussed below.
A systems-based approach is used to model children’s Embodied intuitive
interactions. This approach allows the representation of relationships between
systems and sub-systems, as well as representation of children’s activities on
individual sub-systems or whole systems (Churchman, 1971). If a sub-system is
further investigated in future research, it can be represented by more subsystems. The
model would then allow encoding of the qualities and properties of the interactions
into a formal system (mathematical objects), design prototypes, or design tool kits. It
would then be possible to verify system's properties using reliable methods such as
formal methods (for mathematical encodings) and usability methods.
Section 8.2.1 discusses the development of the model and Section 8.2.2
describes the perception (Figure 29) and action models (Figure 30).
8.2.1 Embodied Cognition as a perceptual system
Embodiment theory claims that thoughts and actions are the results of the brain
and the body working together in cognitive information processing. This has been
discussed in the literature review (Section 2.1). Wilson & Golonka (2013) described
Embodied cognition as a process of a continuous loop of perception and action in an
environment. The perceiver deciphers the environment and makes decision on the
actions to be performed on the environment. The brain plays an important role as
well, but the environment is the originating source of the perception action loop and
not the brain (Wilson & Golonka, 2013). The brain and the body (together
represented by the perceiver) and the environment, all work together as a dynamical,
perceptual system (Hinton, 2014) which is presented in Figure 28.
Children are compulsive interpreters; they try to sense and perceive everything
around them (Andersen, 2001). Rosen (2012) describes this tendency as
166 Embodied intuitive interaction in children
characteristic of a perceptual system. Children interacting with the environment,
perceiving it and acting upon it, form a basic cognitive unit (Gaines, 1989), as shown
in Figure 28. Perceptual systems are internal predictive models of themselves and/or
of their environment that utilise the models’ predictions to control their present
behaviour. Children and all biological systems assess their current change of state
depending not only on past and present circumstances, but also on the future. Such
systems are referred to as ‘natural perceptual systems’.
Children perceive the qualities in the external world through their sensory
systems. Changes to sensory systems due to change in the external environment
result in percepts; that is, mental impressions of things perceived by the senses
(Rosen, 2012). Children process and organise the relationships among these percepts,
and then interact with the external environment through effector mechanisms (such
as hands). Such interactions cause changes in the external environment, which are
then re-perceived (Figure 28).
One of the qualities of perceptual systems is that they are socially distributed as
they experience the environment through physical objects and social activities with
other organisms in the world. Perceptions and actions with the social part of the
environment are evaluated differently to those in the physical artefact part of the
environment (Gaines, 2013). The primary objective of any social interaction is the
explanation and understanding of human actions. Children as distributed perceptual
systems, and the separation of experiences in the environment into physical and
social experiences, are shown in Figure 28.
Embodied intuitive interaction in children 167
Figure 28 Embodied Cognition as distributed perceptual systems, (adapted from Gaines (1989) and Hinton (2014))
The social experiences in the environment represent the other child with whom
every participant was paired in Experiment 1 and Experiment 2. Children-to-social
world interactions in this research study are children-to-children interactions that
occur through the cooperative activity aspect of Embodiment (that is explained in
Section 8.2.2). Perceive and act form the basis of the sensory perceptual systems that
process children’s sensorimotor knowledge, Embodied experiences, and experiential
knowledge. Thus, distributed perceptual systems are ideal starting blocks for
representing the findings of this research study in an interaction model. The MEII
consists of two parts: one representing children perceiving (Figure 29) and the
second one representing acting on artefacts and the social world (that is, other
children and the environment) (Figure 30). These two parts of the model are
discussed below.
8.2.2 Incorporating Embodied aspects of children’s intuitive interaction into the model
Direct interaction and manipulation is associated with a direct relationship
between a child’s action on an object and the outcome of that action. The Model for
Embodied Intuitive Interaction (MEII) represents children perceiving (Figure 29) and
acting (Figure 30) on artefacts and the social world (that is, other children and the
environment). The perceive side of the MEII (i.e. children’s perception of the
physical world and the social world) is shown in Figure 29. Artefacts represent
168 Embodied intuitive interaction in children
products such as physical products, TEIs, and virtual interfaces (shown on physical
virtual continuum of Figure 1).
Figure 29 Perception side of the Model for Embodied intuitive interaction (MEII) in children
Affordances, both physical and perceived, offer children clues as to how they
might interact with products. Children used natural clues that represent the natural
properties of the elements within the artefacts to interpret actions physical
affordances. On the other hand, children used deliberate clues that are incorporated
into the design of artefacts and natural clues to interpret perceived affordances. This
enabled children to understand, learn, and interpret the interactions that they need to
perform on the artefacts. Two types of deliberate clues have been identified in this
research study: Embodied representations that are simulations of the functionality of
symbols (for example, the simulation of balls hitting a fan and, in turn, resulting in a
spin of the fan in the game of Osmo); and symbolic clues (for example, colour codes
on the blocks and the crash symbol to warn of the danger of the stack falling over in
the Jenga app) that depend on children’s past experiences with such symbolic
representations.
Embodied intuitive interaction in children 169
Both natural and deliberate clues are important. If these clues are not reliably
and accurately interpreted for an appropriate interaction, the affordances could either
remain hidden, or result in actions that are incorrect or/and unexpected (false
affordances). For example, swiping left and right on the screen in the Jenga app
rotates the stack of blocks 360 degrees; however, the affordances remained hidden as
children could not interpret and understand the clues to swipe for the intended
purpose. While playing Osmo, some children tapped on the balls on the tablet screen
to direct them to the targets, instead of using the physical objects in the physcial
space. This was a false affordances that children quickly learned about. They then
started playing the correct way, manipulating the physical objects in the physical
space. Hidden and false affordances can result in mistakes and confusion, and thus
contribute to non-intuitive interactions.
Children use epistemic actions on artefacts to find solutions more quickly and
easily. This process frees their minds of the cognitive load. For example, children
moved the physical objects around in Osmo to find the right alignment and
distribution of objects to guide the balls onto the target in the virtual space. Similarly,
they were also seen touching and feeling the Jenga physical blocks to determine
which block to remove from the stack. Thus, scaffolding is represented in the
perception section of the MEII (as shown in Figure 29).
Children work with each other to perceive and decide on the actions to be
performed on the artefacts. Thus, the relationship between children and the social
world during perception phase is through co-operative activity (Figure 29). Children
work with other children and the environment to interact with the artefact. When
children are involved in social play with another child, their interactions with the
game elements are influenced by the interactions of the other child. Each of their
interactions leaves a clue (in some form) for the other children to use during their
interactions. These clues could be in the form of natural clues (such as the shape,
texture, and weight of the design elements), or in the form of symbolic or Embodied
representations. These clues could also be in other forms such as facial expressions,
verbal directives, or in the change in tone of a child’s voice.
Once the children have perceived the clues offered by physical and perceived
affordances, they perform actions on the artefacts and the social world (that is, other
children and the environment) (Figure 30). Cooperation could be in the form of
170 Embodied intuitive interaction in children
physical support or intervention, or in the form of verbal dialogue and discussion.
Children use scaffolds to interact with the artefacts. These scaffolds could be tools
and processes that assist them to perform actions on the artefacts. For example, in
the game of Osmo, while trying to guide the balls onto the targets in the virtual
space, children placed objects on the sides of the physical space to block the balls
from escaping from the virtual space. They used A3 paper edges in the physical
space to guide balls onto the targets in the virtual space.
Children also used imitation strategies as scaffolds in intuitive interaction.
While playing with Monkey Blocks, for example, some saw the symmetry in the
layout and imitated the other child’s strategies to create the other half of the
symmetry. These strategies pertained to the choice of block, and its placement or
position. Similar imitative strategies were used in the physical Jenga game, where
children imitated strategies pertaining to the removal of blocks (e.g. push and pull),
and strategies to decide which block to remove (e.g. tapping on the blocks, and the
visual assessment of the preference for certain blocks). Cooperative activity and
scaffolding are thus presented in the action part of the children’s MEII (see Figure
30).
Figure 30 Actions side of the Model for Embodied Intuitive Interaction (MEII) in children.
Embodied intuitive interaction in children 171
In the course of children’s interactions with artefacts, they also act on the social
world (other children and the environment). These interactions could be a verbal
interaction with other children, or a physical interaction that changes the nature of
the environment (such as the structure of the physical Jenga stack, the structure of
the layout in Monkey Blocks, or the layout of the physical and virtual spaces in
Osmo).
The affordances offered by the artefacts determine actions that can be
performed on them. These actions, in turn, result in changes in the properties of the
artefacts and the social world. Any change in properties of the artefacts and the social
world results in a change in their affordances (Maier & Fadel, 2009). Children’s
interactions with the artefacts and the social world change, and this can further
change their properties. This cyclic phenomenon, referred to as ‘emergence’, is a
result of actions performed on the artefacts and the social world as the result of the
affordances they offer. This phenomenon is represented by the emergence layer in
Figure 30.
As artefacts and the social world evolve, children’s interactions also evolve.
Thus, through a continuous process of perception and action, children develop
knowledge and understanding of various elements of the artefacts and the social
world (Allen & Strathern, 2003). They learn about physical elements, knowledge of
other people, and the behaviour of the environment. Children refine their modes of
perception and their interpretation of what they perceive. They are then equipped
with a better means of sensing and perceiving the environment; this results in
changes to their internal representations and models. Thus, it can be said that
emergence results in changes in existing knowledge, or the development of new
knowledge in children. This process is represented by ‘Knowledge’ in Figure 30.
Thus, the continuous updating of knowledge that translates perceptions into response
is a property of a complex and dynamic Embodied intuitive system that constantly
evolves over time.
The MEII is a continuous cycle of perceive and act. Perception requires the use
of affordances through natural clues, Embodied representations and symbolic clues.
These affordances facilitate children’s decisions and actions on the artefact, using
scaffolding and co-operative activity. Emergence is the cyclic phenomenon that
172 Embodied intuitive interaction in children
occurs as the result of continuous action-perception iterations that result in change in
affordances and, thus, changes in knowledge.
8.3 SUMMARY
This chapter has linked the results and outcomes of Experiment 1 and
Experiment 2 to determine the study’s overall implications for design. Children’s
Embodied intuitive interactions with physical products, virtual interfaces, and TEIs
were discussed in relation to Blackler's (2008) continuum of intuitive interaction, and
the German-based Intuitive Use of User Interfaces (IUUI) research group’s
continuum of knowledge in intuitive interaction resulting in an Enhanced Framework
of Intuitive Interaction (EFII). Using the concept of dynamic anticipatory systems,
MEII - an interaction model that represents children’s Embodied intuitive
interactions was presented. The model represents children perceiving (Figure 29) and
acting (Figure 30) on artefacts and the social world (that is, other children and the
environment). The use of aspects of Embodiment in relation to the model is
discussed.
Perceiving and acting happens in a continuous cyclic manner through use of
aspects of Embodiment. This cyclic process of perceive and act ultimately results in
change in knowledge, and/or in the generation of new knowledge. Chapter 9
concludes this thesis with an overview of its research contributions, outcomes,
limitations, and the possibilities it suggests for future research.
Chapter 9: Contributions and Future Work
Children’s increasing use of complex products and concepts was a driving
force that motivated this study that investigated Embodied intuitive interaction in
children through the lens of human-centred and child-centred design, keeping their
particular experiences and knowledge in mind. Embodied intuitive interaction has
immense potential to offer the design of children’s products. This research
established four objectives to investigate its specific role in the design process. A
Embodied intuitive interaction in children 173
methodology that enabled a thorough elicitation of the specific aspects of
Embodiment that facilitate children’s intuitive interaction was developed. This
methodology comprised data collection that used observations, retrospective
interviews, and co-discovery methods.
Two experiments were carried out: Experiment 1 compared physical and
virtual products for Embodiment and intuitive interaction (Chapter 6), and
Experiment 2 determined the primary predictors of variability in children’s intuitive
interaction (Chapter 7). Chapter 8 discussed the findings of Experiment 1 and
Experiment 2, and their implications for design. The results were then discussed in
relation to the continua of intuitive interaction resulting in EFII - Enhanced
framework of intuitive interaction. MEII – a model for Embodied intuitive
interaction, based on dynamic anticipatory systems, was developed and discussed
(Section 8.2.2). MEII offers an understanding of how children perform Embodied
intuitive interactions with artefacts and the social world.
This research study pioneers the study of Embodiment and children’s intuitive
interaction, and extends the understanding of Embodied intuitive interaction to
interactions with physical products, virtual interfaces, and TEIs. These contributions
and outcomes are significant, not only in the context of children’s interaction, but
also (more generally) in the context of the study of design for Embodied intuitive
interaction.
Section 9.1 presents an overview of the study’s contributions to knowledge,
while 9.2 outlines the research outcomes. Limitations of the research are presented in
Section 9.3, and an overview of potential future research directions is presented in
Section 9.4. The chapter closes with the study’s conclusions in Section 9.5.
9.1 CONTRIBUTIONS TO KNOWLEDGE
This research has provided a number of significant contributions to knowledge.
It has generated new knowledge of, and insights into children’s Embodied intuitive
interaction. This new knowledge, in turn, contributes to the broader context of
children’s interaction with physical products, virtual interfaces, and TEIs.
Specifically, three original contributions to knowledge are outlined in this section: (i)
Knowledge of children’s Embodied intuitive interactions with tactile interactions as
an interaction modality; (ii) Understanding of the role of Embodiment in facilitating
174 Embodied intuitive interaction in children
children’s intuitive interaction; and (iii) A methodology for investigating children’s
Embodied intuitive interaction.
(i) Knowledge of children’s Embodied intuitive interactions with
tactile interactions as an interaction modality
This research study strengthens the understanding of Embodiment and
children’s intuitive interaction from the perspective of tactile interactions as an
interaction modality. Most previous research focussed on full body interactions
and Embodied metaphors as facilitators of intuitive interaction. Thus, this
research study provides new insights into the role of tactile interactions in
Embodied intuitive interaction with physical products, virtual interfaces, and
TEIs. This is a significant step towards incorporating Embodiment in the
design of children’s products on the physical-virtual continuum, as children are
familiar with tactile interactions from birth. Investigating Embodiment through
interaction modalities also opens up possibilities for the exploration of other
sensory-based interaction modalities (such as sound interactions).
(ii) An understanding of the role of Embodiment in facilitating
children’s intuitive interaction
The results from this study have contributed to an Enhanced Framework of
Intuitive Interaction (EFII) (Blackler et al., 2018), an adaptation of which a that
relates to children’s Embodied intuitive interaction was discussed in Section
8.1.2. This EFII could lead to new directions for research into intuitive
interaction, such as intuitive experiences, factors responsible for these
experiences and the characteristics of features (e.g. transfer distance,
indirection, ubiquity). The results from the study are transferrable to the EFII;
this transferability suggests that they could also be applicable to adults. This is
an important contribution because Embodied intuitive interaction is
conventionally associated with children as sensorimotor knowledge is
considered to be more accessible to children than to adults (Brandenburg &
Sachse, 2012). The applicability of the EFII (Figure 27) to adults could lead to
future research that investigates Embodied intuitive interaction in adults.
The EFII allows discussion of intuitive interaction with products on the
physical-virtual continuum: physical products, virtual interfaces, and TEIs. The
Embodied intuitive interaction in children 175
findings from the study concur with Blackler's (2008) findings that physical
affordances should be incorporated in design whenever possible. Where
physical affordances are not possible, perceived affordances should be used.
This research study suggests further implications for design, and provides
design recommendations for children’s Embodied intuitive interactions with
physical products, virtual interfaces, and TEIs (Section 7.3.1). Physical and
perceived affordances could be used together, be sequential in time, or/and
nested in space.
The research study has further highlighted the role of cooperative activity and
scaffolding as Embodied aspects in the design of children’s products.
Appropriate scaffolds can assist children in carrying out perception and action
processes. Emergence is a design aspect representative of dynamic processes in
systems where interactions, behaviours, and environments evolve over time.
Dynamic processes are conducive to Embodied intuitive interactions, as they
facilitate the updating of existing knowledge, and the generation of new
knowledge; in other words, the learning of new concepts. This contribution is
significant because efforts have not focussed in the past on designing dynamic
emergent systems for Embodied interactions. The study provides direction
future research and development in the form of possibilities of using
affordances to design emergent systems.
(iii) Methodology for investigating children’s Embodied intuitive
interaction
A robust research methodology suitable for the study of children’s Embodied
intuitive interaction was developed. The combination of methods used,
consisting of observations, co-discovery, and retrospective interviews proved
to be highly successful in capturing the complexity of children’s decision
making and intuitive and Embodied interactions. The significance of the
methodology was the participatory nature of observations, where children were
observed in natural conditions, playing with real toys, and with others whom
they knew before the experiment. The unobtrusive nature of the children’s
play, in which there was no intervention of any kind from the researcher,
ensured that there was minimal interference to the children’s cognitive
processes. This ensured that the aspects of Embodiment used by children in
176 Embodied intuitive interaction in children
intuitive interaction with the toys in the experiments were analogous to the
aspects used in everyday interactions. Coding schemes developed for
Embodiment and intuitive interaction enabled the recording of children’s
intuitive behaviours, and the identification of aspects of Embodiment in these
behaviours.
The coding of the raw data facilitated the use of methods that allowed analysis
of aspects of Embodiment that contribute to children’s intuitive interaction, and
the relationship between these aspects. These analyses enabled an
understanding of the use of aspects of Embodiment in children’s intuitive
interaction. Due to their effectiveness in the context of children’s play, it is
expected that the methods used in this research can be transferred to the study
of children’s Embodied intuitive interaction in other contexts, such as teaching
and learning (Gillies, 2016) and affective technology for children’s social
connectedness (Pallarino, Free, Mutuc, & Yarosh, 2016).
9.2 RESEARCH OUTCOMES
The primary outcome of this research study is MEII – an interaction model for
Embodied intuitive interaction for the design of children’s products (Figure 29 and
Figure 30). The findings suggest that children are distributed anticipatory systems,
relying on their sensorimotor knowledge, and using their sensory perceptual
structures to perceive and act on their physical and social worlds. MEII, therefore,
consists of children interacting with these physical and social worlds: the physical
world consisting of artefacts, and the social world consisting of other children,
adults, and the environment. Children use physical affordances, perceived
affordances, and scaffolding in cooperation with other children to perceive the clues
in the social world and the artefacts, to decide on the actions to be performed. Once
the decision is made, children perform actions on the artefacts and the social world.
Children use scaffolding in cooperation with other children to the perform
actions and, in the process, change the properties of the artefacts and the social
world. The artefacts, social world, and children’s interaction evolve over repeated
use of the product, and this evolution results in emergence. This process is
responsible for the dynamic nature of MEII which, in turn, is important for children’s
development of new knowledge, or the updating of their existing knowledge and
Embodied intuitive interaction in children 177
learning. This is a significant contribution as it offers insights into how children
interact with the physical and social world. MEII could be applied in various
contexts of children’s interactions in design, research, and development.
9.3 RESEARCH LIMITATIONS
The first limitation of the study is that children could be more familiar to
physical Jenga than the virtual Jenga. More familiar interface features are used more
intuitively (Blackler et al., 2010). Thus, children’s familiarity of physical Jenga could
have affected intuitive interaction in children. However, children’s familiarity to
physical Jenga represents experience and familiarity of children to physical objects,
physical interactions and material properties. Children could have played with
physical Jenga (thus making them more familiar with it) due to the intuitiveness of
the toy. The objective of Experiment 1 was to study what makes certain interfaces
more intuitive for children’s interaction and thus more familiar to them. A between
subjects comparative study between two groups of children, one playing with
physical Jenga and second group playing with virtual Jenga, was conducted. The
pairs were randomly allocated to each of the groups. This ensured that the familiarity
differences between the children participating in the two groups would balance each
other out (Charness, Gneezy, & Kuhn, 2012).
The second limitation of this research is that children were observed playing
with only one type of TEI, (towards the left of the physical virtual continuum). TEIs
differ depending on how the physical and virtual are configured and coupled. Due to
innovation in technology, new configurations of TEIs evolve such as overlapping
physical and virtual spaces (Ullmer & Ishii, 2000). Embodied intuitive interaction
could vary in different TEIs depending on how physical and virtual spaces are
configured. Investigating other TEI configurations could provide additional
guidelines for designing Embodied intuitive TEIs and could be the focus for future
research.
The third limitation of this research study is that the coding of the qualitative data
could have been influenced by researcher bias. To reduce this effect, however,
coding heuristics were developed for both Embodiment and intuitive interaction,
examples of which are given in Appendix D and Appendix E. Heuristics were
developed through a review of the literature on Embodiment and intuitive
178 Embodied intuitive interaction in children
interaction, and then finalised by applying the heuristics specific to Experiment 1 and
Experiment 2 to the raw data. Secondly, the same researcher coded all the data twice,
and performed reliability analysis on the two versions of the coding (as described in
Section 5.4.2).
The fourth limitation of this study was that the children might have been
conscious of being observed and video recorded. This could have compromised their
natural behaviour. This possibility was minimised, however, by observing children in
the context of play with another child. The children were paired for the experiment—
each child with another whom they already knew. The researcher played as the
children’s opposing ‘team’ in Experiment 1. The motivation to win the game, the
context of play, and the presence of another child whom they already knew, made the
children feel comfortable and at ease.
The fifth limitation of the study was presence of parents and teachers
influencing the behaviour of children during the game play. Observational studies in
intuitive interaction require design of studies that evoke natural behaviour in the
participants as much as possible. Children behave differently when around adults
(Gardner, 2000) and this could have affected the outcomes of the studies
(Experiment 1 and Experiment 2). Experiment 1 was thus conducted at a local state
school, in a classroom without a teacher. In Experiment 2 where the study was
conducted at People and Systems lab (PAS lab) at QUT, Parents were asked not to be
present during the study. However, some parents insisted that they wanted to be
present during the study. Some parents were inquisitive about the study itself,
specifically the toys used in the study. Some wanted to know what were the children
asked to do, how were their children performing in the study and some just wanted to
be around as they had young kids to look after. These parents were allowed to stay
back, but behind a two-way mirror so that children could not see their parents and
parents could not intervene or interfere with the study, but parents could see the
proceedings of the study.
Finally, some children, especially young children, got into an argument or fight
while playing. This could affect their game play and their interaction with the toys.
Arguments and fights to an extent where none of the children were emotionally or
physically harmed were allowed (part of cooperative activity). However, attempts
were made to mitigate the situation and when it went beyond control, the experiment
Embodied intuitive interaction in children 179
was stopped and discontinued. Data from such experiments were not analysed and
destroyed.
9.4 FUTURE RESEARCH
(i) Investigation into each of the aspects of Embodiment
This research study has identified the aspects of Embodiment—physical
affordances, perceived affordances, emergence, scaffolding, and cooperative
activity—from the literature on Embodied cognition and decision making.
Further research on individual aspects of Embodiment is now required.
Scaffolding, emergence, and cooperative activity were found to be strongly
correlated to physical affordances. Considering the pivotal role of physical
affordances in children’s Embodied intuitive interaction, it is imperative that
future research investigates ways to incorporate scaffolding, emergence, and
cooperative activity in products for children, using physical affordances.
(ii) Generalisation of the Model for Embodied intuitive interaction (MEII)
The model for Embodied intuitive interaction (MEII) offers an understanding
of children’s Embodied intuitive interaction with artefacts and the social world.
The social world in this research study was limited to two children cooperating
to play a game, and the environment in which the game was played. Researcher
participation in the cooperative play was included in Experiment 1. However,
the researcher’s role in the Embodied intuitive interaction was not considered
in the research analysis. Future research could focus on including a larger
number of children, children from other cultures, and adults (such as teachers
and parents) in the social world. This would provide more insight into the role
of social world in intuitive interaction and result in a more generalised model
for children’s Embodied intuitive interaction.
(iii) Broaden the scope of the research
The methods used in this research can also be applied to investigate children’s
Embodied intuitive interaction in other contexts, such as spaces and services.
Children’s use of space and the objects within them is a topic of interest for
interaction designers in gaming, and in the field of learning sciences. The
methods could also be applied to services for children in healthcare,
180 Embodied intuitive interaction in children
entertainment, and learning. Research on intuitive interaction has mostly
focussed on interactions with interfaces. As interfaces become increasing
incorporated with digital elements, for example TEIs and virtual interfaces,
they generate large amounts of digital data which offer limited possibilities of
interaction with them. Future research could investigate ways to interact with
the digital data intuitively with Embodied interactions. The methods used in
this study could be used to investigate ways to materialise the digital data.
9.5 CONCLUSIONS
This research study investigated the role of Embodiment in children’s intuitive
interaction. The novel approach used in the study elicited natural and reliable
children’s behaviour for Embodied intuitive interactions. The study has taken a
human-centred approach to understanding children’s Embodied intuitive interactions
as the focus of study. Addressing Embodiment from a design perspective, and
determining the design aspects of Embodiment, are novel initiatives towards
identifying the design elements that can facilitate children’s intuitive interaction.
The knowledge resulting from the research also provides a significant
contribution to the domain of child-product interaction. Of significance was the
investigation of the design aspects of Embodiment for intuitive interaction in three
types of children’s products: physical, virtual, and TEIs.
The extent to which the aspects of Embodiment explain the variability in
intuitive interaction was determined, and this allowed inferences to be made about
the contribution of each of the aspects of Embodiment to intuitive interaction. This
includes the notable role of physical affordances in Embodied intuitive interaction,
and the importance of both physical and perceived affordances in Embodied intuitive
interaction with TEIs.
The research has provided empirical evidence to support previous claims in the
literature that physical products are more intuitive than virtual interfaces. The study
further found that TEIs with both physical and virtual elements could be intuitive,
depending on how these elements are configured in the system. This is an important
finding because it allows for the design of children’s products on the physical-virtual
continuum, with variations of physical and virtual compositions. To date, this has
been considered a challenging feat.
Embodied intuitive interaction in children 181
Research findings have resulted in the development of MEII for children. MEII
provides a representation of the use of aspects of Embodiment in perception and
action processes that form the basis of any anticipatory system, such as children.
Designers could use MEII as a tool to evaluate and design Embodied intuitive
products for children. Since MEII is based on children’s everyday behaviour and
interactions in the context of play, it could be transferred to other contexts involving
children. Based on its results and findings, the study has offered design
recommendations that are applicable to three main types of intuitive products for
children: physical, virtual, and TEIs, the most prominent types of products currently
used by children.
The new knowledge generated by this research has implications for the design
of a wide range of children’s products, such as physical products, virtual interfaces,
and TEIs. In particular, recommendations from this research are directed at
informing the design of interfaces for children that incorporate Embodiment to
facilitate intuitive interactions. Rather than focusing on possible technological
innovations, the study has taken a human-centred perspective. Its recommendations
thus aim to improve children’s experience in interacting with products and interfaces.
It is expected that the design guidelines will support the development of relevant
sensory perceptual structures, and the use of relevant knowledge and experiences that
is required for Embodied intuitive interaction with interfaces.
Pioneering the study of children’s Embodied intuitive interaction using play as
a context of study, this research contributes a novel perspective on children’s
interactions with the world of artefacts. It is therefore expected that the new
knowledge developed will stimulate discussions in diverse application domains
involving children.
Finally, this study has consolidated various aspects of Embodiment and children’s
intuitive interaction. It has thus made a significant contribution to the study of
Embodiment and children’s intuitive interaction, and advanced the understanding of
how children interact with artefacts.
Bibliography 183
Bibliography
Adebesin, T. F., De Villiers, M. R., & Ssemugabi, S. (2009). Usability testing of e-learning: an approach incorporating co-discovery and think-aloud. In Proceedings of the 2009 Annual Conference of the Southern African Computer Lecturers’ Association (pp. 6–15).
Ahmet, Z., Jonsson, M., Sumon, S. I., & Holmquist, L. E. (2011). Supporting embodied exploration of physical concepts in mixed digital and physical interactive settings. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (pp. 109–116).
Alexander, G. M. (2003). An evolutionary perspective of sex-typed toy preferences: Pink, blue, and the brain. Archives of Sexual Behavior, 32(1), 7–14.
Allen, P. M., & Strathern, M. (2003). Evolution, emergence, and learning in complex systems. Emergence, 5(4), 8–33.
Als, B. S., Jensen, J. J., & Skov, M. B. (2005). Comparison of think-aloud and constructive interaction in usability testing with children. Interaction Design and Children, 9–16. http://doi.org/10.1145/1109540.1109542
Alter, A. L., Oppenheimer, D. M., Epley, N., & Eyre, R. N. (2007). Overcoming intuition: metacognitive difficulty activates analytic reasoning. Journal of Experimental Psychology: General, 136(4), 569.
Amazon, & Foxconn. (2007). Kindle.
Andersen, P. B. (2001). What semiotics can and cannot do for HCI. Knowledge-Based Systems, 14(8), 419–424.
Anderson, M. (2003). Embodied Cognition: A field guide. Artificial Intelligence, 149(1), 91–130. http://doi.org/http://dx.doi.org/10.1016/S0004-3702(03)00054-7
Anderson, M. (2005). How to study the mind: An introduction to embodied cognition. Embodied Cognition and Perceptual Learning in …. Retrieved from http://cogprints.org/3945/
Anderson, S. (2011). Seductive interaction design: creating playful, fun, and effective user experiences. Pearson Education.
Anthony, L., Stofer, K. A., Luc, A., & Wobbrock, J. O. (2016). Gestures by Children and Adults on Touch Tables and Touch Walls in a Public Science Center. In Proceedings of the The 15th International Conference on Interaction Design and Children (pp. 344–355).
Antle, A. (2007). Designing tangibles for children: what designers need to know. In CHI’07 Extended Abstracts on Human Factors in Computing Systems (pp. 2243–2248).
184 Bibliography
Antle, A. (2011). Exploring how children use their hands to think: an embodied interactional analysis. Behaviour & Information Technology, 1–17. http://doi.org/10.1080/0144929X.2011.630415
Antle, A., Corness, G., & Droumeva, M. (2009a). Human-computer-intuition? Exploring the cognitive basis for intuition in embodied interaction. International Journal of Arts and Technology, 2(3), 235. http://doi.org/10.1504/IJART.2009.028927
Antle, A., Corness, G., & Droumeva, M. (2009b). What the body knows: Exploring the benefits of embodied metaphors in hybrid physical digital environments. Interacting with Computers, 21(1–2), 66–75. http://doi.org/10.1016/j.intcom.2008.10.005
Antle, A., Droumeva, M., & Corness, G. (2008). Playing with the sound maker: do embodied metaphors help children learn? … on Interaction Design and Children, 178–185. Retrieved from http://dl.acm.org/citation.cfm?id=1463754
Antle, A., & Wise, A. (2013). Getting down to details: Using theories of cognition and learning to inform tangible user interface design. Interacting with Computers, 25(1), 1–40. Retrieved from http://iwc.oxfordjournals.org/content/25/1/1.short
Apple. (2001). iTunes.
Apple. (2015). iPad.
Arthur, C. (2010, June). Why Minority Report was spot on. The Guardian. Retrieved from https://www.theguardian.com/technology/2010/jun/16/minority-report-technology-comes-true
Aslan, I., Primessnig, F., Murer, M., Moser, C., & Tscheligi, M. (2013). Inspirations from honey bees: exploring movement measures for dynamic whole body gestures. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (pp. 421–424).
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. http://doi.org/10.1109/38.963459
Baars, B. J., Ramsøy, T. Z., & Laureys, S. (2003). Brain, conscious experience and the observing self. Trends in Neurosciences, 26(12), 671–675.
Baillargeon, R., & Graber, M. (1988). Evidence of location memory in 8-month-old infants in a nonsearch AB task. Developmental Psychology, 24(4), 502.
Bakker, S., Van Den Hoven, E., & Antle, A. N. (2011). MoSo tangibles: evaluating embodied learning. In Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction (pp. 85–92).
Barrett, L. (2011). Beyond the brain: How body and environment shape animal and human minds. Princeton University Press.
Bibliography 185
Barsalou, L. W. (2010). Grounded Cognition: Past, Present, and Future. Topics in Cognitive Science, 2(4), 716–724. http://doi.org/10.1111/j.1756-8765.2010.01115.x
Barsalou, L. W., Niedenthal, P. M., Barbey, A. K., & Ruppert, J. A. (2003). Social embodiment. Psychology of Learning and Motivation, 43, 43–92.
Barsalou, L. W., Santos, A., Simmons, W. K., & Wilson, C. D. (2008). Language and simulation in conceptual processing. In Symbols, embodiment, and meaning (pp. 245–283).
Bastick, T. (1982). Intuition: How we think and act.
Bastick, T. (2003). Intuition: Evaluating the construct and its impact on creative thinking. Stoneman & Lang.
Beaudouin-Lafon, M. (2000). Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 446–453).
Bingham, G. P. (1995). Dynamics and the problem of visual event recognition. Mind as Motion: Explorations in the Dynamics of Cognition, 403–448.
Björk-Willén, P., & Cromdal, J. (2009). When education seeps into “free play”: How preschool children accomplish multilingual education. Journal of Pragmatics, 41(8), 1493–1518.
Blackler, A. (2008). Intuitive interaction with complex artefacts : empirically-based research. VDM Verlag, Saarbrücken, Germany.
Blackler, A., Desai, S., McEwan, M., Dieffenbach, S., & Popovic, V. (2018). Perspectives on the nature of intuitive interaction. In Blackler (Ed.), Intuitive Interaction: Research and Application. CRC Press.
Blackler, A., & Hurtienne, J. (2007). Towards a unified view of intuitive interaction : definitions, models and tools across the world. MMI-Interaktiv, 13(2007), 36–54. Retrieved from http://eprints.qut.edu.au/19116/
Blackler, A., & Popovic, V. (2015). Towards intuitive interaction theory. Interacting with Computers, 27(3), 203–209.
Blackler, A., & Popovic, V. (2016). Intuitive Interaction research – new directions and possible responses. In 2016 Design Research Society 50th Anniversary Conference (pp. 1–11). Brighton.
Blackler, A., Popovic, V., & Mahar, D. (2010). Investigating users’ intuitive interaction with complex artefacts. Applied Ergonomics, 41(1), 72–92. http://doi.org/10.1016/j.apergo.2009.04.010
Blikstein, P. (2013). Digital Fabrication and ’Making’ in Education: The Democratization of Invention. In J. W.-H. Büching (Ed.), FabLabs: Of Machines, Makers and Inventors. Bielefeld: Transcript Publishers.
186 Bibliography
Brandenburg, S., & Sachse, K. (2012). Intuition comes with experience. In Human factors: A view from an integrative perspective. Proceedings of the Human Factors and Ergonomics Society of Europe Conference (pp. 213–223).
Breathnach, H., O’Gorman, L. M., & Danby, S. (2016). “Well it depends on what you’d call play”: Parent perspectives on play in Queensland’s Preparatory Year. Australasian Journal of Early Childhood, 41(2), 77–84.
Brederode, B., Markopoulos, P., Gielen, M., Vermeeren, A., & de Ridder, H. (2005). pOwerball: the design of a novel mixed-reality game for children with mixed abilities. Interaction Design and Children, 32–39. http://doi.org/10.1145/1109540.1109545
Brooks, R. A. (1990). Elephants don’t play chess. Robotics and Autonomous Systems, 6(1), 3–15.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.
Bruner, J. S., Jolly, A., & Sylva, K. (1976). Play: Its role in development and evolution.
Buur, J., & Andreasen, M. M. (1989). Design models in mechatronic product development. Design Studies, 10(3), 155–162.
Caldera, Y. M., Huston, A. C., & O’Brien, M. (1989). Social interactions and play patterns of parents and toddlers with feminine, masculine, and neutral toys. Child Development, 70–76.
Cave, A., Blackler, A., Popovic, V., & Kraal, B. (2014). Examining Intuitive Navigation in Airports. In Design Research So- ciety Conference 2014. Umea, Sweden.
Chalmers, D. J. (1995). Facing up to the problem of consciousness. Journal of Consciousness Studies, 2(3), 200–219.
Chalmers, M. (2001). Book Review: Where the Action Is: The Foundations of Embodied Interaction, Paul Dourish. MIT Press, Cambridge.
Charness, G., Gneezy, U., & Kuhn, M. A. (2012). Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization, 81(1), 1–8.
Chiel, H. J., & Beer, R. D. (1997). The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. Trends in Neurosciences, 20(12), 553–557.
Chin, J., Diehl, V., & Norman, K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 213–218).
Choi, K.-S. (1993). Developmental changes in intuitive thinking of Korean children. Early Child Development and Care, 85(1), 89–95.
Bibliography 187
Churchman, C. W. (1971). The Design of Inquiring Systems Basic Concepts of Systems and Organization.
Cienki, A., & Müller, C. (2008). Metaphor and gesture (Vol. 3). John Benjamins Publishing.
Clark, A. (1997). Being there: Putting brain, body, and world together again. Being There: Putting Brain, Body, and World Together Again. Cambridge, MA: MIT Press.
Clark, A. (2001). Reasons, Robots and the Extended Mind. Mind & Language, 16(2), 121–145. http://doi.org/10.1111/1468-0017.00162
Clark, A. (2005). Beyond the flesh: Some lessons from a mole cricket. Artificial Life, 11(1–2), 233–244.
Clark, A. (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension. Oxford University Press. Retrieved from http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195333213.001.0001/acprof-9780195333213-chapter-1.
Clark, A. (2012). The Cambridge Handbook of Cognitive Science. In K. Ramsey & F. and William (Eds.), The Cambridge Handbook of Cognitive Science (pp. 275–291). Cambridge University Press. http://doi.org/http://dx.doi.org/10.1017/CBO9781139033916.018
Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. The Behavioral and Brain Sciences, 36(3), 181–204. http://doi.org/10.1017/S0140525X12000477
Clement, J. (1994). Use of physical intuition and imagistic simulation in expert problem solving.
Cohen, J. (1992). A power primer. Psychological bulletin (Vol. 112). American Psychological Association.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression/correlation analysis for the behavioral sciences. Routledge.
Cole, M., & Griffin, P. (1980). Cultural amplifiers reconsidered. The Social Foundations of Language and Thought, Essays in Honor of Jerome S. Bruner, 343–364.
Corr, P. J. (2008). The reinforcement sensitivity theory of personality. Cambridge University Press.
Cowart, M. (2004). Embodied cognition. The Internet Encyclopedia of Philosophy.
Crowle, S., Boniface, M., Poussard, B., & Asteriadis, S. (2014). A design and evaluation framework for a tele-immersive mixed reality platform. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8853, 151–158. http://doi.org/10.1007/978-3-319-13969-2_12
188 Bibliography
Cruise, T., Wagner, P., & Brian, D. P. (1996). Mission Impossible. United States: Paramount Pictures.
Cutting, J., & Vishton, P. (1995). Perceiving layout and knowing distances: the integration relative potency and contextual use of different information about depth epstein w. rogers s. perception of space and motion 1995 69--117. Academic Press, San Diego, CA.
Dahl, D. (2017). Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything. Springer.
Dane, E., & Pratt, M. G. (2007). Exploring intuition and its role in managerial decision making. Academy of Management Review, 32(1), 33–54. article.
Dautenhahn, K. (1996). Embodied cognition in animals and artifacts. Proc. AAAI FS Embodied Cognition and Action, 27–32.
Dennett, D. (1991). Consciousness explained. New York: Little Brown & Co.
Desai, S., Blackler, A., & Popovic, V. (2015). Intuitive use of tangibles. In IASDR.
Desai, S., Blackler, A., & Popovic, V. (2016). Intuitive interaction in a mixed reality system (pp. 1–16). DRS.
Dick, P. K. (2002). The Minority Report: And Other Classic Stories (Vol. 4). Citadel Press.
Diefenbach, S., & Ullrich, D. (2015). An Experience Perspective on Intuitive Interaction : Central Components and the Special Effect of Domain Transfer Distance. Interacting with Computers, 27(3), 210–234. http://doi.org/10.1093/iwc/iwv001
Disessa, A. A. (1988). Knowledge in pieces.
Dix, A. (2011). Physical creatures in a digital world. In Proceedings of the 29th Annual European Conference on Cognitive Ergonomics (pp. 11–14).
Djajadiningrat, T., Wensveen, S., Frens, J., & Overbeeke, K. (2004). Tangible products: redressing the balance between appearance and action. Personal and Ubiquitous Computing, 8(5), 294–309.
Dotov, D. G., Nie, L., & De Wit, M. M. (2012). Understanding affordances: history and contemporary. Choices, 33, 269–298.
Dourish, P. (1927). Seeking a foundation for context-aware computing. Human--Computer Interaction, 16(2–4), 229–241.
Dourish, P. (2001). Where the action is: the foundations of embodied interaction. The MIT Press.
Draper, N., & Smith, H. (2014). Applied regression analysis (Third). John Wiley & Sons, Ltd.
Dreyfus, H., & Dreyfus, S. E. (2000). Mind over machine. Free Press.
Druin, A. (2002). The role of children in the design of new technology. Behaviour and Information Technology, 21(1), 1–25.
Bibliography 189
Dunn, B. D., Galton, H. C., Morgan, R., Evans, D., Oliver, C., Meyer, M., … Dalgleish, T. (2010). Listening to your heart how interoception shapes emotion experience and intuitive decision making. Psychological Science. article.
Eelen, J., Dewitte, S., & Warlop, L. (2013). Situated embodied cognition: Monitoring orientation cues affects product evaluation and choice. Journal of Consumer Psychology.
Ekman, P. (2006). Darwin and facial expression: A century of research in review. Ishk.
Ekman, P., Friesen, W. V, & Ellsworth, P. (2013). Emotion in the human face: Guidelines for research and an integration of findings. Elsevier.
Engeström, Y., Miettinen, R., & Punamäki, R.-L. (1999). Perspectives on activity theory. Cambridge University Press.
Entwistle, J. (2015). The Fashioned Body: Fashion, Dress and Social Theory. John Wiley & Sons.
Enyedy, N., Danish, J., Deliema, D., Saleh, A., Lee, C., Morris, N., & Illum, R. (2017). Social Affordances of Mixed Reality Learning Environments: A case from the Science through Technology Enhanced Play project (STEP). In Proceedings of the 50th Hawaii International Conference on System Sciences.
Eriksen, T. H. (2001). Small Places, Large Issues-: An Introduction to Social and Cultural Anthropolog. Pluto Press.
Erzberger, C., & Kelle, U. (2003). Making inferences in mixed methods: the rules of integration. Handbook of Mixed Methods in Social and Behavioral Research, 457–490.
Fan, M., Antle, A. N., & Cramer, E. S. (2016). Exploring the Design Space of Tangible Systems Supported for Early Reading Acquisition in Children with Dyslexia. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 689–692).
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.
Feinstein, N. W., & Meshoulam, D. (2013). Science for what public? Addressing equity in American science museums and science centers. Journal of Research in Science Teaching.
Felemban, S., Gardner, M., Callaghan, V., & Pena-Rios, A. (2017). Towards Observing and Assessing Collaborative Learning Activities in Immersive Environments. In International Conference on Immersive Learning (pp. 47–59).
Fernaeus, Y., & Jacobsson, M. (2009). Comics, robots, fashion and programming: outlining the concept of actDresses. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (pp. 3–8). New York, NY, USA: ACM. http://doi.org/10.1145/1517664.1517669
190 Bibliography
Field, A. (2008). Discovering statistics using SPSS, and sex, drugs and rock “n”roll. SAGE Publications Ltd, ISBN.
Fincham, B. (2016). Theorising Fun. In The Sociology of Fun (pp. 27–46). London: Springer. http://doi.org/10.1057/978-1-137-31579-3_2
Fink, J., Lemaignan, S., Dillenbourg, P., Rétornaz, P., Vaussard, F., Berthoud, A., … Franinović, K. (2014). Which robot behavior can motivate children to tidy up their toys?: Design and evaluation of ranger. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction (pp. 439–446).
Finucane, M. L., Alhakami, A., Slovic, P., & Johnson, S. M. (2000). The affect heuristic in judgments of risks and benefits. Journal of Behavioral Decision Making, 13(1), 1. article.
Fischbein, E. (1987). Intuition in science and mathematics: An educational approach (Vol. 5). Springer.
Fischbein, E. (1999). INTUITIONS AND SCHEMATA IN MATHEMATICAL REASONING, (1968), 11–50.
Fischer, J., Jiang, W., Kerne, A., Greenhalgh, C., Ramchurn, S., Reece, S., … Rodden, T. (2014). Supporting Team Coordination on the Ground : Requirements from a Mixed Reality Game. COOP 2014 - Proceedings of the 11th International Conference on the Design of Cooperative Systems, 49–67. http://doi.org/10.1007/978-3-319-06498-7_4
Fischer, S., Itoh, M., & Inagaki, T. (2015). Screening Prototype Features in Terms of Intuitive Use: Design Considerations and Proof of Concept. Interacting with Computers, 27(3), 256–270. http://doi.org/10.1093/iwc/iwv002
Fisk, A. D., Rogers, W. A., Charness, N., Czaja, S. J., & Sharit, J. (2009). Designing for older adults: Principles and creative human factors approaches. CRC press.
Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.
Fitzmaurice, G. W., Ishii, H., & Buxton, W. A. S. (1995). Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 442–449).
Forgas, J. P. (1995). Mood and judgment: the affect infusion model (AIM). Psychological Bulletin, 117(1), 39. article.
Forrester, J. W. (1995). The beginning of system dynamics. McKinsey Quarterly, (4), 4–16. http://doi.org/10.3401/poms.1080.0022
Franinović, K., & Serafin, S. (2013). Sonic interaction design. Mit Press.
Frederking, J., Cruz, M., Overbeeke, K., & Baskinger, M. (2007). Collaborative Play Through Digital and Physical Interaction. Tangible Play: Research and Design and Tangible and Tabletop Games at IUI, 7. Retrieved from http://elisevandenhoven.com/publications/hoven-iui07wp.pdf#page=33
Bibliography 191
Gaines, B. R. (1989). Social and cognitive processes in knowledge acquisition. Knowledge Acquisition, 1(1), 39–58. http://doi.org/10.1016/S1042-8143(89)80004-4
Gaines, B. R. (2013). Knowledge acquisition: Past, present and future. International Journal of Human-Computer Studies, 71(2), 135–156. http://doi.org/10.1016/j.ijhcs.2012.10.010
Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22(3), 455–479.
Gardner, F. (2000). Methodological issues in the direct observation of parent--child interaction: Do observational findings reflect the natural behavior of participants? Clinical Child and Family Psychology Review, 3(3), 185–198.
Gardner, M., & Elliott, J. B. (2014). The Immersive Education Laboratory : understanding affordances , structuring experiences , and creating constructivist, collaborative processes , in mixed-reality smart environments. Transactions on Future Intelligent Educational Environments, 1(1), 1–13.
Gaver, W. W. (1991). Technology affordances. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 79–84).
Gervais, R., Frey, J., Gay, A., Lotte, F., & Hachet, M. (2016). Tobe: Tangible out-of-body experience. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 227–235).
Gibbs, R. W. (2006). Embodiment and cognitive science. Cambridge University Press.
Gibson. (1979). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum.
Gibson. (2014). The ecological approach to visual perception: classic edition. Psychology Press. (Originally published in 1979)
Gibson, E. (1988). Exploratory Behavior In The Development Of Perceiving, Acting, And The Acquiring Of Knowledge. Annual Review of Psychology, 39(1), 1–41. http://doi.org/10.1146/annurev.psych.39.1.1
Gillies, R. M. (2016). Cooperative learning: Review of research and practice. Australian Journal of Teacher Education, 41(3), 3.
Giummarra, M. J., Gibson, S. J., Georgiou-Karistianis, N., & Bradshaw, J. L. (2008). Mechanisms underlying embodiment, disembodiment and loss of embodiment. Neuroscience & Biobehavioral Reviews, 32(1), 143–160.
Goldman, A., & de Vignemont, F. (2009). Is social cognition embodied? Trends in Cognitive Sciences, 13(4), 154–159.
Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10), 1489–1522.
192 Bibliography
Gopnik, A., Meltzoff, A. N., & Kuhl, P. K. (2009). The scientist in the crib: Minds, brains, and how children learn. HarperCollins.
Gore, J., & Sadler-Smith, E. (2011). Unpacking intuition: A process and outcome framework. Review of General Psychology, 15(4), 304.
Gorriz, C. M., & Medina, C. (2000). Engaging Girls with Computers Through Software Games. Commun. ACM, 43(1), 42–49. http://doi.org/10.1145/323830.323843
Gray, C. D., & Kinnear, P. R. (2012). IBM SPSS statistics 19 made simple. Psychology Press.
Grenier, A. M. (2007). Crossing age and generational boundaries: Exploring intergenerational research encounters. Journal of Social Issues, 63(4), 713–727.
Grufberg, K., & Jonsson, M. (2012). Sciensations: making sense of science by designing with sensors. In Proceedings of the 11th International Conference on Interaction Design and Children (pp. 116–124).
Guha, M. L., Druin, A., & Fails, J. A. (2013). Cooperative Inquiry revisited: Reflections of the past and guidelines for the future of intergenerational co-design. International Journal of Child-Computer Interaction, 1(1), 14–23. http://doi.org/10.1016/j.ijcci.2012.08.003
Haans, A., & IJsselsteijn, W. (2006). Mediated social touch: A review of current research and future directions. Virtual Reality, 9(2–3), 149–159. http://doi.org/10.1007/s10055-005-0014-2
Hammond, K. R. (1993). Naturalistic decision making from a Brunswikian viewpoint: Its past, present, future. Decision Making in Action: Models and Methods, 205–227.
Harnad, S. (2003). Symbol-grounding Problem. Encyclopedia of Cognitive Science.
Harrison, C., & Hudson, S. E. (2009). Providing Dynamically Changeable Physical Buttons on a Visual Display. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 299–308. http://doi.org/10.1145/1518701.1518749
Hartson, H. R., Andre, T. S., & Williges, R. C. (2003). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 15(1), 145–181.
Hasbro, & Scott, L. (2001). Jenga Blocks. Parker Brothers, Hasbro.
Haugeland, J. (1989). Artificial intelligence: The very idea. The MIT Press.
Haugeland, J. (1998). Mind embodied and embedded. Having Thought: Essays in the Metaphysics of Mind, 207–237.
Hay, S., Newman, J., & Harle, R. (2008). Optical tracking using commodity hardware. In Mixed and Augmented Reality, 2008. ISMAR 2008. 7th IEEE/ACM International Symposium on (pp. 159–160).
Bibliography 193
Hayles, N. K. (2010). My mother was a computer: digital subjects and literary texts. University of Chicago Press.
Heath, S. (2002). Embedded systems design. Newnes.
Hertzum, M. (2016). Usability Testing : Too Early ? Too Much Talking ? Too Many Problems ? Journal of Usability Studies (JUS), 11(3), 83–88.
Hespanhol, L., & Tomitsch, M. (2015). Strategies for intuitive interaction in public urban spaces. Interacting with Computers, iwu051.
Heyns, B. (1987). Schooling and cognitive development: Is there a season for learning? Child Development, 1151–1160.
Hinton, A. (2014). Perception, Cognition, and Affordance. In Understanding context: environment, language, and information architecture. “ O’Reilly Media, Inc.”
Hoffmann, M., Schuster, K., Schilberg, D., & Jeschke, S. (2016). Next-generation teaching and learning using the virtual theatre. In Automation, Communication and Cybernetics in Science and Engineering 2015/2016 (pp. 281–291). Springer.
Holmquist, L. E., Ju, W., Jonsson, M., Tholander, J., Ahmet, Z., Sumon, S. I., … Winograd, T. (2010). Wii Science: Teaching the laws of nature with physically engaging video game technologies. CHI 2010 Workshop: Video Games as Research Instruments, Altlanta.
Holyoak, K. J. (1991). 12 Symbolic connectionism: toward third-generation theories of expertise. Toward a General Theory of Expertise: Prospects and Limits, 301.
Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 437–446). http://doi.org/http://doi.acm.org/10.1145/1124772.1124838
Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15(3), 495–514.
Höysniemi, J., Hämäläinen, P., & Turkki, L. (2003). Using peer tutoring in evaluating the usability of a physically interactive computer game with children. Interacting with Computers, 15(2 SPEC.), 203–225. http://doi.org/10.1016/S0953-5438(03)00008-0
Hulin, W. S., & Katz, D. (1935). The Frois-Wittmann pictures of facial expression. Journal of Experimental Psychology, 18(4), 482.
Hummels, C., Smets, G., & Overbeeke, K. (1998). An intuitive two-handed gestural interface for computer supported product design. … Sign Language in Human-Computer …. Retrieved from http://link.springer.com/chapter/10.1007/BFb0053000
Hurtienne, J. (2007). Metaphors as Tools for Intuitive Interaction with Technology. Metaphorik. de, 12(2), 21–52.
Hurtienne, J. (2009). Image schemas and design for intuitive use.
194 Bibliography
Hurtienne, J., & Israel, J. H. J. J. H. (2007). Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction. … on Tangible and Embedded Interaction, 15–17. http://doi.org/http://doi.acm.org/10.1145/1226969.1226996
Hurtienne, J., Klöckner, K., Diefenbach, S., Nass, C., & Maier, A. (2015). Designing with image schemas: resolving the tension between innovation, inclusion and intuitive use. Interacting with Computers, 27(3), 235–255.
Hurtienne, J., Löffler, D., Gadegast, P., & Hußlein, S. (2015). Comparing Pictorial and Tangible Notations of Force Image Schemas (pp. 249–256).
Hurtienne, & Blessing. (2007). Design for Intuitive Use-Testing image schema theory for user interface design. … Conference on Engineering Design, (August), 1–12. Retrieved from http://joernhurtienne.com/Publications_files/Paper_386_HurtienneBlessing.pdf
Husserl, E. (2013). Cartesian meditations: An introduction to phenomenology. Springer Science & Business Media.
Hutchins, E. (2000). Distributed cognition. Internacional Enciclopedia of the Social and Behavioral Sciences.
Ishii, H. (2007). Tangible user interfaces. In A. Sears & J. Jacko (Eds.), Human-Computer Interaction: Design Issues, Solutions, and Applications (pp. 141–157). CRC Press.
Ishii, H. (2008). Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (pp. xv--xxv).
Israel, J. H., Hurtienne, J., Pohlmeyer, A. E., Mohs, C., Kindsmüller, M. C., & Naumann, A. (2009). On intuitive use , physicality and tangible user interfaces Jörn Hurtienne Anja Naumann, 2(4), 348–366.
Israel, J. H., Hurtienne, J., Pohlmeyer, A. E., Mohs, C., Kindsmuller, M., & Naumann, A. (2009). On intuitive use, physicality and tangible user interfaces. International Journal of Arts and Technology, 2(4), 348–366. http://doi.org/10.1504/IJART.2009.02924
Iverson, J. M., & Thelen, E. (1999). Hand, mouth and brain. The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 6(11–12), 11–12.
Jacob, R. J. K., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008). Reality-based interaction: a framework for post-WIMP interfaces. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (pp. 201–210). ACM. http://doi.org/10.1145/1357054.1357089
Jacoby, S., Gutwillig, G., Jacoby, D., Josman, N., Weiss, P. L., Koike, M., … Sharlin, E. (2009). PlayCubes: monitoring constructional ability in children using a tangible user interface and a playful virtual environment. In Virtual Rehabilitation International Conference, 2009 (pp. 42–49).
Jenkins, R. (2014). Social identity. Routledge.
Bibliography 195
Johansson, C., Zeynep, A., Tholander, J., Jonsson, M., Islam Sumon, S., & Aleo, F. (2011). Weather Gods and Fruit Kids--embodying abstract concepts using tactile feedback and full body interaction. In CSCL 2011.
Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone. Cooperative, competitive, and individualistic learning. ERIC.
Johnson, E., & Russo, E. (1981). Product familiarity and learning new information. Advances in Consumer Research, 8(1).
Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason.
Johnson, M. (2010). Metaphor and Cognition. In D. Schmicking & S. Gallagher (Eds.), Handbook of Phenomenology and Cognitive Science (pp. 401–414). Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-90-481-2646-0_22
Jones, L., & Lederman, S. (2006). Human hand function. Oxford University Press.
Jones, N. G. B. (1971). Criteria for use in describing facial expressions of children. Human Biology, 365–413.
Jordà, S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction (pp. 139–146).
Kahler, H., Kensing, F., & Muller, M. (2000). Methods \& tools: constructive interaction and collaborative work: introducing a method for testing collaborative systems. Interactions, 7(3), 27–34.
Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree. American Psychologist, 64(6), 515.
Kail, R. (2012). Children and Their Development (6th ed.). Pearson.
Kemp, J. A. M., & Van Gelderen, T. (1996). Co-discovery exploration: an informal method for the iterative design of consumer products. Usability Evaluation in Industry, 139–146.
Kim, Y. S. (2015). A methodology of design for affordances using affordance feature repositories. AI EDAM, 29(Special Issue 03), 307–323. http://doi.org/10.1017/S0890060415000281
King, S., & Chang, K. (2016). Understanding Industrial Design: Principles for UX and Interaction Design. “ O’Reilly Media, Inc.”
Kirsh, D. (2013a). Embodied cognition and the magical future of interaction design. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 3.
Kirsh, D. (2013b). Thinking with external representations. In Cognition Beyond the Brain (pp. 171–194). Springer.
Kirsh, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 513–549.
196 Bibliography
Kiverstein, J. (2010). Sensorimotor knowledge and the contents of experience. Perception, Action, and Consciousness, 257–274.
Klein, G. A. (2003). Intuition at work: Why developing your gut instincts will make you better at what you do. Currency/Doubleday.
Koo, T., & Li, M. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163.
Kutner, M. H., Nachtsheim, C., & Neter, J. (2004). Applied linear regression models. McGraw-Hill/Irwin.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: U of Chicago P. Chicago, Illinois, USA: Univ. of Chicago Press.
Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Basic books.
Lawry, S., Popovic, V., & Blackler, A. L. (2011). Diversity in product familiarity across younger and older adults. In Diversity \& Unity, 4th World Conference on Design Research, IASDR2011.
Leakey, R. (2008). The origin of humankind. Basic books.
Liamputtong, P. (2006). Researching the vulnerable: A guide to sensitive research methods. Sage.
Lim, K. H., Ward, L. M., & Benbasat, I. (1997). An empirical study of computer system learning: Comparison of co-discovery and self-discovery methods. Information Systems Research, 8(3), 254–272.
Linderoth, J. (2013). Beyond the Digital Divide : An Ecological Approach to Game- Play. Transactions of the Digital Game Research Association, 1(1), 1–17.
Livingstone, S., Marsh, J., Plowman, L., Ottovordemgentschenfelde, S., & Fletcher-Watson, B. (2015). Young children(0-8) and digital technology.
Löwgren, J., & Stolterman, E. (2004). Thoughtful interaction design: A design perspective on information technology. Mit Press.
Macaranas, A. (2013). The Effects of Intuitive Interaction Mappings on the Usability of Body-based Interfaces. SIMON FRASER UNIVERSITY. Retrieved from http://summit.sfu.ca/item/12635
Macaranas, A., Antle, A., & Riecke, B. E. (2015). What is Intuitive Interaction? Balancing Users’ Performance and Satisfaction with Natural User Interfaces. Interacting with Computers, iwv003.
Maguire, M. (2001). Context of use within usability activities. International Journal of Human-Computer Studies, 55(4), 453–483.
Mahmoud, M., & Robinson, P. (2011). Interpreting hand-over-face gestures. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6975 LNCS(PART 2), 248–255. http://doi.org/10.1007/978-3-642-24571-8_27
Bibliography 197
Maier, J. R. A., & Fadel, G. M. (2009). Affordance based design: a relational theory for design. Research in Engineering Design, 20(1), 13–27.
Malafouris, L. (2013). How Things Shape the Mind. MIT Press.
Malinverni, L., Ackermann, E., & Pares, N. (2016). Experience as an object to think with: From sensing-in-action to making-sense of action in full-body interaction learning environments. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 332–339).
Marshall, P., Fleck, R., Harris, A., Rick, J., Hornecker, E., Rogers, Y., … Dalton, N. S. (2009). Fighting for control: children’s embodied interactions when using physical and digital representations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2149–2152).
Marshall, P., Price, S., & Rogers, Y. (2003). Conceptualising tangibles to support learning. Proceeding of the 2003 Conference on Interaction Design and Children - IDC ’03, (c), 101. http://doi.org/10.1145/953540.953551
Mattel, Sekkoïa, & Tavitian, B. (2000). Blokus.
McCall, R. B. (1974). Exploratory manipulation and play in the human infant. Monographs of the Society for Research in Child Development, 1–88.
McEwan, M., Blackler, A., Johnson, D., & Wyeth, P. (2014). Natural Mapping and Intuitive Interaction in Videogames. CHI PLAY ’14 Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, 191–200. http://doi.org/http://dx.doi.org/10.1145/2658537.2658541
McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and evolving a concept. In Graphics interface (Vol. 2000, pp. 179–186).
Meier, B. P., & Robinson, M. D. (2004). Why the sunny side is up associations between affect and vertical position. Psychological Science, 15(4), 243–247.
Mellar, H., & Bliss, J. (1994). Introduction: modelling and education.
Menary, R. (2010). The extended mind. Cambridge, MA: MIT Press.
Microsoft. (2010). XBOX Kinect.
Mihajlov, M., Law, E. L.-C., & Springett, M. (2015). Intuitive Learnability of Touch Gestures for Technology-Na{ï}ve Older Adults. Interacting with Computers, iwu044.
Mihajlov, M., Law, E. L.-C., & Springett, M. (2015). Intuitive Learnability of Touch Gestures for Technology-Naive Older Adults. Interacting with Computers, 27(3), 344–356. http://doi.org/10.1093/iwc/iwu044
Milgram, P., & Colquhoun, H. (1999). A taxonomy of real and virtual world display integration. Mixed Reality: Merging Real and Virtual Worlds, 5–30.
Minsky, M. (1988). Society of mind. Simon and Schuster.
Mistry, P., & Maes, P. (2009). SixthSense: a wearable gestural interface. In ACM SIGGRAPH ASIA 2009 Sketches (p. 11).
198 Bibliography
Moeller, J., & Kerne, A. (2012). ZeroTouch: an optical multi-touch and free-air interaction architecture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2165–2174).
Mohan, G., Blackler, A., & Popovic, V. (2015). Using conceptual tool for intuitive interaction to design intuitive website for SME in India: A case study. In IASDR 2015: Interplay Proceedings (pp. 1500–1521). Brisbane, QLD.
Mohs, C., Hurtienne, J., Israel, J. H., Naumann, A., Kindsmüller, M. C., Meyer, H. A., & Pohlmeyer, A. (2006). IUUI--intuitive use of user interfaces. Usability Professionals, 6(130–133).
Montemayor, J., Druin, A., Farber, A., Simms, S., Churaman, W., & D’Amour, A. (2002). Physical programming: designing tools for children to create physical interactive environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 299–306). New York, NY, USA: ACM. http://doi.org/10.1145/503376.503430
Montessori, M. (2011). Dr. Montessori’s own handbook. Schocken. (Originally published in 1914)
Montessori, M. (2013). The montessori method. Transaction Publishers.
Muller-Tomfelde, C., & Fjeld, M. (2012). Tabletops: Interactive Horizontal Displays for Ubiquitous Computing. Computer, 45(2), 78–81.
Muro, C., Escobedo, R., Spector, L., & Coppinger, R. P. (2011). Wolf-pack (Canis lupus) hunting strategies emerge from simple rules in computational simulations. Behavioural Processes, 88(3), 192–197.
Myers, D. (2007). The powers and perils of intuition. Scientific American Mind, 18(3), 24–31.
Nardi, B. A. (1996). Context and consciousness: activity theory and human-computer interaction. mit Press.
Nathan, M. J. (2008). An embodied cognition perspective on symbols, gesture and grounding instruction. Symbols, Embodiment and Meaning: A Debate, 18, 375–396.
Natural Motions, & Scott, L. (2011). Jenga app.
Naumann, A., & Hurtienne, J. (2010). Benchmarks for intuitive interaction with mobile devices. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 401–402).
Naumann, A., Hurtienne, J., Israel, J., Mohs, C., Kindsmüller, M., Meyer, H., & Hußlein, S. (2007). Intuitive Use of User Interfaces: Defining a Vague Concept. In D. Harris (Ed.), Engineering Psychology and Cognitive Ergonomics SE - 14 (Vol. 4562, pp. 128–136). Springer Berlin Heidelberg. http://doi.org/10.1007/978-3-540-73331-7_14
Naumann, A., Hurtienne, J., Kindsmüller, M. C., Pohlmeyer, A. E., Clemens, C., Knapheide, C., … Israel, J. H. (2009). Exploring Design Criteria for Intuitive Use. In Mensch \& Computer Workshopband (pp. 73–75).
Bibliography 199
Nielsen, J. (1994). Usability inspection methods. Conference Companion on Human Factors in Computing Systems - CHI ’94, 25(1), 413–414. http://doi.org/10.1145/259963.260531
Noddings, N., & Shore, P. J. (1984). Awakening the Inner Eye. Intuition in Education. ERIC.
Noë, A. (2004). Action in perception. MIT press.
Noldus. (1989). Observer XT.
Norman, D. (1993). Things that make us smart: Defending human attributes in the age of the machine. Basic Books.
Norman, D. (1999). Affordance, conventions, and design. Interactions, 6(3), 38–43. http://doi.org/10.1145/301153.301168
Norman, D. (2010). Natural user interfaces are not natural. Interactions, 17(3), 6. http://doi.org/10.1145/1744161.1744163
Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books.
Nunnally, J., & Lemond, C. (1974). Exploratory behavior and human development. Advances in Child Development and Behavior, 8, 59–109.
O’Brien, M. (2010). Understanding human-technology interactions: The role of prior experience and age. ProQuest Dissertations and Theses. Georgia Institute of Technology.
O’hara, K., Harper, R., Mentis, H., Sellen, A., & Taylor, A. (2013). On the naturalness of touchless: Putting the “interaction” back into NUI. ACM Transactions on Computer-Human Interaction (TOCHI), 20(1), 5.
O’Malley, C. E., Draper, S. W., & Riley, M. S. (1984). Constructive interaction: A method for studying human-computer-human interaction. In Proceedings of IFIP Interact (Vol. 84, pp. 269–274).
Olson, I. C., Atrash Leong, Z., Wilensky, U., & Horn, M. S. (2011). It’s Just a Toolbar!: Using Tangibles to Help Children Manage Conflict Around a Multi-touch Tabletop. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 29–36). New York, NY, USA: ACM. http://doi.org/10.1145/1935701.1935709
Pallarino, T., Free, A., Mutuc, K., & Yarosh, S. (2016). Feeling Distance: An Investigation of Mediated Social Touch Prototypes. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (pp. 361–364).
Payne, S. J. (1991). Display-based action at the user interface. International Journal of Man-Machine Studies, 35(3), 275–289.
Pearl, J. (1984). Heuristics: intelligent search strategies for computer problem solving. Addison-Wesley Pub. Co., Inc., Reading, MA.
200 Bibliography
Perry, M., Church, R. B., & Goldin-Meadow, S. (1988). Transitional knowledge in the acquisition of concepts. Cognitive Development, 3(4), 359–400.
Piaget, J. (1952). The Origins of Intelligence in Children. International University Press.
Piaget, J. (2013). The construction of reality in the child (Vol. 82). Routledge. (Originally published in 1955)
Plessner, H., Betsch, C., & Betsch, T. (2011). Intuition in judgment and decision making. Psychology Press.
Popovic, V. (2003). An approach to knowledge generation by research and its utilisation in practice: situating doctoral research around the artifacts.
Popovic, V., Kraal, B. J., Blackler, A. L., & Chamorro-Koc, M. (2012). Observational research and verbal protocol methods. In P. Israsena (Ed.), Design Research Society (DRS) 2012 Conference - Research : Uncertainty, Contradiction and Value (pp. 311–324). Bangkok, Thailand: Department of Industrial Design, Faculty of Architecture, Chulalongkorn University. Retrieved from http://eprints.qut.edu.au/54222/
Pure Digital Technologies, & Cisco Systems. (2006). Flip Video Cameras.
Queensland Studies Authority. (2006). Early years curriculum guidelines, 1. Retrieved from http://www.qsa.qld.edu.au/downloads/learning/ey_cg_06.pdf
Rapp, D. N., & Kurby, C. A. (2008). The “ins” and “outs” of learning: Internal representations and external visualizations. Visualization: Theory and Practice in Science Education, 29–52.
Rasmussen, J. (1985). The role of hierarchical knowledge representation in decisionmaking and system management. IEEE Transactions on Systems, Man, and Cybernetics, (2), 234–243.
Rasmussen, J. (1993). Deciding and doing: Decision making in natural contexts. Decision Making in Action: Models and Methods, 158–171.
Regenbrecht, H., Lum, T., Kohler, P., Ott, C., Wagner, M., Wilke, W., & Mueller, E. (2004). Using Augmented Virtuality for Remote Collaboration. Presence: Teleoperators and Virtual Environments, 13(3), 338–354. http://doi.org/10.1162/1054746041422334
Resnick, L. B. (1986). The development of mathematical intuition. Perspectives on Intellectual Development: The Minnesota Symposia on Child Psychology, 19, 159–194.
Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., & Silverman, B. (1998). Digital Manipulatives : New Toys to Think With, (April), 281–287.
Ricci, A., Piunti, M., Tummolini, L., & Castelfranchi, C. (2015). The mirror world: Preparing for mixed-reality living. IEEE Pervasive Computing, 14(2), 60–63. http://doi.org/10.1109/MPRV.2015.44
Bibliography 201
Robert, D., Wistorrt, R., Gray, J., & Breazeal, C. (2011). Exploring mixed reality robot gaming. Proceedings of the Fifth, 125–128. http://doi.org/10.1145/1935701.1935726
Robertson, T. (1997). Cooperative work and lived cognition: a taxonomy of embodied actions. Proceedings of the 5th Conference on European Conference on Computer-Supported Cooperative Work, 205–220. http://doi.org/10.1007/978-94-015-7372-6_14
Robins, B., & Dautenhahn, K. (2014). Tactile interactions with a humanoid robot: novel play scenario implementations with children with autism. International Journal of Social Robotics, 6(3), 397–415.
Rogers, Y. (2011). Interaction design gone wild: striving for wild theory. Interactions, 18(4), 58–62. http://doi.org/10.1145/1978822.1978834
Rosen, R. (2012). Anticipatory systems. In Anticipatory systems (pp. 313–370). Springer.
Rosenbaum, E., Eastmond, E., & Mellis, D. (2010). Empowering programmability for tangibles. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction (pp. 357–360). New York, NY, USA: ACM. http://doi.org/10.1145/1709886.1709974
Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis. In Essentials of behavioral research: Methods and data analysis (p. p.361). McGraw-Hill Humanities Social.
Ruff, H., & Saltarelli, L. (1993). Exploratory play with objects: basic cognitive processes and individual differences. New Directions for Child Development, (59), 5–16. http://doi.org/10.1002/cd.23219935903
Rutter, M. (1985). Family and school influences on cognitive development. Journal of Child Psychology and Psychiatry, 26(5), 683–704.
Sakai, M., & Sugano, M. (2016). Human tracking system embedded in stuffed animal. In Global Humanitarian Technology Conference (GHTC), 2016 (pp. 842–844).
Salomon, G. (1997). Distributed cognitions: Psychological and educational considerations. Cambridge University Press.
Santos, B. S., Cardoso, J., Ferreira, B. Q., Ferreira, C., & Dias, P. (2016). Developing 3D Freehand Gesture-Based Interaction Methods for Virtual Walkthroughs: Using an Iterative Approach. In Handbook of Research on Human-Computer Interfaces, Developments, and Applications (pp. 52–72). IGI Global.
Sapounidis, T., Demetriadis, S., & Stamelos, I. (2015). Evaluating children performance with graphical and tangible robot programming tools. Personal and Ubiquitous Computing, 19(1), 225–237. http://doi.org/10.1007/s00779-014-0774-3
Saxberg, B. (1987). Projected free fall trajectories. Biological Cybernetics, 56(2), 159–175.
202 Bibliography
Schafer, G. J., Green, K. E., Walker, I. D., Lewis, E., Fullerton, S. K., Soleimani, A., … others. (2013). Designing the LIT KIT: An Interactive, Environmental Mixed-Technology Robotic System for Enhancing Children’s Picture-book Reading.
Schneider, B., Wallace, J., Blikstein, P., & Pea, R. D. (2013). Preparing for future learning with a tangible user interface: The case of neuroscience, 6(2), 117–129.
Schon, D. (1982). Intuitive thinking? A digression stimulated by U-shaped curves. U-Shaped Behavioral Growth, 227–247.
Schore, A. (2010). The right brain implicit self: A central mechanism of the psychotherapy change process. Knowing, Not-Knowing and Sort of Knowing: Psychoanalysis and the Experience of Uncertainty, 177–202.
Scott, L. (2006). Jenga. Board game], Hasbro: Pawtucket, RI.
Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424.
Seo, D. W., & Lee, J. Y. (2013). Direct hand touchable interactions in augmented reality environments for natural and intuitive user experiences. Expert Systems with Applications, 40(9), 3784–3793.
Seo, J., Arita, J., Chu, S., Quek, F., & Aldriedge, S. (2015). Material Significance of Tangibles for Young Children. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 53–56). New York, NY, USA: ACM. http://doi.org/10.1145/2677199.2680583
Shaer, O., & Hornecker, E. (2010). Tangible user interfaces: past, present, and future directions. Foundations and Trends in Human-Computer Interaction, 3(1--2), 1–137.
Shaer, O., & Jacob, R. J. K. (2009). A Specification Paradigm for the Design and Implementation of Tangible User Interfaces. ACM Trans. Comput.-Hum. Interact., 16(4), 20:1--20:39. http://doi.org/10.1145/1614390.1614395
Shapiro, L. (2011). Embodied cognition. New York: Routledge.
Shapiro, L. A. (2004). The mind incarnate.
Sharlin, E., Watson, B., Kitamura, Y., Kishino, F., & Itoh, Y. (2004). On tangible user interfaces, humans and spatiality. Personal and Ubiquitous Computing, 8(5), 338–346.
Sherman, L., Druin, A., & Montemayor, J. (2001). StoryKit: tools for children to build room-sized interactive experiences. CHI’01 Extended …, 197–198. Retrieved from http://dl.acm.org/citation.cfm?id=634186
Simmons, J. P., & Nelson, L. D. (2006). Intuitive confidence: choosing between intuitive and nonintuitive alternatives. Journal of Experimental Psychology: General, 135(3), 409.
Bibliography 203
Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2007). The affect heuristic. European Journal of Operational Research, 177(3), 1333–1352. article.
Søndergaard, D. M. (2013). Virtual materiality, potentiality and subjectivity: How do we conceptualize real-virtual interaction embodied and enacted in computer gaming, imagination and night dreams&quest. Subjectivity, 6(1), 55–78.
SPSS Statistics. (2016). SPSS Statistics. Armonk, NY: IBM Corp.
Strawson, G. (1999). The self and the SESMET. Journal of Consciousness Studies, 6, 99–135.
Streeck, J. (2013). Interaction and the living body. Journal of Pragmatics, 46(1), 69–90. http://doi.org/10.1016/j.pragma.2012.10.010
Streeck, J., Goodwin, C., & LeBaron, C. (2011). Embodied interaction in the material world: An introduction. Embodied Interaction: Language and Body in the Material World, 1–26.
Swaak, J., & De Jong, T. (2001). Discovery simulations and the assessment of intuitive knowledge. Journal of Computer Assisted Learning, 17(3), 284–294. http://doi.org/10.1046/j.0266-4909.2001.00183.x
Swann, L., Popovic, V., Thompson, H., Blackler, A., & Kraal, B. (2015). Relationships between user experience and intuitiveness of visual and physical interactions. In Proceedings of the 6th IASDR (The International Association of Societies of Design Research Congress) (pp. 1900–1916).
Tangible Play. (2014a). Newton.
Tangible Play. (2014b). Osmo. Retrieved from https://www.playosmo.com/en/?gclid=Cj0KCQjwub7NBRDJARIsAP7wlT8ouMB5AovBX3Yt55E94hkb_efik1uIcXmNCIYA13YTAm4lLvHyhwwaAjP2EALw_wcB
Taraborelli, D., & Mossio, M. (2008). On the relation between the enactive and the sensorimotor approach to perception. Consciousness and Cognition, 17(4), 1343–1344.
TEI 2017. (2017). TEI 2017 call for papers. Retrieved November 24, 2014, from http://allconferencecfpalerts.com/cfp/view.php?eno=4132
Terrenghi, L., Kirk, D., Sellen, A., & Izadi, S. (2007). Affordances for manipulation of physical versus digital media on interactive surfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1157–1166).
Thelen, E. (2008). Grounded in the world: Developmental origins of the embodied mind. Developmental Perspectives on Embodiment and Consciousness, 99–129.
Thelen, E., & Smith, L. B. (1996). Dynamic systems approach to the development of cognition and action. Cambridge, USA: The MIT Press.
Thelen, E., & Smith, L. B. (2006). Dynamic systems theories. Handbook of Child Psychology.
204 Bibliography
Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460.
Turner, J. S. (2013). Super organisms and superindividuality: the emergence of individuality in a social insect assemblage. From Groups to Individuals, 219–242.
Uden, L., & Dix, A. (2000). Iconic interfaces for kids on the Internet. In IFIP world computer congress (pp. 279–286).
Ullmer, B., & Ishii, H. (2000). Emerging frameworks for tangible user interfaces. IBM Systems Journal, 39(3.4), 915–931.
Ullrich, D., & Diefenbach, S. (2010). From magical experience to effortlessness: an exploration of the components of intuitive interaction. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 801–804).
van den Hoven, E., & Mazalek, A. (2011). Grasping gestures: Gesturing with physical artifacts. AI EDAM, 25(3), 255–271.
Vanity Fair. (1917, December). Lazy Susan.
Veeriah, V., Pilarski, P. M., & Sutton, R. S. (2016). Face valuing: Training user interfaces with facial expressions and reinforcement learning. arXiv Preprint arXiv:1606.02807.
Vera, A. H., & Simon, H. A. (1993). Situated action: A symbolic interpretation. Cognitive Science, 17(1), 7–48.
Vogiatzaki, E., Gravezas, Y., & Solutions, I. S. A. T. (2013). Rehabilitation System for Stroke Patients using Mixed-Reality and Immersive User Interfaces.
Vogt, P. (2002). The physical symbol grounding problem. Cognitive Systems Research, 3(3), 429–457.
Wang, K. (2015). Research design in counseling. Nelson Education.
Want, R., Fishkin, K. P., Gujar, A., & Harrison, B. L. (1999). Bridging physical and virtual worlds with electronic tags. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 370–377).
Wensveen, Overbeeke, Djajadiningrat, & Kyffin. (2004). Freedom of fun, freedom of interaction. Interactions, 11(5), 59–61.
Whitebread, D., Basilio, M., Kuvalja, M., & Verma, M. (2012). The importance of play: A report on the value of children’s play with a series of policy recommendations. Brussels, Belgium: Toys Industries for Europe.
Wickens, C. D., Gordon, S. E., Liu, Y., & Lee, J. (1998). An introduction to human factors engineering.
Williams, A., Kabisch, E., & Dourish, P. (2005). From interaction to participation: Configuring space through embodied interaction. Proceedings of the Ubicomp 2005, LNCS 3660, 287–304. http://doi.org/10.1007/11551201_17
Bibliography 205
Wilson, A., & Golonka, S. (2013). Embodied Cognition is Not What you Think it is. Frontiers in Psychology, 4(Article 58), 1–13. http://doi.org/10.3389/fpsyg.2013.00058
Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625–36. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12613670
Wundt, W. (1973). The language of gestures (Vol. 6). Walter de Gruyter.
Yoon, S. H., Huo, K., & Ramani, K. (2016). TMotion: Embedded 3D mobile input using magnetic sensing technique. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 21–29).
Yu, B., Bongers, N., Van Asseldonk, A., Hu, J., Funk, M., & Feijs, L. (2016). LivingSurface: Biofeedback through Shape-changing Display. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 168–175).
Zaman, B., Abeele, V., Markopoulos, P., & Marshall, P. (2009). Tangibles for children,: the challenges. {CHI EA} ’09: Proceedings of the 27th International Conference on Human Factors in Computing Systems, 4729–4732.
Zigelbaum, J., Kumpf, A., Vazquez, A., & Ishii, H. (2008). Slurp: tangibility spatiality and an eyedropper. In CHI’08 Extended Abstracts on Human Factors in Computing Systems (pp. 2565–2574).
Zsambok, C. E., & Klein, G. E. (1997). Naturalistic decision making. Lawrence Erlbaum Associates, Inc.
Zuckerman, O., Arida, S., & Resnick, M. (2005). Extending tangible interfaces for education: digital montessori-inspired manipulatives. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 859–868).
Zuckerman, O., Gal, T., Keren-Capelovitch, T., Karsovsky, T., Gal-Oz, A., & Weiss, P. L. T. (2016). DataSpoon: Overcoming Design Challenges in Tangible and Embedded Assistive Technologies. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 30–37).
Appendices 207
Appendices
Appendix A
STRENGTH OF AGREEMENT BASED ON ICC VALUES
Table 12 Strength of agreement based on ICC values, adapted from (Koo & Li, 2016)
ICC Strength of Agreement Less than 0.5 Poor Between 0.5 and 0.75 Moderate Between 0.75 and 0.9 Good Greater than 0.9 Excellent
208 Appendices
Appendix B
PARAMETRIC AND NON-PARAMETRIC STATISTICAL ANALYSIS
Table 13 Differences between assumptions for parametric and non-parametric data analysis
methods, adapted from Field (2008)
Parametric Non-parametric Assumed distribution Normal Any Assumed variance Homogeneous Any Type of data Ratio or Interval Ordinal or Nominal Data set relationships Independent Any Measures of central tendency Mean Median Benefits Draws more conclusions Simplicity, less affected by
outliers Correlation test Pearson Spearman Between subjects, 2 groups Independent measures t-test Mann-Whitney test Between subjects, greater than 2 groups
Independent measures one-way ANOVA
Kruskal-Wallis test
Within subjects, 2 groups Matched pair t-test Wilcoxon test Within subjects, greater than 2 groups
Repeated measures one-way ANOVA
Friedman’s test
Appendices 209
Appendix C
THRESHOLDS FOR EFFECT SIZE
Table 14 Thresholds for interpreting effect size, adapted from Cohen (1992, p.40) for correlation and
Rosenthal & Rosnow (1991, p.361) for Mann Whitney U test.
Statistical Test
Relevant effect size Small Medium Large Very Large
Mann Whitney U Test
/√ 0.20 0.50 0.80 1.30
Correlation ^2/√ ^2 4 0.10 0.30 0.50 0.70
210 Appendices
Appendix D
Coding heuristics for Types of Interaction
Table 15 Coding heuristics for Types of Interaction
Sub-themes Example Heuristics Type of interaction
Intuitive - Unconscious reasoning (Blackler, 2008) - Less verbalisations (Bastick, 1982, 2003; Blackler, 2008; Chalmers, 1995) - High degrees of certainty and confidence about decisions and interactions (Blackler, 2008; Hammond, 1993; Simmons & Nelson, 2006) - Fast decision making (Plessner, Betsch, & Betsch, 2011)
Non-Intuitive - Conscious reasoning (Blackler, 2008) - More Verbalisations of behaviour and interactions (Bastick, 1982, 2003; Blackler, 2008) - high degrees of uncertainty and lack of confidence about decisions and interactions (Blackler, 2008; Simmons & Nelson, 2006) - Slow decision making (Plessner, Betsch, & Betsch, 2011) - Attention to visual features in interactions (Liu & Gale, 2011)
Partially-Intuitive - Limited verbalisations (Bastick, 1982, 2003; Blackler, 2008; Chalmers, 1995) - A combination of intuitive and non-intuitive processes (some automation, but higher level processes requiring attentive control (Baylor, 2001; Levi, 2015)
Appendices 211
Appendix E
CODING HEURISTICS FOR ASPECTS OF EMBODIMENT
Table 16 Coding heuristics for Aspects of Embodiment
Theme Groups
Sub-themes Example Heuristics
Aspects of Embodiment
Physical affordance
- Interactions and behaviours determined by physical and material properties (Gibson, 1974/2014) - Rely on natural clues to determine actions offered by interfaces (Dotov et al., 2012)
Perceived affordance
- Interactions and behaviours determined by prior experience with similar things (Blackler et al., 2010) - Learned conventions (Norman, 2013) - Rely on deliberate clues inserted by designer for specific interactions with the interface (Dotov et al., 2012)
Emergence - Adaption in behaviour and interactions as dynamic systems evolve in their properties (Maier & Fadel, 2009) - Adaption in behaviour and interactions as knowledge evolves over time (Allen & Strathern, 2003)
Scaffolding - Offloading tasks into epistemic actions (Kirsh & Maglio, 1994) - Use of the environment, physical objects, tools, processes, and support mechanisms in interactions (Loorbach, Karreman, & Steehouder, 2013).
Cooperative activity
- Well-articulated division of labour (Xiao, 2005) - interactions performed by different people to achieve a common goal (Terrenghi et al., 2007). - Integrate contributions of all people involved in the activity (D. W. Johnson & Johnson, 1994) - Access the status information (Felemban, Gardner, Callaghan, & Pena-Rios, 2017) - Visibility of others interactions and behaviours (Felemban et al., 2017) - Delegate work to the rest of the team(Frederking, Cruz, Overbeeke, & Baskinger, 2007) - Lack of intrusion upon other’s activities (Schneider, Wallace, Blikstein, & Pea, 2013)
212 Appendices
Appendix F
CONSENT FORM FOR CHILDREN AND PARENTS
CONSENT FORM FOR QUT RESEARCH PROJECT – Questionnaire, Interview and Observations–
Children’s intuitive interaction
QUT Ethics Approval Number 1300000826
RESEARCH TEAM CONTACTS
Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic
0411 441 584 07 3138 7030 07 3138 [email protected] [email protected] [email protected]
STATEMENT OF CONSENT
By signing below, you are indicating that you:
Have read and understood the participant information document regarding this
project.
Have had any questions answered to your satisfaction.
Understand that if you have any additional questions you can contact the research
team.
Understand that you are free to withdraw at any time, without comment or penalty.
Understand that you can contact the Research Ethics Unit on 07 3138 5123 or email
if you have concerns about the ethical conduct of the
project.
Have discussed the project with your child and what is required of them if
participating.
Understand that the project will include an audio and video recording.
Agree to participate in the project.
Name
Appendices 213
Signature
Date
STATEMENT OF CHILD CONSENT
Your parent or guardian has given their permission for you to be involved in this research
project. They have talked to you about participating in a research project related to playing
with toys and apps. Please colour one of the faces below that shows how you feel about
taking part in this study.
Name
Date
Please return this sheet to the investigator
214 Appendices
Appendix G
CONSENT FORM FOR SCHOOL PRINCIPAL
PRINCIPAL CONSENT FORM FOR QUT RESEARCH PROJECT
Children’s intuitive interaction
QUT Ethics Approval Number 1300000826
RESEARCH TEAM CONTACTS
Ms. Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic
0411 441 584 07 3138 7030 07 3138 2669
[email protected] [email protected] [email protected]
STATEMENT OF CONSENT FROM THE PRINCIPAL
By signing below, you are indicating that you:
Have read and understood the information document regarding this project.
Have had any questions answered to your satisfaction.
Understand that if you have any additional questions you can contact the research
team.
Understand that you are free to withdraw at any time, without comment or penalty.
Understand that you can contact the Research Ethics Unit on 07 3138 5123 or email
if you have concerns about the ethical conduct of the
project.
Understand that the project will include an audio and video recording.
Agree to allow us to approach parents and children at your school and potentially
conduct experiments at the school.
Name
Signature
Appendices 215
Date
Please return this sheet to the investigator.
Appendix H
IMAGE RELEASE CONSENT FORMS
Image Release: Parents and Children
PLEASE RETURN THIS COMPLETED FORM TO: Shital Desai
A COPY WILL BE PROVIDED FOR YOUR RECORDS
If you agree to give consent regarding the use of your image in the audio and video
recording for research purposes, please read and complete the consent below.
PARENT/GUARDIAN CONSENT
I agree to the University using, reproducing and disclosing photographic or video
images of me as explained in the Image Release Information document, Participant
Information Sheet and Consent Form.
I agree that I will make no claim against QUT for any payment or fee for appearing
in the video recording and release QUT from any other claims arising out of the
University’s use of the images of me.
I understand that the anonymity afforded to me as a participant in the research
project “Children’s intuitive interaction” will be rescinded if I appear in this video.
By signing below, you are indicating that you have discussed participation in the
video recording with your child and you are the legal guardian to provide consent to
participate.
Name of
Parent/Guardian
Signature of
Parent/Guardian
216 Appendices
Date
CHILD CONSENT
Your parent or guardian has given their permission to use your image in the audio
and video recording for research purposes. They have discussed with you about your
participation in the video recording of your playtime and interview. Please colour
one of the faces below which shows how you feel about being audio and video
recorded.
Name
Date
Please return this sheet to the investigator.
Appendices 217
Appendix I
IMAGE RELEASE INFORMATION SHEET itle
Image Release: Parents of children participating in the research QUT Ethics Approval Number 1300000826
A photographic image (including a video recording) which is sufficiently clear to enable your child to be identified as an individual is personal information. Queensland University of Technology (QUT) seeks to comply with the Information Privacy Principles as set out in the Information Privacy Act 2009. QUT shall, from time to time, endorse a privacy policy (see www.mopp.qut.edu.au ) to ensure that personal information is used and disclosed only in ways which are consistent with privacy principles and will otherwise comply with QUT’s privacy obligations under statute. In general, personal information is not disclosed or published except where an individual’s consent has been obtained.
QUT is seeking your consent to use an image of you and/or your child in media releases and research publications.
Participation in this release is voluntary.
You and your child can still participate in the experiment even if you opt not to provide consent to use the images and videos in presentations and publications.
Your decision to participate or to not participate will in no way impact upon your current or future relationship with Ms Shital Desai, Assoc Prof. Althea Blackler, Prof. Vesna Popovic or with QUT.
If you or your child have any questions please ensure you have discussed them and are comfortable with the response before providing consent. You might choose to discuss participation with the following people:
The researchers: Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic
Family or friends.
What is the release about? Your child will be observed with another child who is known to him/her playing with a toy for 60-90 minutes. The observations will be audio and video recorded to determine how comfortable your child is playing with the toy. The video recording will be 60-90 minutes in length.
Why do you want to include my child? It is important to include your child in the recording as their facial expressions, reaction to situations and conditions and the way they use the toy will enable us to determine whether the toy is comfortable to play with.
What will you ask my child to do? Your child will be asked to play with a toy or an app along with another child who is known to your child. The observations will be followed with a brief interview where your child will be asked about his/her experience playing with the toy or the app. The playtime and interview will form the basis of the video content. The length of each filming session might vary; it is estimated that your and your child’s involvement would require a time commitment of between one to two hours.
218 Appendices
The interview will probe your child to determine the attributes contributing to the satisfaction or dissatisfaction of playing with the toy.
Are there any benefits for me or my child in taking part? While the filming and publication of this video is not expected to provide any direct benefits to you or your child, the video is not expected to be of detriment to your child either. The research community in general seeks to benefit from this audio and video recording as it would result in a framework for designing products for children that facilitate intuitive use. A gift voucher of 20$ will be given to you as a token of thanks.
Are there any risks for me or my child in taking part? We believe that there are no risks
beyond normal day‐to‐day living associated with your and your child’s participation in this project. If
your child becomes upset while playing, the filming will be stopped. It is not the intention of the
video to portray any emotional discomfort.
Confidentiality The faces and speech of all children will be included in the video. QUT understands that you and your children might not wish to be named in the video. As a result the names of all children will be excluded from the video. QUT will only identify your child in the video on the basis of their association with the researcher, i.e. child(ren) in research program.
Who will see the video? The video will be used by the researchers to analyse the data and in research presentations and the images will be used in research publications and presentations.
Can I change my mind? You can decide to withdraw your participation at any stage of the experiment. However, after the experiment and once the image or video has been used in publications and presentations or in data analysis, it will not be possible to withdraw.
I am interested – what should I do next? All persons appearing in this video will be required to sign the attached Consent Form, acknowledging that they have read and understood the Image Release Information Sheet, and agree to allow the use of their image and voice in the video for QUT research purposes.
If you have any questions about this video, please do not hesitate to contact:
Shital Desai 0411 441 584 [email protected]
Thank you for helping with this research project. Please keep this sheet for your information.
Appendices 219
Appendix J
PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 1 AT
SCHOOL
PARTICIPANT’s PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT
– Experiment 1: Questionnaire, Observational Study, Interview –
Children’s intuitive interaction
QUT Ethics Approval Number 1300000826
RESEARCH TEAM
Principal Researcher:
Shital Desai, PhD student
Associate Researchers:
Associate Professor Althea Blackler and Professor Vesna Popovic
School of Design – Creative Industries Faculty – Queensland University of Technology (QUT)
DESCRIPTION OF RESEARCH
This study investigates methods to design products for children that are intuitive to use. It will focus on product features that enable children to interact with the products naturally and with ease. It investigates how physical interaction with products makes them easier and more intuitive to use for children. By looking at how children play with toys and with apps on a tablet, this research will inform us if children use products more naturally when they are able to sense, perceive and act to take a decision in regards to the use of the product. The outcome of this work will be a design framework to develop intuitive products for children.
220 Appendices
PARTICIPATION
Participation by your child in this study will involve the following:
Your child with your help will complete a 10‐minute questionnaire about the toys and apps
that they play with. The questionnaire is provided in the information pack to be filled out and
send it back with your child to the school along with the signed consent forms.
On the day of the experiment, your child will be paired with another child from his/her class
at the school. Together, they will play with a toy or an app for 30 minutes at the school. The
observations will be audio and video recorded for analysing the data after the experiments.
Children will be encouraged to talk and think aloud to describe how they are using the toy or
the app and why are they following certain strategies while playing the game.
The observations will be immediately followed up with a 20‐minute interview with the
children to determine their satisfaction levels playing with the toy or the app. Children will be
asked about the features of the toy or the app that made it intuitive and satisfying to use. The
interview will be audio and video recorded for analysis.
The toys that that have been shortlisted for the experiments are available in local Australian toy
shops and comply with the Australian toy standard AS/NZ 8124. The apps will be compatible to IOS
and Android platforms and will be downloaded from ITunes and Google Play. The toys and apps are
age appropriate as per recommendations on the toy packaging and the app store.
You are most welcome to stay through the experiments. However, we do ask that you do not
interfere with the experiment or help the children in playing with the toy or the app to ensure
validity of results. Please indicate in the questionnaire whether you would like to be present at the
experiments. You will be contacted preferably by email once the date and time for the experiments
is finalised.
It is not mandatory for you to sit through the experiments. Your child will be supervised and
cared for at all times.
EXPECTED BENEFITS
This research will lead to a better understanding of how to design products for children that facilitate
intuitive use. The research outcomes will benefit the design community in creating engaging
products for children.
RISKS
There will be no more risk to your child than day‐to‐day activities. Shital Desai, the chief investigator
has undergone first aid training and will be present at all times during the experiments.
PRIVACY AND CONFIDENTIALITY
All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial
Appendices 221
expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child's facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, QUT Gardens Point Campus for mandatory minimum duration of 5 years. The access to cabinets and the laboratory premises is available to authorised personnel only. The data (video recordings) will primarily be stored on a laboratory server which is located in the QUT library. The data will be imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT computer is secured and password protected. The data will be accessed only by the researchers, Ms Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.
CONSENT TO PARTICIPATE
Participation in this research project is voluntary. A decision not to participate will not adversely
affect your child’s academic achievement or their relationship with their teachers or the school. You
and your child might withdraw from the study at any time without giving any reason.
We would like to ask you and your child to sign written consent forms (enclosed) to confirm your
agreement to participate.
QUESTIONS / FURTHER INFORMATION ABOUT THE PROJECT
If have any questions or require further information please contact one of the research team
members below.
Ms Shital Harshad Desai Aspro Althea Blackler Prof Vesna Popovic 0411 441 584 07 3138 7030 07 3138 2669 [email protected] [email protected] [email protected]
CONCERNS / COMPLAINTS REGARDING THE CONDUCT OF THE PROJECT
QUT is committed to research integrity and the ethical conduct of research projects. However, if you
do have any concerns or complaints about the ethical conduct of the project you might contact the
QUT Research Ethics Unit on 07 3138 5123 or email . The QUT
Research Ethics Unit is not connected with the research project and can facilitate a resolution to
your concern in an impartial manner.
222 Appendices
Thank you for helping with this research project. Please keep this sheet for your
information.
Appendix K
PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 1 AT PAS
LAB
PARTICIPANT’s PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT
– Experiment 1: Questionnaire, Observational Study, Interview –
Children’s intuitive interaction
QUT Ethics Approval Number 1300000826
RESEARCH TEAM
Principal Researcher:
Shital Desai, PhD student
Associate Researchers:
Associate Professor Alethea Blackler and Professor Vesna Popovic
School of Design – Creative Industries Faculty – Queensland University of Technology (QUT)
DESCRIPTION OF RESEARCH
This study investigates methods to design products for children that are intuitive to use. It will focus on product features that enable children to interact with the products naturally and with ease. It investigates how physical interaction with products makes them easier and more intuitive to use for children. By looking at how children play with toys and with apps on a tablet, this research will inform us if children use products more naturally when they are able to sense, perceive and act to take a decision in regards to the use of the product. The outcome of this work will be a design framework to develop intuitive products for children.
PARTICIPATION
Appendices 223
Participation by your child in this study will involve the following:
Your child with your help will complete a 10‐minute questionnaire about the toys and apps
that they play with. The questionnaire is provided in the information pack to be filled out in
your spare time and send it back along with the signed consent forms. You could also bring
them to the lab on the day of the experiment to be handed over to the researchers.
On the day of the experiment, your child will be paired with another child who is known to
him/her. Together, they will play with a toy or an app for 30 minutes at the People and
Systems lab at QUT. The observations will be audio and video recorded for analysing the data
after the experiments. Children will be encouraged to talk and think aloud to describe how
they are using the toy or the app and why are they following certain strategies while playing
the game.
The observations will be immediately followed up with a 20‐minute interview with the
children to determine their satisfaction levels playing with the toy or the app. Children will be
asked about the features of the toy or the app that made it intuitive and satisfying to use. The
interview will be audio and video recorded for analysis.
The toys that that have been shortlisted for the experiments are available in local Australian toy
shops and comply with the Australian toy standard AS/NZ 8124. The apps will be compatible to IOS
and Android platforms and will be downloaded from iTunes and Google Play. The toys and apps are
age appropriate as per recommendations on the toy packaging and the app store.
You are most welcome to stay through the experiments. However, we do ask that you do not
interfere with the experiment or help the children in playing with the toy or the app to ensure
validity of results. It is not mandatory for you to sit through the experiments. Your child will be
supervised and cared for at all times.
EXPECTED BENEFITS
This research will lead to a better understanding of how to design products for children that facilitate
intuitive use. A gift voucher of $20 will be given to you as a token of thanks. Apart from this, there
will be no direct benefit for you and your child. The research outcomes will benefit the design
community in creating engaging products for children.
RISKS
There will be no more risk to your child than day‐to‐day activities. Shital Desai, the chief investigator
has undergone first aid training and will be present at all times during the experiments.
PRIVACY AND CONFIDENTIALITY
All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research
224 Appendices
dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child's facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, QUT Gardens Point Campus for mandatory minimum duration of 5 years. The access to cabinets and the laboratory premises is available to authorised personnel only. The data (video recordings) will primarily be stored on a laboratory server which is located in the QUT library. The data will be imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT computer is secured and password protected. The data will be accessed only by the researchers, Ms Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.
CONSENT TO PARTICIPATE
Participation in this research project is voluntary. A decision not to participate will not adversely
affect your child’s academic achievement or their relationship with their teachers or the school. You
and your child might withdraw from the study at any time without giving any reason.
We would like to ask you and your child to sign written consent forms (enclosed) to confirm your
agreement to participate.
QUESTIONS / FURTHER INFORMATION ABOUT THE PROJECT
If have any questions or require further information please contact one of the research team
members below.
Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic 0411 441 584 07 3138 7030 07 3138 2669 [email protected] [email protected] [email protected]
CONCERNS / COMPLAINTS REGARDING THE CONDUCT OF THE PROJECT
QUT is committed to research integrity and the ethical conduct of research projects. However, if you
do have any concerns or complaints about the ethical conduct of the project you might contact the
QUT Research Ethics Unit on 07 3138 5123 or email . The QUT
Research Ethics Unit is not connected with the research project and can facilitate a resolution to
your concern in an impartial manner.
Thank you for helping with this research project. Please keep this sheet for your information.
Appendices 225
Appendix L
PARTICIPANT INFORMATION SHEET FOR EXPERIMENT 2
PARTICIPANT’S PARENT INFORMATION SHEET FOR QUT RESEARCH PROJECT
– Experiment 2: Questionnaire, Observational Study, Interview –
Children’s intuitive interaction
QUT Ethics Approval Number 1300000826
RESEARCH TEAM
Principal Researcher:
Shital Desai, PhD student
Associate Researchers:
Associate Professor Althea Blackler and Professor Vesna Popovic
School of Design – Creative Industries Faculty – Queensland University of Technology (QUT)
DESCRIPTION
This study investigates methods to design products for children that are intuitive to use. It will focus on product features that enable children to interact with the products naturally and with ease. It investigates how physical interaction with products makes them easier and more intuitive to use for children. By looking at how children play with toys and with apps on a tablet, this research will inform us if children use products more naturally when they are able to sense, perceive and act to take a decision in regards to the use of the product. The outcome of this work will be a design framework to develop intuitive products for children.
PARTICIPATION
Participation by your child in this study will involve the following:
226 Appendices
Your child with your help will complete a 10‐minute questionnaire about the toys and apps
that they play with. The questionnaire is provided in the information pack to be filled out in
your spare time and send it back along with the signed consent forms. You could also bring
them to the lab on the day of the experiment to be handed over to the researchers.
On the day of the experiment, your child will be paired with another child who is known to
him/her. Together, they will play with a toy for around 90 minutes, at the People and Systems
lab at QUT. The observations will be audio and video recorded for analysing the data after the
experiments. Children will be encouraged to talk and think aloud to describe how they are
using the toy or the app and why are they following certain strategies while playing the game.
The observations will be immediately followed up with a 20‐minute interview with the
children to determine their satisfaction levels playing with the toy or the app. Children will be
asked about the features of the toy or the app that made it intuitive and satisfying to use. The
interview will be audio and video recorded for analysis.
The toys that that have been shortlisted for the experiments are available in local Australian toy
shops and comply with the Australian toy standard AS/NZ 8124. The apps will be compatible to IOS
and Android platforms and will be downloaded from iTunes and Google Play. The toys and apps are
age appropriate as per recommendations on the toy packaging and the app store.
You are most welcome to stay back through the experiments. However, we do ask that you do not
interfere with the experiment or help the children in playing with the toy or the app to ensure
validity of results. It is not mandatory for you to sit through the experiments. Your child will be
supervised and cared for at all times.
EXPECTED BENEFITS
This research will lead to a better understanding of how to design products for children that facilitate
intuitive use. A gift voucher of $20 will be given to you as a token of thanks. Apart from this, there
will be no direct benefit for you and your child. The research outcomes will benefit the design
community in creating engaging products for children.
RISKS
There will be no more risk to your child than day‐to‐day activities. Shital Desai, the chief investigator
has undergone first aid training and will be present at all times during the experiments.
PRIVACY AND CONFIDENTIALITY
All data collected will be only used for the research project titled, “Children’s intuitive interaction”. The results will be published in research publications and presented at conferences. The video recordings will be used for analysis and images will be used in research publications and presentations. However, we will not identify you or your child by names. Wherever possible, the faces in the images will be blurred. Images cannot be blurred if it is being used to describe facial expressions of your child while playing with the toy. Name and contact details of you and your child will be stored for future communications in relation to the experiments. Consent will be obtained from you and your child to use the data collected during experiments and the images of your child in publications, research presentations and any other form of research dissemination. You will be asked to sign the consent forms and the image release form. If you decide not to sign the image release form, your child’s facial features in the image will be pixelated if the image is used in publications and presentations, but your child can still participate in the experiment. Research data will be stored in secured locked cabinets in the School of Design, Block D, Gardens
Appendices 227
Point Campus, QUT for mandatory minimum duration of 5 years. The access to cabinets and the
laboratory premises is available to authorised personnel only. The data (video recordings) will
primarily be stored on a laboratory server which is located in the QUT library. The data will be
imported on a QUT computer hard disk for analysis. The access to the laboratory server and the QUT
computer is secured and password protected. The data will be accessed only by the researchers, Ms
Shital Desai, Associate Professor Althea Blackler, Professor Vesna Popovic.
CONSENT TO PARTICIPATE
Participation in this research project is voluntary. A decision not to participate will not adversely
affect your child’s academic achievement or their relationship with their teachers or the school. You
and your child might withdraw from the study at any time without giving any reason.
We would like to ask you and your child to sign written consent forms (enclosed) to confirm your
agreement to participate.
QUESTIONS / FURTHER INFORMATION ABOUT THE PROJECT
If have any questions or require further information please contact one of the research team
members below.
Ms Shital Harshad Desai Assoc Prof. Althea Blackler Prof. Vesna Popovic
0411 441 584 07 3138 7030 07 3138 2669
[email protected] [email protected] [email protected]
CONCERNS / COMPLAINTS REGARDING THE CONDUCT OF THE PROJECT
QUT is committed to research integrity and the ethical conduct of research projects. However, if you
do have any concerns or complaints about the ethical conduct of the project you might contact the
QUT Research Ethics Unit on 07 3138 5123 or email . The QUT
Research Ethics Unit is not connected with the research project and can facilitate a resolution to
your concern in an impartial manner.
Thank you for helping with this research project. Please keep this sheet for your
information.
228 Appendices
Appendix M
RECRUITMENT EMAIL
2)
PARTICIPATE IN RESEARCHInformation for Prospective Participants
QUT Ethics approval Number: 1300000826
The following research activity has been reviewed via QUT arrangements for the conduct of research involving human participation. If you choose to participate, you will be provided with more detailed participant information, including who you can contact if you have any concerns.
12/1/2013 Ms. Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic People and Systems Lab, Creative Industries Faculty, D Block, School of Design, Gardens Point Campus, Queensland University of Technology, Brisbane. Dear Parents, Guardians and Children,
My name is Shital Desai from the School of Design, Creative Industries Faculty, Queensland University of Technology and I’m doing a PhD titled ‘Embodied Children’s intuitive interaction’. This study investigates methods to design products for children that are intuitive to use. Intuitive use of products results in engaging, easy to use, stress free interaction with the product. The study will observe children interacting with products that are commonly used by them in everyday life such as toys and games. The toys will be appropriate to the child’s age group, example of such toys are Elefun, Tip It Balancing, Hungry Hungry Hippoes, etc. The apps are intangible versions of the toys. I’m looking for boys and girls aged 5 years to 14 years to participate in an experiment that would last for approximately 60 minutes. The experiment requires that the participant should not have played with the toy/app before the experiment. I will confirm with you about this when I contact you to schedule and organise the experiment and also on the day of the experiment. If you have played with the toy before, I will have to unfortunately decline your participation. There will be no more risk to the participants than day‐to‐day activities. The children will be under constant supervision of the researchers and their parents/guardians at all times. The chief investigator has undergone first aid training and will be present at all times during the experiments. A comprehensive Research Safety assessment has been undertaken and approved by the Health and
Appendices 229
Safety department at Creative Industries faculty, QUT. An ethics approval has been obtained from the Human Research Ethics Committee (HREC) at QUT. The project is funded by the Department of Industry, Innovation, Science, Research and Tertiary Education. The funding body will not have access to personally identifying information about you that might be obtained during the project. A gift voucher of A$20 will be given to every participant family as a token of thanks. If you would like to participate in this study, please contact the research team at [email protected] or 0411 441 584. You will be provided with further information to ensure that your decision and consent to participate is fully informed. Many thanks for your consideration of this request. Yours Sincerely, Ms. Shital Desai, Assoc. Prof Althea Blackler, Prof. Vesna Popovic
Queensland University of Technology
[email protected], 0411 441 584
[email protected] , 0410 736 494
[email protected] , 0439715407
Appendix N
EDUCATION QUEENSLAND APPROVAL
230 Appendices
Appendices 231
232 Appendices
Appendix O
QUT HUMAN RESEARCH ETHICS COMMITTEE APPROVAL
Appendices 233
234 Appendices
Appendix P
Arrangements in Monkey Blocks given to children in Experiment 2
Figure 31 Arrangements for Monkey Blocks game in black and white and colour