Characterization of Executive Dysfunction in Real World Tasks: Analysis of Behaviours Performed During the
Completion of the Multiple Errands Test
by
Sidrah Arshad
A thesis submitted in conformity with the requirements for the degree of Masters of Science
Graduate Department of Rehabilitation Science University of Toronto
© Copyright by Sidrah Arshad (2011)
ii
Characterization of executive dysfunction in real world tasks:
Analysis of behaviours performed during completion of the
Multiple Errands Test
Sidrah Arshad
Degree of Masters of Science
Graduate Department of Rehabilitation Science »
University of Toronto
2011
Abstract
This study furthers our understanding of the impact of executive dysfunction on everyday
activities in stroke survivors. A classification system was developed to analyze a wide range of
behaviours performed by 14 stroke survivors and 12 matched controls on the Baycrest Multiple
Errands Test, a task requiring participants to buy specific items and collect certain information
on the main floor of the hospital. The event recorder was used to code the occurrences and
frequencies of behaviours performed by participants. Results demonstrated that participants with
stroke performed significantly more task specific relevant inefficient behaviours (p < .05) and
non-task specific irrelevant behaviours (p < .10) compared to controls. This study indicates the
importance of performing a detailed analysis of behaviours performed to better understand the
impact of ED in everyday life.
iii
Acknowledgments
I would like to wholeheartedly thank my supervisor, Dr. Deirdre Dawson without whom this
thesis would not have been possible. I am especially grateful of her for having me as her student
and giving me the opportunity to pursue my graduate studies, attend conferences and meetings,
and co-supervise the research project of two Masters of Occupational Therapy students at the
University of Toronto. Her advice, expertise, and guidance throughout my research will always
be remembered and cherished. I would also like to thank my program advisory committee
members, Drs. Nicole Anderson and Helene Polatajko for their valuable and helpful feedback
and continued support throughout this study. I also owe my deepest gratitude to the various
funding bodies who have provided me with financial support: The Heart and Stroke Foundation
Centre for Stroke Recovery, Finkler Graduate Student Fellowship, Ontario Research Coalition,
Jack & Rita Catherall Funds and the University of Toronto.
The findings in this research would not have been possible without access to participant videos
and sensitive information from an existing database, for which I am thankful to the McDonnell
Foundation for their grant to Dr. D. Stuss supporting this work and to Dr. Deirdre Dawson for
allowing me to work with this information extensively.
I would like to thank my colleagues who have supported me along the way and to the faculty and
staff of the Graduate Department of Rehabilitation Science and Baycrest. On a final note, I
would like to offer my regards to my parents, sisters and brother who have greatly supported me
throughout my studies, and especially to my husband Abdul Samad Ahmed, for the
encouragement to pursue this degree and for his technical expertise involved in the research.
iv
Table of Contents
Abstract ........................................................................................................................................... ii Acknowledgments .......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv List of Tables ................................................................................................................................. vi List of Appendices ........................................................................................................................ vii Chapter 1 Introduction .................................................................................................................... 1 Chapter 2 Literature Review ........................................................................................................... 3
Introduction ................................................................................................................................ 3 2.1 Description, Definitions, and Neuroanatomy of Executive Functions ............................... 3 2.2 Executive Dysfunction: Prevalence and Impact on Everyday Life .................................... 4
2.3 Theories of Executive Functions ........................................................................................ 5 2.3.1 Theory of Goal Neglect .......................................................................................... 6 2.3.2 Adaptive Coding Model .......................................................................................... 7 2.3.3 Supervisory Attentional System (SAS) .................................................................. 7
2.3.4 Fractionation of the Supervisory System ................................................................ 8 2.3.5 Transcending the Default Mode .............................................................................. 9
2.3.6 Executive Knowledge (Structured Event Complexes) ......................................... 10 2.4 Assessment of Executive Functions .................................................................................. 11
2.4.1 Traditional Measurements of Executive Functions .............................................. 11
Critique of Traditional Measurements of Executive Functions: Ecological Validity ....... 16 2.4.2 Real World Measurements of Executive Functions .............................................. 18
Ecological Validity of Read-World Measurements of Executive Functions .................... 32 Conclusion................................................................................................................................ 34
Chapter 3 Describing the Methodology: Event Recording ........................................................... 36 Background .............................................................................................................................. 36
3.1 Event Recorder: Behaviour Tracker ................................................................................. 37 3.2 Procedure for Using Behaviour Tracker ........................................................................... 38
3.2.1 Creating the codes ................................................................................................. 38
3.2.2 Structuring the Codes ............................................................................................ 39 3.2.3 Coding Behaviors .................................................................................................. 40 3.2.4 Reliability .............................................................................................................. 41
3.3 Other Application of the Codes and Conclusion .............................................................. 41 Chapter 4 Characterization of executive dysfunction in real world tasks: Analysis of
behaviours performed during completion of the Multiple Errands Test .................................. 42 Abstract .................................................................................................................................... 42
Introduction .............................................................................................................................. 42 Materials and Methods ............................................................................................................. 45
Participants ........................................................................................................................ 45
The BMET ........................................................................................................................ 47 Coding procedure .............................................................................................................. 48
Results ...................................................................................................................................... 49 Behaviour classification .................................................................................................... 49 Related results ................................................................................................................... 51 Further investigation of results ......................................................................................... 53 'Stroke only' vs. 'control only' behaviours ......................................................................... 54
v
Discussion ................................................................................................................................ 54
Further investigation of results ......................................................................................... 57 'Stroke only' vs. 'controls only' behaviours ....................................................................... 59 Future Directions .............................................................................................................. 60
Study limitations ............................................................................................................... 60 Conclusions .............................................................................................................................. 61
Chapter 5 Discussion .................................................................................................................... 62 Theories of executive functions ............................................................................................... 63 The importance of real world assessments and behaviour analysis ......................................... 65
Improving the BMET ............................................................................................................... 67 Limitations ............................................................................................................................... 70 Future directions ....................................................................................................................... 72 Summary & conclusions .......................................................................................................... 72
References ..................................................................................................................................... 73 Appendix A: Behaviour Tracker Modes ....................................................................................... 88
(a) Configuration Mode .......................................................................................................... 88 (b) Record Mode ..................................................................................................................... 88
(c) Editor Mode ...................................................................................................................... 89 (d) Viewer Mode .................................................................................................................... 89
Appendix B: BMET Participant Package ..................................................................................... 90
Appendix C: Meta ......................................................................................................................... 92 Appendix D: Metb ........................................................................................................................ 92
Appendix E: Metc ......................................................................................................................... 93 Appendix F: Metd ......................................................................................................................... 93 Appendix G: List of Behaviours Observed During the BMET .................................................... 94
Appendix H: Behaviour Classification ......................................................................................... 96
vi
List of Tables
Table 2.1 Traditional Measurements of EF .................................................................................. 12 Table 2.2 Real World Measurements of EF .................................................................................. 19 Table 4.1 Participant Characteristics ............................................................................................ 46 Table 4.2 Inter-rater Reliability .................................................................................................... 49 Table 4.3 Behaviour Classification ............................................................................................... 51
Table 4.4 Classification of participants' behaviours on the BMET. Differences in means,
SD, range and p values between stroke-ED, stroke and control groups for each behaviour
category ......................................................................................................................................... 52 Table 4.5 Specific behaviours findings. Differences in mean, SD, range and p values
between stroke-ED, stroke and control groups on two behaviours .............................................. 53 Table 4.6 'Stroke only' vs. 'control only' behaviours .................................................................... 54
vii
List of Appendices
Appendix A: Behaviour Tracker Modes ....................................................................................... 88 Appendix B: BMET Participant Package ..................................................................................... 90
Appendix C: Meta ......................................................................................................................... 92 Appendix D: Metb ........................................................................................................................ 92 Appendix E: Metc ......................................................................................................................... 93 Appendix F: Metd ......................................................................................................................... 93 Appendix G: List of Behaviours Observed During the BMET .................................................... 94
Appendix H: Behaviour Classification ......................................................................................... 96
1
Chapter 1 Introduction
The main goal of this thesis was to further our understanding of the impact of executive
dysfunction on everyday performance in people following stroke. This is important because
executive dysfunction affects many aspects of day-to-day activities such as preparing a meal and
taking public transportation to more challenging tasks involving decision making such as
managing a budget. Individuals with stroke may do relatively well on traditional
neuropsychological tests, yet still manifest executive impairments (Burgess et al., 2006; Chan,
Shum, Toulopoulou, & Chen, 2008; Manchester, Priestley, & Howard, 2004). This in part seems
to be due to the highly structured nature of traditional neuropsychological tests. Also, since the
ecological validity (the degree to which test performance reflects real world performance
(Chaytor & Schmitter-Edgecombe, 2003)), of these tests is not strong, poor performance on the
tests tells us relatively little about how people will perform in their daily life (Burgess et al.,
2006).
For this reason, the Multiple Errands Test (MET) was created. The MET involves participants
performing real-life tasks such as shopping and collecting information and has good ecological
validity (Dawson et al., 2009). It allows the examiners to observe participants' abilities to plan,
organize and perform tasks in an efficient manner. Examining the underlying real world
behaviours that are particularly problematic for people with executive impairments as they
attempt to achieve tasks on the MET will provide a better understanding of the impact of
executive dysfunctions on everyday activities. In turn, this will enable clinicians to work more
effectively with people with executive dysfunction and implement effective interventions, which
will enhance their overall quality of life.
This thesis is organized in a manuscript format. Chapter 2 provides the background literature to
understand executive functioning in the context of everyday behaviours. Relevant theories of
executive functions, which help explain these processes, are discussed. A large section of the
chapter is dedicated to the assessments of executive function, including traditional and
naturalistic, real world measures. These are reviewed on the basis of the processes hypothesized
to be measured by the test and the ecological validity of each is also discussed.
2
Chapter 3 details the methodology used in the study. A description of how the software
Behaviour Tracker was used to more comprehensively document observable behaviours during
the MET is provided. This approach to analyzing behaviours in the MET is unique and Chapter
3 details the entire process. Chapter 4 is written in a format to be submitted for publication to
Neurorehabilitation and Neural Repair. The chapter consists of the overall research component
of the thesis and its findings. The final chapter (Chapter 5) is a general discussion that brings
together the entire thesis and integrates the findings reported in Chapter 4 with the background
literature provided in Chapter 2.
3
Chapter 2 Literature Review
Introduction
This chapter reviews the literature regarding executive dysfunction within the stroke population.
A review of the literature was performed using PubMed and PsycINFO between 1990 and 2011
with keywords related to executive function. These consisted of executive function, executive
dysfunction, neuropsychological test, cognition, prefrontal cortex, activities of daily living
(ADLs), assessment, stroke, and acquired brain injury. This chapter reviews the definitions of
executive functions and the relevant theories of executive function. A real world environment is
necessary when assessing executive function. Hence, ecological validity of both traditional and
real world measurements of executive functions are also discussed.
2.1 Description, Definitions, and Neuroanatomy of Executive Functions
Executive functions (EF) are those higher-order cognitive processes that enable problem solving,
information processing, decision making and the formulation of goals in daily life (Jurado &
Rosselli, 2007; Levine, Turner, & Stuss, 2008). In general, the term EF refers to a group of
cognitive abilities which include: attention, planning, reasoning, monitoring, complex problem
solving, verbal reasoning, decision making, inhibition of irrelevant information, social
functioning and the ability to deal with novel situations (Alexander & Stuss, 2003; Bryan &
Luszcz, 2000; Keil & Kaszniak, 2002; Levine et al., 2008). Together, these skills allow one to
plan, initiate and complete an action, while monitoring one‘s own behaviour and the
environment.
Most theorists emphasize the distinction between routine and non-routine behaviours when
attempting to describe the role of EF (Gilbert & Burgess, 2008). Routine processing refers to
automatic, well rehearsed operations that an individual is able to use in familiar situations
(Gilbert & Burgess, 2008), while non-routine processing is important in novel and varying
situations requiring sustained attention (e.g. distractions) or where stimulus-response
relationships are unclear (Gilbert & Burgess, 2008). The term EF has become synonymous with
non-routine processing (Gilbert & Burgess, 2008) and EF are considered to exert higher-level
4
modulation on lower-level routine processing in different situations. This allows an individual to
behave flexibly while taking into account environmental information and past consequences in
familiar and unfamiliar situations.
Behavioural and neuroimaging studies support the notion that EF are governed, at least in part,
by different structures in the frontal lobes (Gilbert & Burgess, 2008). This leads to a common
problem where many researchers use the term ―frontal functions‖ when describing EF even if
they are not examining the anatomy in particular (Stuss & Alexander, 2000). In addition, terms
such as ―dysexecutive syndrome‖ (Stuss, 2007) and ―dysexecutive control‖ (Stuss & Alexander,
2000) are also used interchangeably. However, this thesis will only use the term EF as described
by Stuss (2006) when discussing these high-order processes.
There is considerable debate over whether EF are discrete, fractionable processes governed by
different regions in the frontal lobes or if they are a single, unitary function (Jurado & Rosselli,
2007; Stuss & Alexander, 2000). Stuss (2007) argues in favour of the former based on empirical
evidence and divides EF into four different functional categories which are characterized by a
grouping of similar behaviours and anatomical structure (Cicerone, Levin, Malec, Stuss, &
Whyte, 2006). These include: (1) executive cognitive functions, (2) behavioural-emotional self-
regulatory functions, (3) energization regulating functions, and (4) metacognitive processes
(Stuss, 2007). Others try to define executive functioning from a unitary perspective in which all
areas of executive functioning can be explained by one single underlying ability such as
behavioural inhibition (Barkley, 1997; Duncan & Miller, 2002; Jurado & Rosselli, 2007).
2.2 Executive Dysfunction: Prevalence and Impact on Everyday Life
Executive dysfunction is one of the most critical and prevalent problems in the acquired brain
injury (ABI) population, which include people with stroke, traumatic brain injury (TBI) and
other forms of brain injury (Levine et al., 2008). Executive dysfunction can also be observed in
people with frontal dementia, multiple sclerosis, Alzheimer‘s disease, Parkinson‘s disease,
Huntington‘s disease and various other psychiatric disorders including schizophrenia and
depression (Alexander & Stuss, 2003; Levine et al., 2008; Stuss, 2007). Similarly, conditions
such as depression, anxiety and sleep deprivation, which all interact with the frontal lobes, can
also lead to deficits in executive functioning (Alexander & Stuss, 2003).
5
Executive dysfunction can have a devastating effect on everyday living, including the ability to
perform daily tasks at home and at work, and to maintain an independent life (Godbout, Grenier,
Braun, & Gagnon, 2005; Knight, Alderman, & Burgess, 2002). People with stroke and TBI have
a difficult time regulating their behaviours in response to the changes occurring in their
environments (Minassian, Perry, Carlson, Pelham, & DeFilippis, 2003). Some of the common
cognitive deficits associated with executive dysfunction include: lack of judgment, inability to
concentrate, and difficulties in organization and intellectual abilities (Alexander & Stuss, 2003;
Gillen, 2008). Specific behavioural problems include lack of flexibility, impulsivity,
distractibility and poor self-control (Alexander & Stuss, 2003). Some patients suffer from an
inability to self-monitor their behaviours despite appropriate feedback (Minassian et al., 2003),
while others have difficulties in sustaining and reinitiating past behaviours (Lezak, 2004).
Most executive impairments are mediated by the frontal lobes and are regulated by the lateral
prefrontal areas (Alexander & Stuss, 2003; Stuss, 2007). However, diffuse damage to posterior
regions and subcortical structures connected to the frontal cortex can cause similar deficits
(Elliot, 2003; Levine et al., 2008). Similarly, damage to cortical connections to basal ganglia,
cerebellar and thalamic areas may also result in executive impairments (Alexander & Stuss,
2003). Further, it is important to note that executive impairments do not only occur as a result of
extensive cortical damage and may even occur in the absence of cortical lesions due to the nature
of connectivity between frontal lobes and other regions in the brain (Alexander & Stuss, 2003).
2.3 Theories of Executive Functions
Theories related to EF provide a better understanding of the underlying systems involved in
executive functioning and offer guidance on the appropriate assessments that can be used with
individuals with executive dysfunction. Various theories have been proposed in order to better
understand EF. Each of them is useful, however none alone entirely explain these higher-order
cognitive processes. Moreover, although each of the theories is different from the other, they all
deal with the frontal lobes, especially the prefrontal cortex (Turner & Levine, 2004). Further, all
of the theories have been proposed following studies of adults with frontal damage (Chan, Shum,
Toulopoulou, & Chen, 2008; Turner & Levine, 2004). The following section will provide a
review of some of the more widely employed and discussed theories of EF.
6
2.3.1 Theory of Goal Neglect
Duncan and colleagues emphasize the importance of goals in human behaviour (Duncan, 1986,
1995; Duncan & Owen, 2000; Duncan et al., 2000). They note that human behaviour is goal-
oriented and is directed by goal lists or sub-goals that are created in response to environmental
and internal demands (Burgess & Robertson, 2002). Goal lists are formulated and stored in mind
by individuals so they can behave optimally in response to external stimuli (Chan et al., 2008). It
is assumed that actions are directed by goal lists, which are a sequence of task requirements that
must be accomplished in order to achieve a desired goal (Turner & Levine, 2004). These lists
provide a road map of what actions and operations are needed when a current situation does not
match the goal state. In such situations, these goal lists are pursued and a series of actions are
activated to resolve the discrepancy. When goals are changed or new goals are set, a new goal
list is selected and specific actions are carried out that will lead towards goal attainment (Chan et
al., 2008).
An important characteristic of goal-directed behaviour is that it seeks new actions to achieve task
completion if previously elected actions have failed (Burgess & Robertson, 2002). According to
Duncan, Emslie, Williams, Johnson and Freer (1996), patients with damage to the frontal lobe
are usually disorganized and this can be illustrated by the fact that these patients are unable to
construct and use goals and goal lists in an effective manner. Duncan and associates (1996) use
the term goal neglect when describing this phenomenon in patients with damage to the frontal
lobe. These patients are able to understand and remember their goals but they seem to have
difficulty in maintaining the goals in working memory such that their actions become random
and they exhibit neglect of the intended goal. Duncan and colleagues (1996) found goal neglect
to be common in patients with frontal lesions in situations that were novel and those that had
multiple simultaneous requirements.
Duncan's theory of goal neglect emphasizes the importance of the prefrontal cortex (PFC) in goal
formulation, goal selection and goal monitoring (Turner & Levine, 2004). Duncan (1986) argues
that this is because the main function of the PFC is to organize and control actions in accordance
with desired goals. He also argues that it is common to observe a discrepancy between desired
goals and actions following frontal lobe lesions (Duncan, 1986). Duncan proposes that the PFC
has neural flexibility, which allows it to be recruited for different tasks and enables global
7
functioning of the PFC. However, the specific neurons, regions and structures of the PFC that
explain these processes have not been defined.
2.3.2 Adaptive Coding Model
The underlying idea behind the adaptive coding model (Duncan, 2001; Duncan and Miller, 2002)
is that since frontal regions are involved in many cognitive tasks, it is important to understand
the PFC as a global, adaptive coder. It suggested that the PFC controls many processes ranging
from working memory and attention to cognitive control. Duncan and Miller (2002) argued that
the neurons of the PFC are highly adaptable in nature and can meet various task demands. They
also argue that the PFC has a role in selecting and integrating information relevant to the current
situation. Hence, the frontal lobes are able to take on actions as required, without having specific
regions mediate specific cognitive demands.
This adaptive ability of the frontal lobes has been demonstrated by functional imaging research.
Duncan & Owen (2000) compared the areas of the PFC involved in five different executive
cognitive demands. They found similar activation patterns regardless of the cognitive demands
studied, as well as overlap of the areas activated for each of the executive cognitive demands
(Duncan & Owen, 2000). In contrast, Cabeza and Nyberg (2000) demonstrated different regions
in the PFC to be related to different functions such as attention, working memory and language.
Also, Stuss (2006) questions the global, adaptive ability of the PFC as the theory is unable to
explain how the PFC regions select and abandon information. The theory also does not address
how the PFC integrates sensory, memory and task-relevant information and coordinates
subcortical and other regions (Wood and Grafman, 2003).
2.3.3 Supervisory Attentional System (SAS)
The SAS theory, developed by Norman and Shallice (1986) is one of the most influential
theories of frontal lobe functions. Norman and Shallice posited that the SAS is comprised of two
systems: contention scheduling and the supervisory system. These systems are important in
initiating, regulating and monitoring human actions and behaviours. The role of contention
scheduling is to control routine motor behaviours and cognitive operations in familiar situations
and prioritize the sequence of these behaviours (Burgess & Robertson, 2002; Chan et al., 2008;
Turner & Levine, 2004). The supervisory system is important in regulating non-routine and
8
novel behaviours where planning and decision making are involved (Chan et al., 2008). The
supervisory system is thought to be located in the PFC although it is unclear which regions in the
PFC are involved in the processes controlled by the supervisory system (Wood & Grafman,
2003). The location of the contention scheduler is also undefined (Wood & Grafman, 2003)
though Normal and Shallice (1986) argue that the contention scheduler may be located in the
basal ganglia and premotor cortex since both of these areas function as output targets for the
PFC.
That being said, the main difference between these two systems is in their ability to modulate
routine and non-routine behaviours. Research suggests that the PFC is indeed much more
implicated in non-routine than routine behaviours (Shallice, 2002). However, Wood & Grafman
(2003) argue that neuropsychological research has demonstrated that knowledge about routine
behaviour is also impaired following lesions in the PFC (Allain, Le Gall, Etcharry-Brouyx,
Aubin, and Emilie, 1999; Sirigu et al., 1996). They also argue that novel tasks activate anterior
PFC, while familiar tasks activate medial and posterior regions of the PFC (Koechilin, Corrado,
Pietrini, & Grafman, 2000). In addition, they note that it is unclear how the model represents the
integration of sensory input with memory information and the neuropsychological properties of
the PFC neurons (Wood & Grafman, 2003). Regardless of the lack of agreement in relation to
the specific neuroanatomical regions involved in the two systems, the SAS theory puts forth the
concept of multitasking performance in daily life (Burgess, 2000; Burgess, Veitch, Costello, &
Shallice, 2000) and the theory posits that an individual may have limited ability to respond to
multiple stimuli.
2.3.4 Fractionation of the Supervisory System
Stuss, Shallice, Alexander and Picton (1995) wanted to investigate whether there are different
areas in the PFC that mediate different cognitive processes. They used Norman and Shallice‘s
(1986) supervisory system as a starting point to examine if it could be further fractionated and
defined (Stuss et al., 1995; Stuss & Alexander, 2007). They recruited patients with single focal
frontal lesions to determine whether specific regions were important for different, specific
cognitive processes. The results led Stuss and his colleagues to classify four different properties
of PFC function: energization, behavioural self-regulation, metacognition, and EF (which
include task setting and monitoring) (Stuss & Alexander, 2007). Energization is the process that
9
allows an individual to initiate, concentrate and maintain a response, and lesions in the superior
medial regions of the PFC have shown to cause impairments in this ability (Stuss, 2006; Stuss &
Alexander, 2007). Damage to the left dorsolateral PFC has been associated with problems in task
setting, which refers to the ability to establish a stimulus-response relationship, while right lateral
lesions are related to impairments in monitoring, which is the ability to ensure quality by
checking performance and modifying actions and behaviours (Stuss, 2006; Stuss & Alexander,
2007).
It is important to note that although fractionation has been demonstrated among these processes,
this does not imply that they are independent. Instead, these processes work together with other
networks of the frontal and posterior regions in response to the complexity and duration of the
context (Stuss & Alexander, 2007; Vuilleumier & Driver, 2007). Specifically, these processes
work independently in simple automated tasks, but with increased task demands, different frontal
regions are involved to the point where all frontal regions may be employed (Stuss & Alexander,
2007). In contrast to Duncan‘s adaptive coding model where the PFC has unified control of the
EF with little specificity, Stuss and colleagues' investigation suggests fractionated supervisory
executive control.
2.3.5 Transcending the Default Mode
Mesulam (2002) emphasizes a central role of the PFC in conquering the hypothetical default
mode, which is a state directed by inflexible stimulus-response relationships and is unresponsive
to context and experience. In the default mode, actions are triggered via automatic reactions and
fulfill immediate satisfaction (Mesulam, 2002). Default actions are carried out automatically
without consideration of alternative responses and are hard-wired in nature (Mesulam, 2002). In
this state, the conscious is focused on here-and-now despite contextual feedback (Mesulam,
2002; Turner & Levine, 2004).
Mesulam (2002) explains that frontal lobe damage results in the implementation of the default
mode, however the main role of the PFC and EF is to suppress and minimize the effect of this
mode. This is carried out via five executive processes: (1) working memory, which is the ability
to actively hold and manipulate relevant information, (2) inhibition of distractibility, which is the
ability to ignore and suppress distractor stimuli or events, (3) novelty seeking, which is the
tendency to pursue novel and uncertain situations, (4) conditional mapping of emotional
10
significance, which is the ability to integrate emotion with action and experience, and, (5)
encoding of context and perspective, which refers to the ability to process the influence of the
environment, background and feedback from others (Mesulam, 2002). With the help of these
executive processes, the PFC is able to overcome the default mode to allow more context-driven
responses to occur (Mesulam, 2002; Turner & Levine, 2004).
2.3.6 Executive Knowledge (Structured Event Complexes)
Grafman (1995) explains the function of the PFC using neural representations of knowledge.
According to this framework, higher-order cognition is controlled by knowledge domains, which
are organized in a hierarchical manner in the PFC (Levine & Turner, 2004). A knowledge unit
refers to a set of events or actions that are linked sequentially (Grafmam, 1995; Turner & Levine,
2004). A series of knowledge units linked together temporally are structured event complexes
(SECs), which are encoded in sequence and represent morals, social customs, beliefs, event
themes and other related features (Turner & Levine, 2004; Wood & Grafman, 2003). These
SECs are stored independently and are retrieved in an episodic form. The SECs come into action
when an individual needs to perform a goal and remain activated until the goal is achieved. This
activation pattern of the SECs is consistent with the firing patterns of the PFC neurons (Wood &
Grafman, 2003).
This framework predicts that different aspects of SECs are stored in different regions of the PFC.
For instance, nonsocial features would be stored in the dorsolateral PFC whereas social aspects
of the event would be stored in the ventromedial PFC and would have strong connections with
posterior regions. This notion is also supported by research suggesting that damage to the
ventromedial PFC leads to impairments in social behaviour, while dorsolateral PFC damage
leads to problems in reflective behaviour (Grafman et al., 1996). The framework predicts that
damage to the PFC would lead to partial or incomplete retrieval of SECs which would in turn
lead to observable impairments in everyday life.
Wood and Grafman (2003) are critical of the theories mentioned above because they assume that
these theories take a processing approach, which entails that cognition in the PFC can be
explained on the basis of performance, without specifying how the information is represented in
the PFC in the first place. In contrast, their approach, which is representational in nature, is more
concerned with the form in which information is stored in the brain and the localization of
11
different characteristics of various stimuli. Although this framework is able to explain the ―what‖
in terms of neurophysiology, organization and connectivity patterns of the PFC, it fails to explain
―how‖ these processes are governed by the PFC (Turner & Levine, 2004).
Each of the theories and models mentioned above are valuable and it is important to understand
them before moving on to the next section on the assessments of EF, however it is interesting to
note that many of the traditional measures were developed before any of the theories were
created and some of the recent measures may also be lacking in a theoretical framework.
2.4 Assessment of Executive Functions
The EF have conventionally been evaluated using traditional neuropsychological measurements.
Although, neuropsychological assessment of executive functioning is critical (Marcotte, Scott,
Kamat, & Heaton, 2010), it is also essential to understand the everyday impact executive
dysfunction can have on an individual (Manchester, Priestley, & Howard, 2004). As a result, a
number of researchers have developed real-life assessments and performance-based
measurements which resemble the challenges and situations faced by people in everyday life.
This section provides an overview of both traditional and naturalistic assessments of EF. One of
the foci is how these measures describe and assess elements of performance.
2.4.1 Traditional Measurements of Executive Functions
In this thesis, the term ‗traditional measures of EF‘ refers to the routine pencil-and/or-paper tests,
such as the Wisconsin Card Sorting Test, that are typically administered in a clinical or
laboratory-type setting. According to Hughes and Graham (2002), one of the most common
problems with traditional measures of EF is the difficulty in differentiating between automatic
and controlled actions. When an individual attempts a novel task, the performance slowly
changes from being controlled to being automatic as the person grasps the task and draws on past
experience to complete it. However, because traditional measures are usually structured and
defined in nature, it is difficult to differentiate when the performance becomes automatic. A
related problem with traditional measures is that of novelty of stimuli (Jurado & Rosselli, 2007).
Most traditional measures of EF assess the ability to deal with new problems. However, the
problems are no longer novel after the first administration of the test and this may result in a
practice effect (Salthouse, Atkinson, & Berish, 2003). Nonetheless, many measures of EF are
12
traditional neuropsychological tests, which are highly structured in nature and are used because
they are thought to be sensitive to frontal lobe damage (Miyake, Friedman, Emerson, Witzki, &
Howerter, 2000). Three of the most commonly used are the Wisconsin Card Sorting Test
(WCST), the Trail Making Test (TMT), and the Stroop Color and Word Test (Stroop) (Rabin,
Barr, & Burton, 2005). The following section provides a brief description of these tests and the
processes hypothesized to be measured. The psychometrics of all of these tests are well-
established (Strauss, Sherman, & Spreen, 2006). This is followed by a critique of the traditional
measures of EF on the basis of ecological validity.
Table 2.1 Traditional Measurements of EF
The Wisconsin Card Sorting Test
The Trail Making Test
The Stroop
2.4.1.1 The Wisconsin Card Sorting Test (WCST)
The WCST is one of the most popular frontal lobe tests of executive functioning. In this test, the
participants are presented with four stimulus cards, the first has a red triangle, the second has two
green stars, the third has three yellow crosses and the fourth has four blue circles on them
(Strauss et al., 2006). The participants are then given 64 or 128 cards one by one, each of which
has one to four coloured shapes on it. Each card has designs similar to the stimulus cards varying
in terms of shape itself, the color of the shapes and the number of shapes depicted. The cards are
shown in a set order that is unknown by the participant (Keil & Kaszniak, 2002). Participants are
asked to match the cards to one of the four stimulus cards and determine the rule underlying the
pattern without any help from the examiner or the environment (Bryan & Luszcz, 2000). When
the participants match a card, they are informed whether it is right or wrong. However, they are
not informed when the examiner changes the underlying pattern. The participants‘ task is to use
the information provided as feedback to correctly identify the underlying pattern for as many of
the cards as possible. This requires the participants to form a cognitive set (Minassian et al.,
2003), and to be flexible and adjust to the changing patterns, and use previous patterns to guide
future responses.
Performance on the WCST is scored in a number of ways, however executive control is assessed
by the number of categories completed, which is the number of sequences of 10 consecutive
correct patterns achieved, and the number of perseverative errors performed (Strauss et al.,
13
2006). Perseverative errors are those that occur when participants continue to sort the cards
according to the principle of a previously incorrect sort (Bryan & Luszcz, 2000) and do not
change their responses even when the responses continue to be incorrect. These perseverative
errors allow the examiner to measure the patients‘ ability to monitor and pay close attention to
the feedback provided by the examiner.
The WCST has been validated as a measure of several components of EF (Keil & Kaszniak,
2002). However, its functional specificity may not be too strong since other processes need to be
intact in order for successful performance to occur on the WCST (Keil & Kaszniak, 2002). For
instance, the tasks on the WCST require numerous non-executive cognitive processes such as
basic visual processing, numerical ability, rule induction ability, speed processing, and any
significant deficit in one or more of these processes may affect WCST performance (Strauss et
al., 2006). This suggests that impaired task performance does not necessarily imply executive
dysfunction. In addition, many researchers and compendia of neuropsychological assessments
note that the WCST cannot be used on its own as a predictor of frontal focal lesions (Anderson,
Bigler, & Blatter, 1995; Demakis, 2003; Henry & Crawford, 2004a; Lezak, Howieson, & Loring,
2004; Strauss et al., 2006) since impaired performance on this test can be due to a variety of
reasons (Stuss et al., 2000). Moreover, different EF contribute differently to a variety of complex
executive tasks and simply relying on the WCST as a measure of executive functioning is not
adequate (Miyake, Emerson, & Friedman, 2000; Miyake et al., 2000).
The WCST appears to be sensitive to frontal damage (Keil & Kaszniak, 2002), however, in a
recent review Nyhus and Barceló (2009) revealed that many researchers have demonstrated that
damage to other regions such as temporal (Barceló, Escera, Corral, & Periañez, 2006;
Giovagnoli, 2001), parietal (Gonzalez-Hernandez et al., 2002; 2003; Lie, Specht, Marshall, &
Fink, 2006; Rogers, Andrews, Grasby, Brooks, & Robbins, 2000), subcortical (Monchi, Petrides,
Petre, Worsley, & Dagher, 2001; Mukhopadhyay et al., 2008; Rogers et al., 2000) as well as
hippocampal regions (Giovagnoli, 2001; Igararshi et al., 2002; Nagahma et al., 1997) may affect
WCST performance. Consequently, it should be viewed as a measure that requires a large
distributed neural network (Stuss et al., 2000). It is interesting to note that when the WCST was
developed by Esta Berg in 1948, only healthy participants were used, however it has now
become the leading sorting task associated with frontal lobe damage.
14
2.4.1.2 The Trails Making Test (TMT)
The TMT is another popular and frequently used neuropsychological assessment. It consists of
two parts. In Part A, the participants are required to sequentially connect the numbers 1-25 that
are placed randomly on a piece of paper (Jurado & Rosselli, 2007; Strauss et al., 2006). In Part
B, participants are required to alternate between numbers and letters, again working sequentially
(1 to A to 2 to B to 3 to C and so on) in order to create another trail (Jurado & Rosselli, 2007).
The time it takes to complete each part is recorded which includes time spent by the examiner
indicating any errors made by the participants and the time it takes participants to correct these
errors. Total time to complete Part B is considered to be a measure of executive functioning
(Chaytor, Schmitter-Edgecombe, & Burr, 2006). This is because the tasks are thought to require
specific cognitive abilities such as speed of processing, monitoring, inhibition and scanning
abilities (Chaytor et al., 2006; Jurado & Rosselli, 2007; Strauss et al., 2006). Part A is presumed
to measure visual search and motor skills, while Part B appears to measure higher level cognitive
abilities such as mental flexibility and divided attention (Bowie & Harvey, 2006; Strauss et al.,
2006).
Some disagree with the notion that the TMT, in particular the nature of switching of Part B to be
a sensitive measure of frontal lobe dysfunction (Stuss et al., 2002). Furthermore, both parts A
and B of the TMT measure different constructs of executive functioning, one researcher may use
Part B as an assessment of mental flexibility while another might categorize the test as a measure
of attention and may use it only to test for perceptual-motor speed (Chaytor & Schmitter-
Edgecombe, 2003). In addition, Lezak et al. (2004) noted the ambiguity in recording the time
taken by the examiner to point out mistakes and speed taken by the participant in correcting
them, which may lead to reduced reliability of the test. Strauss and colleagues (2006) mention
that errors on the TMT tend to be fairly uncommon in individuals with moderate to severe head
injury, which further questions the reliability of error scores. On a related note, practice effects
are also present over short retest intervals on the TMT (Strauss et al., 2006), which may hinder
its use if the examiner wants to repeat the assessment.
2.4.1.3 The Stroop Colour-Word Interference Test
The Stroop is another measure used to assess EF because of its sensitivity to determine
proneness to interference (Bryan & Luszcz, 2000). The test consists of three trials, each with a
15
time limit of 45 seconds (Strauss et al., 2006). In the first trial, participants are presented with a
Word Page consisting of a list of 100 color words (red, green, blue) written in black ink. The
participants are asked to read the color words (Chaytor et al., 2006; Strauss et al., 2006). In the
second trial, a Color Page with 100 Xs in either red, green or blue ink are given and participants
are asked to read the color of the word ‗X‘ is written in (Chaytor et al., 2006; Strauss et al.,
2006). The third trial includes a Color-Word Page with 100 words from the first trial written in
colors from the second trial (e.g., word 'red' is printed in green ink) and the participants are asked
to name the color the word is written in (Chaytor et al., 2006; Strauss et al., 2006). The test
produces three scores: (1) word-reading score from the first trial, (2) color-naming score from
the second trial and (3) color-word score from the third trial (Strauss et al., 2006). The
interference score is also determined for the third trial (Strauss et al., 2006). The examiner
measures interference by recording the time it takes the participant to read the color-word minus
one of the other scores from the other trials (Bryan & Luszcz, 2000). This interference score
indicates the time it takes to suppress or inhibit reading a word and the time it takes to name a
color, which is known as the Stroop effect (Strauss et al., 2006). This is considered a measure of
executive functioning for the Stroop. Inhibition, concentration, selective attention and cognitive
flexibility are other executive processes thought to be tapped by the Stroop (Bryan & Luszcz,
2000; Jurado & Rosselli, 2007; Strauss et al., 2006). Participants are slower at reading the color-
word in the third trial because of the inability to ignore and inhibit a habitual response of reading
the word rather than the name of the color used to write it.
Strauss and colleagues (2006) note that since the three trials in the Stroop are organized in a
congruent manner, it may reduce the involvement of working memory and allow the participants
to employ one strategy for the entire trial. This may make it easier for the participants to keep
one goal in mind as task demands stay the same in each trial. Another difficulty associated with
the Stroop is the fact that alternate versions of this test do not reveal the same Stroop effect
(Shilling, Chetwynd, & Rabbitt, 2002).This would make it hard for clinicians to compare results
if they had been using different versions of the test. In addition, although the Stroop appears to
be sensitive to the frontal lobe damage, many advise that other neural systems are involved
during task performance and it is important to use multiple assessments of EF as the Stroop only
taps into certain aspects of executive abilities (Boone et al., 1998; Keil & Kaszniak, 2002; Pineda
& Merchan, 2003). Furthermore, the interference score, which is determined using the third trial,
16
yields only marginal/acceptable reliability, hence this score should be supplemented with other
data (Strauss et al., 2006).
These commonly used traditional measures are extremely vital because their contribution to
research is enormous. However, it is important to assess their ecological validity as outlined in
the next section.
Critique of Traditional Measurements of Executive Functions: Ecological Validity
Ecological validity refers to whether the findings obtained in a testing environment can be
generalized to those occurring in a natural, real world setting such as home, work and the
community (Chaytor & Schmitter-Edgecombe, 2003). In other words, it refers to whether
performance on the test is related to performance in daily life (Dawson et al., 2009). Burgess and
colleagues (2006) presented a critique of traditional tests of EF and highlighted many of the key
points associated with the lack of ecological validity in these tests. First of all, they adopted
Kvavilashvili and Ellis‘s (2004) definition of ecological validity which refers to both the
representativeness of the test to a situation encountered outside of the testing arena and the
generalisability of test results to predict similar problems in related circumstances of everyday
life. They noted that some of the traditional measures were never created to test for significant
cognitive deficits and were a result of a variety of experimental investigations (Burgess et al.,
2006). As a result, performance on these tests tells us very little about how people deal with the
same cognitive deficits in their everyday life.
Another key argument presented by Burgess and associates (2006) is that the methods in which
traditional tests are implemented have very little in common with situations encountered in
everyday life. This poses a challenge to both the idea of representativeness and generalisability
of the test and its results as it is not clear if what is being measured is related to performance in
the real world and whether the results obtained can explain performance in other situations
(Burgess, Alderman, Evans, Emslie, & Wilson, 1998; Chan et al., 2008; Manchester et al., 2004).
Since the ecological validity of these tests is not strong, poor performance on these tests does not
predict how people will perform in their daily life. For example, the situation encountered in the
WCST of sorting cards and others are unlike everyday circumstances and are rarely if ever
applied in real world tasks (Burgess et al., 2006).
17
The third argument is that traditional tests such as the WCST and Stroop are basic, short and
simplified tasks (Burgess et al., 2006) that are to be completed one at a time. These tasks are also
highly structured and defined in nature. As a result, they do not resemble the spontaneous and
uncontrolled nature of real-life situations. The nature of the test environment also has an impact
on the participant‘s performance. In many assessments, the environment is kept quiet with no
distractions if possible and the examiner controls the initiation and completion of tasks (Chaytor
& Schmitter-Edgecombe, 2003). In addition, it is the examiner that provides the instructions to
complete the tasks and is usually supportive regardless of the outcome of the test (Chaytor &
Schmitter-Edgecombe, 2003). These types of restrictions are highly unlikely to occur in a real
world environment where situations are distracting and hardly encouraging. Therefore, an
environment consisting of these restrictions will tell us little about participants‘ performance in
an everyday situation (Chaytor & Schmitter-Edgecombe, 2003). Hence, there are a whole
multitude of executive processes that these tests are unable to measure.
Kingstone and colleagues (2005) whose research is grounded in the field of attention take this a
step further and argue that there are two main assumptions with traditional laboratory-based
research which present a great problem in the clinical world. The first is that it is assumed that
the processes and conditions in the lab are similar if not the same as the ones occurring in the
real world. This is comparable to what Burgess and associates (2006) have argued which
highlights the lack of representativeness of traditional measures. The second assumption is that
to attain maximum results, one should minimize variability in a situation. However, there are
many variables occurring in naturalistic situations and it is important to take into account all of
them in order to understand and make inferences.
Chan and associates (2008) expand further to identify the reason behind the weak
generalisability of traditional tests to everyday circumstances. Since these assessments were
developed for basic experimental brain research (Burgess et al., 2006), they are only successful
in measuring at the impairment level (e.g. problems in attention) and fail to grasp the
complicated nature of responses that are required to carry out the many multistep tasks in daily
life (Chan et al., 2008; Lewis, Babbage, & Leathem, 2011). This may explain the reason why
many patients with ABI are able to do relatively well on these traditional neuropsychological
tests but still manifest executive impairments (Stuss, Floden, Alexander, Levine, & Katz, 2001).
18
Spooner and Pachana (2006) also note that there is a general problem in ecological validity
research in which certain traditional tests are believed to measure specific constructs such as
working memory even if they do not do so. For instance, the varying ways of defining executive
functioning (see section 2.1 in this chapter) make it harder to categorize certain
neuropsychological tests (Chaytor & Schmitter-Edgecombe, 2003). This also leads to difficulty
in selecting an appropriate test to measure a specific construct. For example, certain tests are
labelled with a specific construct that they measure only on the basis of face validity and may
have not been tested using construct validity. Even if the test is measuring the specified
construct, this may not be enough to explain the clinical findings at the individual level (Burgess
et al., 2006) because these inferences are based on theoretical constructs and are unable to
explain failure in performance.
Ecological validity is important when studying real world behaviours and traditional measures
are lacking in this. As a result, it is essential to take a closer look at the naturalistic, real world
assessments in the next section of the chapter.
2.4.2 Real World Measurements of Executive Functions
As mentioned above, traditional neuropsychological tests can measure isolated cognitive and
executive processes but are less effective in predicting everyday life performance following
executive dysfunction (Burgess et al., 2006). Daily life performance often requires multitasking
and the ability to generate strategies to deal with novel situations (Manchester et al., 2004). For
this reason, many have suggested the need for testing in real world environments using
naturalistic, real world assessments (Burgess et al., 2006, Godbout et al., 2005; Goverover et al.,
2005).
This section of the chapter provides a review of the naturalistic assessments and questionnaires
published in peer-reviewed journals, which are specific to EF and have been validated against
other neuropsychological measures of EF. Specific keywords such as: naturalistic and real world
were used in order to find these measures. The questionnaires included measure the impact of
executive dysfunction on leading an independent life, while the performance-based assessments
measure one or more everyday activities such as cooking or shopping that require executive
abilities. The psychometrics of these measures have not been reviewed, rather the following
section discusses the executive processes hypothesized to be tapped. Assessments that only
19
measure nonexecutive components like the State-Trait Anxiety Inventory (Spielberger, Gorsuch,
& Lushene, 1970), and those that use virtual reality or simulation such as the Hotel Test (Manly,
Hawkins, Evans, Woldt, & Robertson, 2002), Executive Secretarial Task (Lamberts, Evans, &
Spikman, 2010), Six Elements Test (Shallice & Burgess, 1991), Behavioural Assessment of the
Dysexecutive Syndrome (BADS) (Wilson, Alderman, Burgess, Emslie. & Evans, 1996) and
Iowa Gambling Task (Bechara, Damásio, Damásio, & Anderson, 1994) are excluded. The
review of the assessments is followed by an overview of the ecological validity of real world
measures.
Table 2.2 Real World Measurements of EF
Questionnaires
The Behaviour Rating Inventory of Executive Function - Adult Version
The Dysexecutive Questionnaire
The Profiles of the Executive Control System
Performance-based
Assessments
The Kitchen Task Assessment
The Rabideau Kitchen Evaluation-Revised
The Cooking Task
The Executive Function Performance Test
The Executive Function Route-Finding Task
The Instrumental Activities of Daily Living Profile
The Multiple Errands Test
Questionnaires
2.4.2.1 Behaviour Rating Inventory of Executive Function - Adult Version (BRIEF-A)
The BRIEF-A, a 75-item questionnaire for adults based on the original BRIEF developed for
children and adolescents, measures EF in daily life over the previous month (Roth, Isquith, &
Gioia, 2005). The BRIEF-A includes two versions: self-report and an informant report (Roth et
al., 2005) in which each item is scored on a scale of 1 to 5 and higher scores reflect greater
difficulty experienced by the patient. The informant report can be used by itself if the patient is
unable to complete the self-report. The BRIEF-A yields an overall score called the global
executive composite, which consists of two separate indexes called the Behavioural Regulation
Index (BRI) and Metacognition Index (MI). The BRI is composed of four scales: (1) Inhibit,
which refers to the ability to resist an impulse, (2) Shift, which refers to the ability to be flexible
and switch attention, (3) Emotional Control, refers to modulation of emotional responses and (4)
Self-monitor, which refers to checking one‘s own actions during and after goal attainment. The
MI Index is comprised of five scales: (1) Initiate, which refers to the ability to begin the task, (2)
Working Memory, which refers to the process of maintaining relevant information in mind in
20
order to complete a task, (3) Plan, refers to the ability to set goals and develop a strategy to
complete the task, (4) Task Monitor, which refers to checking and ensuring the task is performed
in an organized manner and leads to completion, and (5) Organization of Materials, refers to the
ability to establish and maintain all the necessary materials required to achieve the task (Roth et
al., 2005).
The scores are calculated for each scale, indices and for the global summary composite. T-scores
are based on comparisons to a normative sample composed of 1050 self reports and 1200
informant reports.(Roth et al., 2005). Rabin and colleagues (2006) compared the BRIEF-A with
various neuropsychological tests of EF in patients with amnestic mild cognitive impairment and
significant cognitive complaints. They found a moderate inverse correlation between the self-
report BRI and the Weschler Memory Scale-III (WMS-III) Visual Reproduction II (Rabin et al.,
2006). They also found a significant correlation between the adjusted Geriatric Depression Scale
and both the MI and BRI for self-report version only (Rabin et al., 2006). However they failed to
report any strong relations with neuropsychological measures of EF. Rabin et al. (2006)
suggested that this finding may be due to the fact that the BRIEF-A is measuring different
aspects of EF than those tapped by performance-based neuropsychological tests.
In contrast, Garlinghouse, Roth, Isquith, Flashman and Saykin (2010) compared the subjective,
self-report of working memory scale of BRIEF-A in patients with schizophrenia with the Digit
Span subtest of the Wechsler Adult Intelligence Scale-III. They found that the patient group
reported poorer subjective working memory and performed worse on the Digit Span Backwards
than the comparison group (Garlinghouse et al., 2010). The authors, however, emphasize the
differences between the working memory demands being measured by the BRIEF-A and the
Digit Span. The BRIEF-A asks whether the patient had problems with different behaviours that
require working memory over the past month, while the Digit Span measures the ability to
maintain and manipulate digits in working memory (Garlinghouse et al., 2010). In addition, the
difference between the subjective nature of the self-report versus the objective features of the
Digit Span may also have an impact on the degree of correlation.
2.4.2.2 Dysexecutive Questionnaire (DEX)
The DEX provides an analysis of the impact of executive dysfunction in the real world (Gillen,
2008). It is a 20-item questionnaire that measures the range of impairments related to
21
dysexecutive syndrome (Wilson et al., 1998) such as problems in abstract thinking, decision
making and planning, confabulation, temporal sequencing, lack of insight, disinhibition and
perseveration (Malloy & Grace, 2005; Strauss et al., 2006). It also assesses personality-related
changes such as impulsivity, aggression, euphoria, apathy, lack of insight, distractibility and
unconcern for social rules (Malloy & Grace, 2005; Strauss et al., 2006). The questions on the
DEX tap into four areas: emotional or personality changes, motivational changes, behavioural
changes and cognitive changes (Strauss et al., 2006; Wilson et al., 1998). Each question is scored
on a 5-point Likert type scale (0 = never and 5 = very often) of problem severity. The DEX is
presented in two versions: one is completed by the participant and the other by a relative,
caregiver or a clinician who has close contact with the participant (Strauss et al., 2006; Wilson et
al., 1998).
Wilson et al. (1996) conducted a factor analysis on other‘s ratings on the DEX by caregivers and
found three factors related to the symptoms reported by caregivers. These factors included
behavioural, cognitive and emotional components (Wilson et al., 1996). In addition, Burgess and
colleagues (1998) also performed a factor analysis and stated five symptoms as reported by the
caregivers namely: inhibition, intentionality, executive memory, positive and negative affect.
They found that the first three factors were well correlated with executive tasks, whereas the
latter two factors had much weaker correlations (Burgess et al., 1998). They also noted that
compared to other tests, the DEX was able reflect the patients‘ lack of insight into their problems
as most patients rated themselves as having less severe, executive impairments than their
caregivers (Burgess et al., 1998).
Bogod, Mateer & MacDonald (2003) compared the DEX (other and self ratings) and Self-
Awareness of Deficits Interview (SADI) as independent measures of self-awareness with
measures of EF and IQ in participants with TBI and failed to support previous findings that the
DEX discrepancy score is associated with executive functioning as reported by Burgess et al.
(1998) and Wilson et al. (1996). They found only a marginal relationship between the DEX and
the SADI. Moreover, in comparison to the DEX, only the SADI had better correlations with
measures of EF. These conflicting results may in part depend on the rater as Bennett and
colleagues (2005) found that professionals provided more accurate assessment of executive
dysfunction in comparison to the ratings provided by caregivers. Nonetheless, the DEX is one of
the few rating scales able to measure self and others perceptions of executive behaviours.
22
2.4.2.3 Profiles of the Executive Control System (PRO-EX)
The PRO-EX is not a questionnaire but a rating scale that assesses executive functioning in
everyday situations (Braswell et al., 1992) on the basis of observation of activities over a time
period. The rating scale is completed by a caregiver, significant other or clinician and measures
current level of functioning in seven different component areas: goal selection, planning and
sequencing, initiation, execution, timesense, awareness of deficits, and self-monitoring (Braswell
et al., 1992). The PRO-EX assesses how impairments in these component areas impact everyday
activities (Proctor, Wilson, Sanchez, & Wesley, 2000). Goal selection refers to the ability to
create goals and have a sense of intention during goal setting. Planning and sequencing refer to
organization skills needed to form written or oral plans. Initiation refers to the ability to begin an
action independently or with physical prompts. Execution deals with the ability to carry out a
multistep action. Timesense refers to the ability to complete an action in a specific time period
and monitor time efficiently. Awareness of deficits assesses awareness of deficits post-injury and
the need to use compensatory strategies. Self-monitoring refers to the ability to evaluate and
modify behaviours as needed. These component areas are rated from 1-7, while self-monitoring
is rated on a scale from 1-6 with the highest possible score of 48 that can be achieved on the
PRO-EX (Braswell et al., 1992).
Proctor and colleagues (2000) investigated the relationship between EF and working memory in
eight adolescents with closed head injury. They used the PRO-EX to measure executive
dysfunction and the recognition memory task (RMT) to assess working memory and found a
strong positive correlation between the two measures. A positive relationship was found between
goal selection, planning and sequencing, awareness of deficits, self-monitoring and RMT scores.
They also found moderate positive correlations between initiation, execution, timesense and
RMT scores (Proctor et al., 2000). The PRO-EX was able to differentiate between the patient
group and the control group, however the RMT was not. The results of the RMT showed that
some patients were able to perform as well as matched controls. In contrast, when comparing the
patient group with the controls on the PRO-EX, the results indicated that the controls functioned
significantly different in their daily lives than did the patient group (Proctor et al., 2000). This is
an important finding because it indicates that the PRO-EX, which is a naturalistic questionnaire
is able to tap into and differentiate between executive abilities. The PRO-EX has recently been
23
suggested to be used with older adults with late-life mood and anxiety disorders that demonstrate
executive dysfunction (Mohlman, 2005).
Performance-based Assessments
2.4.2.4 Kitchen Task Assessment (KTA)
The KTA is a performance-based assessment of cognition and EF while performing a specific
instrumental activity of daily living (IADL) (Baum & Edwards, 1993). It involves a simple
cooking task in which participants are required to prepare a cooked pudding (Baum & Edwards,
1993). The KTA has three purposes: to determine which of the six EF components (mentioned
below) is affecting performance; to determine the individual‘s ability for independent
functioning; and to determine the level of assistance required to complete the task (Baum et al.,
1993). The KTA uses a structured cueing and scoring system to assess six executive
components: (1) initiation, which assesses if the individual is able to start the task, (2)
organization, which assesses if the individual is able to gather the necessary items needed to
perform the task, (3) performance of all steps, which assesses if the individual is able to execute
all the necessary steps to complete the task, (4) sequencing, which assesses the ability of the
individual to arrange the steps in a chronological manner, (5) judgment and safety, which
assesses the individual‘s ability to complete the task safely, and (6) task completion, which
measures the ability of the individual to know when the task is finished (Baum & Edwards,
1993). The participants are scored from 0 to 3 on the basis of the number and type of cues
needed to successfully complete the task. A total score of 18 points suggests the need for total
assistance. According to Baum and Edwards (1993), the information collected on this everyday
task and the different aspects of behaviour can be used by clinicians in training caregivers as well
as the participants.
One limitation of the KTA is that it only measures performance on one task, which is not enough
to judge a participant‘s overall everyday performance (Baum et al., 2008). In addition, the six
cognitive components do not represent a full range of EF. For example, the test does not include
planning, even though it is important in kitchen performance (Josman & Birnboim, 2001).
Moreover, Josman and Birnboim (2001) note that the sequencing component does not include an
external criterion, such as following a list of steps provided in which the ability to arrange and
follow steps can be tested (Josman & Birnboim, 2001).
24
2.4.2.5 The Rabideau Kitchen Evaluation-Revised (RKE-R)
Like the KTA, the RKE-R is a performance-based assessment, which requires preparation of a
meal. The difference between the two tests however, is that the KTA is more focused on the
execution of the cooking task, whereas the RKE-R primarily assesses the planning aspect of EF
(Josman & Birnboim, 2001). In this test, participants are required to prepare a cold sandwich
with two fillings and a hot instant beverage (Neistadt, 1992). The RKE-R assesses the
participants‘ status over time, their functional ability and the information gathered to determine
the amount of assistance needed to complete these tasks and treatment plans (Neistadt, 1992).
The two tasks involve 40 detailed component steps listed in the order they are usually performed.
The examiner evaluates these steps as part of a scoring system and measures the level of cueing
needed by the participants. Each step is scored between 0 and 3 for a maximum total of 120
points, where a score of zero signifies no assistance while a score of three indicates direct
intervention needed to complete the step.
One limitation of the RKE-R is that the scoring system is very long (Josman & Birnboim, 2001).
This is because the scoring system includes a combination of performance components as well as
performance steps. This makes the scoring method more complicated as some of the
performance steps are routine steps executed during task completion, whereas performance
components refer to more complex cognitive abilities, which are performed after understanding
written instructions (Josman & Birnboim, 2001). Moreover, like the KTA, the results on the
RKE-R cannot be generalized to the individual‘s overall EF as specific cognitive components
such as the ability to know when the task is complete is not assessed (Josman & Birnboim,
2001).
A recent study by Yantz, Johnson-Greene, Higginson and Emmerson (2010) compared the
results of the RKE-R with various other neuropsychological measures in participants with stroke.
They found significant correlations between RKE-R performance and almost all
neuropsychological measures such as the MMSE, the Brief Test of Attention, the Hopkins
Verbal Learning Test-Revised (HVLT-R) and the Rey Complex Figure Test. The HVLT-R
Delayed Recall score had the strongest relationships with the RKE-R performance. Yantz and
colleagues (2010) performed a post-hoc groupwise analysis and found that patients who had
25
more than five errors on the RKE-R had significantly worse performance on measures related to
attention, learning and memory, visuospatial organization and cognitive estimation.
2.4.2.6 The Cooking Task
The Cooking Task is another naturalistic, performance-based assessment that relies on kitchen
performance to assess underlying executive impairments. The information gathered from the test
can be used to better understand the actual impairments and discharge planning (Chevignard et
al., 2008). The test requires participants to make two separate dishes: a baked, chocolate cake
and an omelette for two people (Chevignard et al., 2008). This cooking assessment is different
from the ones mentioned earlier as the environment where the test takes place also includes
distracting utensils and ingredients to mimic what is normally present in a kitchen and ensures an
ill-structured environment (Chevignard et al., 2008). In addition, the test assesses multitasking
abilities of participants as they attempt to complete the assessment.
The Cooking Task is able to differentiate between brain injured and control participants
(Chevignard et al., 2008). Chevignard and colleagues (2008) demonstrated that brain injured
participants made significantly more errors and performed dangerous behaviours compared to
controls. Moreover, more than half of the brain injured group was not able to complete the test
(Chevignard et al., 2008). They performed a regression analysis and found that the best predictor
of the total number of errors in the Cooking Task was the scores of the Six Elements Test, the
number of perseverative errors in the WCST, and verbal fluency. Also, the cognition sub-score
of the DEX was significantly associated with the total number of errors in the cooking task
(Chevignard et al., 2008).
This test is unique as it involves a thorough classification and quantification of errors which
takes into account personal, contextual and environmentally-related errors (Chevignard et al.,
2008). In addition, the qualitative and descriptive analysis of errors provides specific information
that can be used in rehabilitation planning (Chevignard et al., 2008).
2.4.2.7 Executive Function Performance Test (EFPT)
The EFPT is another performance-based measure that assesses executive dysfunction by
incorporating four real world tasks including preparing a meal, using the telephone, paying bills
and managing one‘s medication (Baum et al., 2008). The EFPT uses the meal preparation task of
26
the KTA but replaces the task of preparing pudding with preparing oatmeal. The test utilizes the
same cueing and scoring system and assesses the same five EF components as the KTA (Baum et
al., 2008). The EFPT results in three scores: (1) the executive function component score, (2) the
task score and (3) the total score (Baum et al., 2008). This information reveals which EF
components are impaired, whether the participant is able to live independently and helps the
caregivers and family members of the participant to understand what kind of support and/or
reinforcement is needed to gain optimal performance at home (Baum et al., 2008).
The EFPT has several advantages. First of all, it is easily administered after brief training (Baum
et al., 2008; Wolf, Stift, Conor, Baum, & The Cognitive Rehabilitation Research Group, 2010).
Second, the authors suggest that it is able to isolate the specific executive process during the
completion of four real world activities (Baum et al., 2008; Wolf et al., 2010). Last but not least,
the EFPT allows the examiner to objectively measure the participant‘s activities rather than
relying on self-reports (Baum et al., 2008; Wolf et al., 2010). The EFPT has shown to
discriminate between healthy controls and participants with mild and moderate stroke (Baum et
al., 2008) and is able to detect executive abilities as early as one week following stroke (Wolf et
al., 2010). The EFPT has been used on other clinical populations such as multiple sclerosis
(Goverover et al., 2005) and schizophrenia and was able to differentiate between individuals that
differed on the degree of pathological signs and phases of the disease (Katz, Tadmor, Felzen, &
Hartman-Maeir, 2007).
One major limitation of the EFPT was found in the study with schizophrenic participants (Katz
et al., 2007). In this study, the initiation component of EF was not able to differentiate between
the groups (Katz et al., 2007). This is because the test includes structured activities and
instructions from the examiner, which limit the ability to measure initiation.
2.4.2.8 Executive Function Route-Finding Task (EFRT)
The EFRT is a naturalistic, performance-based assessment that measures executive abilities
related to route-finding or wayfinding (Boyd & Sautter, 1993). In this test, participants are asked
to find an unfamiliar place within a facility. Performance is rated on six different abilities: task
understanding, information seeking, retaining directions, error detection, error correction and on-
task behaviour (Boyd & Sautter, 1993). Task understanding assesses both the ability to
understand instructions and the ability to grasp the nature of an open-ended task with very few
27
guidelines provided. Information seeking involves the plans and strategies adopted by the
participants, the type of information sought and how the information is searched for. Retaining
directions assesses how the participants are able to retain important information received, such as
paraphrasing or note taking (Boyd & Sautter, 1993). Error detection measures if the participants
are able to note any discrepancy between directions and performance or surroundings (Boyd &
Sautter, 1993). Error correction assesses the level of independence demonstrated in
troubleshooting and adjusting behaviour. Finally, on-task behaviour measures the extent to
which the participants are focused on completing the task in the presence of various distractions.
These executive abilities are rated using a Likert-type scale ranging from 1 to 4, where one
represents extensive dependence on the examiner and four indicates independent completion of
the test (Boyd & Sautter, 1993).
Webber and Charlton (2001) used the EFRT in older adults to study the nature of difficulty in
wayfinding and concurrent validity of the EFRT. In their study, participants were asked to find
their way from a specified location to one of the two rooms in their residence. One room was a
familiar room (e.g. the dining room) while the other room was an unfamiliar room (e.g. the
cleaner‘s room) (Webber & Charlton, 2001). The results demonstrated a significant correlation
between the EFRT when finding an unfamiliar location and Rivermead Behavioural Memory
Test (RBMT) scores, which reflected deficits in everyday memory functioning (Webber &
Charlton, 2001). A lower, but significant relationship was also found between EFRT when
finding a familiar location and the RMBT scores (Webber & Charlton, 2001). However, no
significant relationship was found between the EFRT and the Wechsler Adult Intelligence Scale-
Revised and the MMSE scores. Webber and Charlton (2001) also reported that approximately
one-third of the participants had some difficulty in finding their way to a familiar room, while
half of the participants had difficulty finding the unfamiliar room. The key wayfinding
difficulties that were observed in this study were: the inability to maintain attention to the task,
checking and correcting errors, asking for help and remembering where to go (Webber &
Charlton, 2001). These results are consistent with what was demonstrated by Boyd and Sautter
(1993) in young adults with head injury.
Spikman, Deelman and van Zomeren (2000) investigated the presence and nature of executive
impairments after closed head injury in 51 participants compared to 45 healthy controls as they
completed various tests of EF, planning and attention. They also wanted to study whether these
28
tests were able to differentiate between patient and control groups. Spikman and colleagues
(2000) found that of all the EF, planning and attention tests1 they investigated, only the EFRT
showed a significant difference between the two groups. Spikman et al. (2000) concluded that
problems in executive functioning can be observed only in tasks that mimic everyday situations
and allow the participants to generate a strategy on their own. They also reported that the patient
group often needed more cues than the healthy control group to continue working on the task. In
addition, the patient group sought information less adequately and had more difficulty with
detecting and correcting errors in comparison to the control group (Spikman et al., 2000).
2.4.2.9 The Instrumental Activities of Daily Living (IADL) Profile
The IADL Profile extends the ADL Profile (Dutil, Bottari, Vanier, & Gaudreault, 2005; Dutil,
Forget, Vanier, & Gaudreault, 1990) which measures independence in 17 ADL and IADL tasks
ranging from grooming to making a budget. Due to the challenges faced by clinicians during task
analysis on the ADL profile, which resulted in low inter-rater reliability, the authors decided to
refine the tasks involved and improve the scoring system (Bottari, Dassa, Rainville & Dutil,
2009a; 2009b; 2010). This led to the development of the IADL Profile, which consists of eight
tasks, six of which are associated with the overall goal of receiving unexpected guests for a meal
(including putting on outdoor clothing, going to the grocery store, shopping for grocery items,
preparing a hot meal for guests, having a meal with guests, cleaning up after the meal) and two
of which are single complex tasks (including obtaining information and making a budget).
These tasks are measured via direct observation by occupational therapists in the participants‘
home and community environments and performance is analyzed according to four operations
related to EF: (1) goal formulation, which refers to the ability to find a solution to solve a
problem situation, (2) planning, which refers to the ability to generate a strategic plan of action
after reviewing all the alternatives, (3) carrying out the task, which refers to the ability to initiate
the plan and adapt to novel and ambiguous situations, and (4) attainment of the initial goal,
which refers to the ability to verify that the task was executed as planned and make the necessary
adjustments (Bottari et al., 2009a; 2010)
1 The Modified Tinker Toy Test, the Ecological Planning Task, the Spatial Learning Task with Self Set subgoals,
the Tower of London Test, the Reaction Time Task, the Modified PASAT, the Stroop test and the Trail Making
Test.
29
The IADL Profile measures both the type of difficulties encountered and the type and amount of
assistance needed to complete the tasks (Bottari et al., 2009b). The first six tasks are assessed on
all four operations of EF, while the final two tasks are measured using only three operations for a
total of 30 items (Bottari et al., 2009b). This is because the goal formulation operation is not
rated as it is the examiner who prepares these goals (Bottari et al., 2009b). Each operation is
scored using a five-level ordinal scale (dependence, verbal and physical assistance, verbal or
physical assistance, independence with difficulty, independence without difficulty) (Bottari et al.,
2009a; 2009b).
Bottari and colleagues (2009a) examined the relationship between three EF measures (The
Stroop, Tower of London test and WMS-III) and three indices of TBI severity (Glasgow Coma
Scale (GCS), Post-traumatic Amnesia (PTA) and coma length) with the IADL Profile in 100
patients with moderate/severe TBI. They reported all three indices of TBI to be significantly
correlated with the IADL Profile, with the highest correlation with coma length. In terms of the
EF measures, the IADL Profile was modestly correlated with the Working Memory Index
(WMI) of the WMS-III and the Tower of London test (Bottari et al., 2009a). The study however,
failed to observe any significant relationship with the interference score of the Stroop (Bottari et
al., 2009a). The authors argue this may have resulted because of the differences in the two tasks:
the structured inhibition task of the Stroop and the uncontrolled nature of the real world
environment where multiple distractors are present and the IADL Profile is administered (Bottari
et al., 2009a). In addition, Bottari et al. (2009a) also noted that they did not find any significant
relation between gender and any of the IADL Profile scores suggesting that the test is applicable
to both men and women even though the test mostly centers around a meal preparation task.
The authors point out several advantages of the IADL Profile compared to other real world
measures of EF. According to Bottari and colleagues (2010), the EFPT and the Multiple Errands
Test (MET) (discussed below) give more structured instructions compared to the instructions
provided in the IADL Profile, which simply states that the participants have to prepare a meal for
an unexpected guest. They are given $20 for their expenses, however they need to make their
own plans in terms of the meal they wish to prepare for their guest, the ingredients they will
need, which store they would need to go to purchase the items and how they would get there.
The authors argue that even though the goals of these real world measures are fairly similar, the
instructions given in the EFPT and the MET eliminate the likelihood of observing the
30
participant‘s ability to formulate goals and plans since the participants are asked to follow
instructions. In contrast, the unstructured nature of the instructions provided in the IADL Profile
allows a more detailed assessment of the participant‘s ability to perform in everyday tasks. In
addition, Bottari et al. (2010) argue that the MET assesses executive dysfunction on the basis of
the number and type of errors observed, however the IADL Profile measures errors, difficulties
as well as task related abilities to provide important information for treatment planning and
interventions.
2.4.2.10 The Multiple Errands Test (MET)
The MET is a naturalistic, performance-based assessment of EF which allows the examiner to
observe participants doing real-life tasks such as shopping and collecting information. It was
created by Shallice and Burgess in 1991, however since then, simplified hospital (Knight et al.,
2002) and shopping mall (Alderman, Burgess, Knight, & Henman, 2003) versions have been
created with a defined number of tasks and scoring system. The MET involves observing clients
moving around a real world environment (e.g. shopping mall or a hospital complex) to purchase
specific items (e.g. buy local stamps) and collect specific pieces of information (e.g. closing
hours of the library). The test consists of four sets of tasks (12 subtasks in total) which are
undertaken within the constraints of a set of rules (e.g. you may not spend more than $7.50) and
allows the examiner to observe errors made and strategies used. The test allows participants to
structure, plan, monitor and execute tasks efficiently. It also places multitasking demands on the
participants which is a unique feature of the MET (Burgess, 2000; Brugess et al., 2000; Shallice
& Burgess, 1991).
Several studies have reported that the MET is able to discriminate between participants with and
without brain injury (Alderman et al., 2003; Dawson et al., 2009; Rand, Rukan, Weiss, & Katz,
2008), and initial results have demonstrated that patients who did relatively well on traditional
neuropsychological tests of EF performed worse on the MET as compared with controls
(Shallice & Burgess, 1991). Significant correlations have been demonstrated with the DEX
(other ratings) (Dawson et al., 2009), the Assessment of Motor and Process Skills the Sickness of
Impact Profile (Dawson et al., 2005a) and the Zoo Map subtest of the BADS (Rand et al., 2008)
suggesting that the MET measures several aspects of executive functioning.
31
Andre, Anderson, Stuss and Dawson (2009) used a unique approach by assessing the strategies
used by participants with TBI and stroke, and matched healthy controls as they attempted to
complete the Baycrest version of the MET (BMET). The strategies were classified as the use of
print, personal or environmental resources and money management. They found that the total
number of strategies observed was not significantly different between cases and controls (Andre
et al., 2009). They also reported that participants with stroke used more environmental resources
compared to the participants with TBI (Andre et al., 2009). These findings are important because
they put forth the idea that compared to traditional measures of EF, different aspect of
performance on the MET (e.g. strategies used, errors committed and naturalistic behaviours
performed) can be further analyzed to understand the impact of executive dysfunction in real
world situations.
Alderman and associates (2003) proposed an interesting way to analyze errors in the brain-
injured and control participants which were committed during the completion of the MET. They
argued that it is not enough to measure the number of errors committed by patients and controls.
Instead, it is important to identify the qualitative differences in performance in both groups
(Alderman et al., 2003). This is because they found that participants in both groups committed
similar types of error such that both groups can be categorized as either rule breakers (for
breaking rules) or task failers (for failing to complete assigned tasks). They devised a more
sensitive scoring method in which errors were weighted based on their ―normality‖ (Shallice and
Evans, 1978). This way, normal errors or errors performed by 95 percent of controls were given
a score of one; errors committed by five percent of controls were given a score of two, and errors
unique to the brain-injured group were given a score of three (Alderman et al., 2003). This
proved to be a better scoring method and accounted for significant differences between the
groups. Dawson et al. (2009) also used the weighted scoring method and reported significant
differences between participants with stroke and their matched controls on the MET. They found
that patients with stroke committed more rule breaks and performed significantly worse on the
number of tasks completed, and patients with TBI scored significantly worse on the weighted
error score and a trend towards worse performance on tasks omitted and time to completion.
Similarly, Rand and associates (2008) also employed this method and found that participants
with stroke were more likely to make mistakes, which resulted in rule breaking and inability to
multitask during the completion of the MET.
32
Alderman and colleagues (2003) furthered the analysis by looking more closely into the
executive impairments in rule breakers vs. task failers by evaluating the responses of caregivers
on the DEX questionnaire. They compared five symptoms in the DEX questionnaire, namely
inhibition, intentionality, executive memory, positive affect and negative affect (Burgess et al.,
1998) with the two groups and found that rule breakers exhibited more executive memory
symptoms related to confabulation, inability in temporal sequencing and perseveration
(Alderman et al., 2003). In contrast, task failures showed more symptoms of negative affect such
as apathy and lack of emotion. In the context of the MET, the relation between rule breakers and
executive memory symptoms can be explained by the fact that these participants were either
more likely to fail to carry out the instructions they received or were unable to understand the
instructions (Alderman et al, 2003). In other words, rule breakers had problems in monitoring
their behaviours and as a result were not able to follow the rules correctly. In contrast, the
relationship between task failers and negative affect can be described on the basis of lack of
initiation in this group of participants; that they were unable to complete the task because they
failed to initiate those in the first place (Alderman et al., 2003).
This section provided an overview to the naturalistic assessments of EF. A great deal of variety
exists in these assessments, from questionnaires assessing everyday activities to tests measuring
the ability to prepare a meal and follow a shopping list. These measures may provide a great deal
of information in terms of the impact of executive dysfunction on everyday life, hence it is
important to discuss their ecological validity.
Ecological Validity of Read-World Measurements of Executive Functions
There are two approaches that need to be considered when it comes to ecological validity:
verisimilitude and veridicality. Verisimilitude refers to the degree of similarity in cognitive
demands between the testing arena and everyday environment (Chaytor & Schmitter-
Edgecombe, 2003; Franzen and Wilhelm, 1996). Thus, to achieve verisimilitude, tests must
include tasks that resemble situations in everyday life and there must be considerable
relationship between the complexity of the test and the behaviour that is being tested (Marcotte,
Scott, Kamat, & Heaton, 2010). This relationship allows the test to closely approximate the
participant‘s ability to perform those tasks in daily life and infer more conclusive results
(Spooner & Pachana, 2006). Chaytor and Schmitter-Edgecombe (2003) suggest that this
33
approach requires tests of EF to have more face validity than traditional measures and to be able
to simulate cognitive tasks in daily life. As a result, these tests are more likely to identify
challenges that people have in completing real world tasks, however may not discriminate brain-
injured population from controls or identify the etiology of brain injury. This view shifts away
dramatically from the traditional focus where tests have been used continuously to diagnose
brain damage. The idea behind the verisimilitude approach is that performance on these tests
would improve with increase in functional skills even if the brain damage remains (Chaytor &
Schmitter-Edgecombe, 2003).
Some standardized neuropsychological assessments have developed while keeping in mind the
verisimilitude approach to ecological validity. These tests include the Test of Everyday Attention
(Robertson, Ward, Ridgeway, & Nimmo-Smith, 1996), the RBMT (Wilson, Cockburn, &
Baddeley, 1985) and the BADS (Wilson et al., 1996). These tests attempt to measure everyday
skills in attention (e.g. searching a telephone directory), memory (e.g. remembering the location
of an item), and executive functioning (e.g. problem solving), respectively. Although this has not
been clinically proven, it can be argued that the MET demonstrates verisimilitude as the setting
encountered in the MET is more naturalistic and life-like. In addition, an individual is more
likely to go shopping for various items and collect specific pieces of information in their
everyday life. Moreover, the MET allows the individual the freedom to plan and execute tasks at
will with very little restriction on the time and manner chosen by the participant to complete the
tasks.
Veridicality refers to the degree to which results on an assessment are related to the scores on
other measures of everyday functioning (Franzen & Wilhelm, 1996). This approach involves the
use of statistical methods to understand the relationship between neuropsychological tests and
measures of real world functioning such as employment status, clinical and behavioural
observations. Tests like the EFPT, which the occupational therapists administer can be tested for
veridicality by determining the correlation of results with other measures of functioning such as
clinician‘s ratings to build treatment plans and guide intervention. The EFPT already
demonstrates significant correlations with standardized measures which assess working memory,
verbal fluency and attention (Baum et al., 2008). This test can also be tested for verisimilitude
because it uses real world tasks that are necessary to support independent living such as heating
up a light meal, managing medications, using the telephone and paying bills (Baum et al., 2008).
34
The assessments described in this chapter have been developed relatively recently and
demonstrate a greater degree of ecological validity than do traditional measures of EF. Yet, there
are still a number of outstanding issues. Most of the tests that demonstrate ecological validity
have not attained widespread use because they have only been used by limited number of
researchers or across a few neurological groups (Marcotte, et al., 2010), and have limited
psychometrics and lack of theoretical base, time to administer and ease of administration. In
addition, since many tests have been created by researchers in their own facility, they have only
been employed in one laboratory which poses a challenge to their use in general (Marcotte, et al.,
2010). Another obstacle in developing ecologically valid measure is that it needs to be
challenging enough so that it results in a normal distribution of function across individuals
(Marcotte, et al., 2010). This is important to avoid scenarios where everyone either achieves a
perfect score or fails the test. However, this reveals another critical problem for the assessments.
If the difficulty level of the assessment is too high, some tests may transform from a test-like
measure to a game-like measure and lose the real world flavour it is supposed to possess
(Marcotte, et al., 2010). This may particularly be the case with measures that are using virtual
reality to simulate everyday situations, where the equipment and the environment may feel like
an arcade game (Marcotte, et al., 2010).
Despite these challenges, Marcotte and Grant (2010) suggest that it is essential to develop and
implement new EF assessments that demonstrate better ecological. This is because we still lack
the ability to fully understand the impact of executive dysfunction on everyday activities. As
Burgess and colleagues (2006) indicate, it would be wise to develop measures whose design
begins with observations of real world behaviours rather than looking at limited behaviours
tested in the lab and inferring findings to real world scenarios. Marcotte and Grant (2010) also
suggest the use of discreet technologies such as video cameras to improve measurement of
behaviours occurring in natural settings. This will allow researchers to measure performance
under common demands with naturally occurring distractions.
Conclusion
This chapter reviewed the literature on executive dysfunction in stroke population and provided
the background on the definitions, relevant theories, and traditional and naturalistic assessments
35
of EF. It is important to have an understanding of these topics before moving to the next
chapters, which discuss my master‘s research.
36
Chapter 3 Describing the Methodology: Event Recording
Background
One difficulty in understanding the impact of executive dysfunction on behaviours in everyday
life is that environments have differential effects on behaviour. This is one reason why most
assessments occur in very constrained circumstances, as discussed in Chapter 2. Unfortunately,
as mentioned in Chapter 2, it is being recognized that the information gained form such
assessments provides limited information regarding performance in more naturalistic settings
(Burgess et al., 2006). Thus, to understand the impact of executive dysfunction fully, it is
important to assess performance in natural environments. This study (described in Chapter 4)
employed an event recorder to study the behaviours performed on the Baycrest version of the
MET (BMET) which is a naturalistic assessment that assesses participant's ability to purchase
specific items and obtain certain pieces of information within the restriction of a set of rules
(Dawson et al., 2009) to better understand the impact of executive dysfunction on the
performance of everyday life tasks in a natural setting.
An event recorder is a device on which the user can record specific information about an event.
There are a variety of different types of event recorders, ranging from recorders used in the
assessments of cardiac patients, which monitor a patient's heart rhythm, flight data recorders,
which record data on the operations of aircraft controls and performance and are involved in the
investigation of airplane accidents, to computer software that enables behaviour analysis. This
chapter focuses on describing the use of the latter.
Researchers use event recorders to observe and record multiple, mutually occurring, or
overlapping behaviours and events as they naturally occur. Event recorders can be used to record
behaviours from videotapes as well as from naturalistic observations. They allow coding of the
occurrence, frequency and duration of behaviours and events. These behaviours can then be
viewed and analyzed using a variety of graphic, descriptive, aggregate and statistical
representations.
A literature search was conducted on PsycINFO using the following keywords: event recorder,
event recording, event recorder software and behaviour. It was found that several researchers
37
have used event recorders in studies involving infants and children to measure infant feeding
behaviour (Taylor, Lujan, & Vázquez-Geffroy, 2000), mother-child conflict behaviours (Huang,
Teti, Caughy, Feldstein, & Genevro, 2007), in-home injuries (Morrongiello, Ondejko, &
Littlejohn, 2004a; 2004b), modification of classroom behaviours (Sibley, Abbot, and Cooper,
1969) and volunteer tutors‘ teaching performance when working with children with
developmental disabilities (Tindall & van der Mars, 2005). This methodology has also been used
to assess job environments of adolescents (Ruggiero & Steinberg, 1981), office seating
behaviours in adults (Dowell, Yuan, & Green, 2001) and aggressive behaviours in young adults
(Warden, Grasso, & Luyben, 2009). However, to the best our knowledge event recording has not
been used in relation to naturalistic assessment of executive dysfunction following stroke. This
technology was selected for use in this study because it would allow for a more detailed
exploration of multiple behaviours occurring concurrently.
3.1 Event Recorder: Behaviour Tracker
This study employed the event recorder called the Behaviour Tracker (Behaviour Tracker, 2003),
chosen for its low cost, compatibility with several Windows-based operating systems, and
satisfactory technical support to codify the behaviours of stroke participants and their matched
controls as they attempted to complete the Baycrest version of the MET (BMET).
This software consists of four different modes (see Appendix A for screenshots of the four
modes described below) and allows coding of the occurrence, frequency and duration of
behaviours. (1) The configuration mode allows for the creation of template files with predefined
keys on the keyboard that can be used to specify each behaviour to be coded. For example, when
participant goes to the gift shop, this behaviour is named as ‗P in GS,‘ assigned the letter A on
the keyboard and specified to have its duration coded. (2) The record mode allows tracking and
recording of the behaviours named in the configuration mode. It consists of start, stop and pause
buttons. This mode also permits modifiers to be added, that is a descriptor of the behaviour to
describe its uniqueness. For example, when the participant purchases Coke, this behaviour is
named as 'P takes Coke' and it can be modified to describe whether the participant purchases a
can or a bottle of Coke. (3) The editor mode allows viewing and modification of recorded
sessions. It displays both the original data collected and the edited data side by side to allow
editing of the frequency, duration and deletion of behaviours collected, as well as modification of
38
the descriptors added. For example, if one incidence of the behaviour ‗participant picks up card‘
was missed in the original recorded session, then another occurrence of this behaviour can be
added using this mode. (4) The viewer mode allows the data to be viewed in several formats such
as graphs, raw data, aggregate and detailed. It also permits exportation as a spreadsheet for
further analysis.
3.2 Procedure for Using Behaviour Tracker
3.2.1 Creating the codes
The codes for each configuration file were created by starting with the inefficiencies and partial
task failures identified in the score sheets employed by Dawson et al. (2009). These formed the
framework to which more behaviours were added. Additional behaviours were added in the
following ways:
1. Several strategies identified by Andre, Anderson, Stuss and Dawson (2009) were
incorporated, for example: 'participant looks at map', 'participant looks at task sheet',
'participant checks watch', 'participant looks at signage and surroundings', 'participant
looks at candy rack for the price of Mars bar', 'participant self-talks' and 'participant asks
staff for help'.
2. The theories of executive function were reviewed to assist in the identification of key
behaviours to be recorded. For instance, it was important to code overlapping and
concurrently occurring behaviours such as 'participant checking watch while walking' or
'participant stopping and marking task sheet' to note dual-task behaviours that were
expressed during the BMET. This also allowed us to look into the stopping and walking
behaviours of the participants because those with executive dysfunction may demonstrate
problems with dual-tasking and have to stop in order to perform certain behaviours such
as marking task sheet or map.
3. Stuss et al.‘s (1995) fractionation theory of attentional functions also played an important
role in shaping the final configuration files. Stuss and his colleagues (1995) identify task
setting and monitoring processes which may be affected following damage to different
areas of the frontal lobes. Task setting and monitoring behaviours specific to the BMET
that were incorporated included behaviours specific to each task such as: 'participant buys
39
four stamps' and 'participant puts stamp on urgent letter collected at information desk to
mail'. It also included those that were performed at any time the test such as: 'participant
checks off task sheet' and 'participant checks watch'.
4. Other behaviours were added after reading written descriptions of participants‘
performance prepared by the test administrators and watching some videotapes of
participants with stroke and controls to capture those behaviours not yet captured by the
coding system so that an exhaustive list of codes existed to document behaviours
common to all participants as well as those that were unique to a specific participant.
3.2.2 Structuring the Codes
In order to document all the behaviours performed, four separate configuration files were
created. The first three files included codes that were specific to certain locations and tasks. The
fourth file included codes that occurred at any time during the administration of the BMET. The
files were organized this way since a participant can only be in one location at one point in time.
This meant that only two of the four configuration files were needed to be coded for any event;
for example, if the participant was in the gift shop (GS), only the configuration file that had all
the behaviours occurring at the GS and the one that consisted of behaviours occurring at any time
during the BMET had to be accessed. In addition, as one configuration file contained a
maximum of 37 events (which corresponded to the 26 letters, space bar and 10 number keys on
the keyboard) and the number of behaviours needed for this study exceeded that amount, it was
deemed best to separate them in a way which made coding much easier and more accurate. This
organization also made the logistics of coding easier. It was possible to play a participant‘s
BMET video on one side of the computer monitor and have four separate files open next to one
another on the monitor; Behaviour Tracker allowed simultaneous access to multiple files with
one click of the mouse.
The files were named Meta, Metb, Metc and Metd. Meta focused on the tasks that were carried
out in the GS and at the lotto booth (see Appendix C). These included: (1) buy a birthday card,
(2) buy four stamps, (3) write down the price of Mars bar, and (4) write down the opening time
of the GS on Friday. Metb included the following tasks performed in the cafeteria, at the
Information Desk (ID) and at the mailbox, respectively: (5) buy Coke, (6) collect something for
the examiner and do what is necessary, and (7) mail something to Dr. Deirdre Dawson (see
40
Appendix D). Metc included the tasks performed near the phone, the library and the parrot cage
(PC), namely: (8) telephone Katherine and tell her your name, your location and the time, (9)
write down the closing time of the library, (10), meet examiner 10 minutes after starting the test
at the PC and tell time, and (11) tell the examiner when the test is finished (see Appendix E). The
task of writing down the number of entrances/exits on the main floor was not given a separate
code and was incorporated as part of the behaviours coded in the final configuration file. Metd
consisted of behaviours that occurred at any time during the entire test (see Appendix F).
Appendix G lists all the behaviours and events coded using the Behaviour Tracker.
All behaviours were coded as either frequency or duration events. Frequency behaviours were
coded in terms of how often they occurred and were typically of very brief duration, for example,
'participant looks at task sheet'. The modifier was used to further describe the behaviour, for
example, 'while walking'. Duration behaviours were coded in terms of how long the behaviour
lasted, for example, 'participant looks at map'. The software also allowed frequency counts of
duration behaviours. As shown in Appendix C, frequency behaviours were represented using a
lightning icon (blue arrow) and duration behaviours were displayed using a stop-watch icon (red
arrow) buttons on the Behaviour Tracker.
3.2.3 Coding Behaviors
The process of coding participants‘ BMET videos using the Behaviour Tracker is explained
below. Each participant‘s video was first watched to ensure a general understanding of it before
proceeding to the coding stage. This process took approximately half an hour to an hour and a
half depending on the length of the video and the time it took to take notes. During this stage,
written descriptions of participants' test performance by test administrators were also read. This
was followed by the coding procedure. During coding, all four configuration files were open
with the video being played on one side of the monitor. It was ensured that the start and stop
times for each configuration file and the video were the same to have accurate coding of how
long it took each participant to complete the BMET. Also, notes were taken if a behaviour or a
modifier was missed during coding. After coding was completed, the files were saved using the
BMET assigned identification and the configuration file name. The editor mode was then used to
make the necessary corrections to the coded files, based on the notes taken, and if necessary, the
video was watched again and followed using the codes in the editor mode. Once the editing and
41
reviewing was completed, the viewer mode was used to export both raw and aggregate data to a
spreadsheet for each of the four configuration files. These spreadsheets were then merged into
one file for further analysis. In general, it took three to four hours to code each video completely.
The process of becoming familiar with the codes took several hours of training as some
participants were very quick and a number of behaviours and events occurred simultaneously.
Also, every BMET video was different, which impeded keeping to a single pattern of coding. For
example, some participants were only able to complete half of the test and did not perform
certain behaviours; on the other hand, other participants made an attempt to complete every task
but were not able to finish them. Training in coding consisted of coding one video several times
and comparing results. This also helped improve the accuracy of coding and following a video at
the same time.
3.2.4 Reliability
To ensure within-rater reliability, one BMET video was randomly selected and re-coded and the
responses were identical. Reliability was further established by watching two BMET videos of
participants with one of the co-investigators and comparing results. In both instances, the results
were fairly similar. Four separate videos were also watched by a second rater and the results
were compared with the coding conducted by the first author (SA). The second rater, an upper
level MScOT student, had been trained on coding the BMET videos. Intraclass correlation
coefficients were calculated to evaluate inter-rater reliability (see section on Reliability in
Chapter 4 for inter-rater agreement for further details).
3.3 Other Application of the Codes and Conclusion
To the best of our knowledge, this is the first attempt to conduct a detailed analysis of BMET
behaviours using an event recorder. The coding described was developed for the naturalistic
observation of executive dysfunction in participants with stroke and matched controls as they
worked on the BMET in order to provide a detailed analysis of the behaviours performed. This
coding system or some modification of it is also being used to assess executive dysfunction in
participants with traumatic brain injury and their matched controls in another related study.
42
Chapter 4 Characterization of executive dysfunction in real world tasks: Analysis of behaviours performed during completion of the
Multiple Errands Test
(In preparation for submission to Neurorehabilitation and Neural Repair)
Authors: Arshad, S., Anderson, N., Polatajko, H., & Dawson, D.
Abstract
The purpose of this study was to understand the impact of executive dysfunction on everyday
activities in stroke participants. A classification system was developed to analyze behaviours
performed by 14 stroke participants and 12 healthy control participants matched for age,
education and gender on the Baycrest Multiple Errands Test (BMET), a task requiring
participants to purchase different items and gather certain information within the main floor of a
hospital. The study employed an event recorder to code the occurrences and frequencies of
behaviours as participants attempted to complete tasks. It was found that participants with stroke
performed significantly more task specific relevant inefficient behaviours (p < .05) and non-task
specific irrelevant behaviours (p < .10) than controls. In addition, participants with stroke were
significantly more likely to ask staff for directions to a location, and significantly less likely to
go to the 0.99¢ card rack first and use the map while walking in comparison to controls (p < .05).
These differences between stroke participants and controls indicate that future research should
account for a wide range of behaviours occurring in a test situation and highlight the importance
of assessment in a naturalistic setting.
Introduction
Executive dysfunction (ED) is thought to have a significant impact on an individual's ability to
perform everyday activities independently (Godbout, Grenier, Braun, & Gagnon, 2005; Royall et
al., 2007). This can occur following a variety of conditions leading to frontal lobe damage such
as after traumatic brain injury and stroke (Levine, Turner, & Stuss, 2008). Executive functions
are higher-order cognitive abilities that involve attention, planning, inhibition, reasoning,
decision making and problem solving (Alexander & Stuss, 2003; Bryan & Luszcz, 2000; Keil &
Kaszniak, 2002; Levine, Turner, & Stuss, 2008). They help an individual to formulate and
complete goal-directed behaviours and to make decisions in novel and complex situations in life
43
(Cicerone, Levin, Malec, Stuss, & Whyte, 2006). This complexity related to executive functions
makes it particularly difficult to assess.
Recently, researchers have critiqued traditional clinical and laboratory-based assessments of
executive functions. They argue that these assessments were not developed to assess ED and
were instead a result of basic experimental brain research for psychological investigations
(Burgess et al., 2006). These assessments measure function at the impairment level (e.g.,
problems in attention) and are relatively poor to predict the impact ED has on everyday
performance and during completion of complex real world tasks (Alderman, Burgess, Knight, &
Henman, 2003; Burgess et al., 2006; Chan, Shum, Toulopoulou, & Chen, 2008; Keil &
Kaszniak, 2002; Lewis, Babbage, & Leathem, 2011). In addition, Burgess et al. argue that
traditional assessments are usually highly structured in nature and are not representative of the
situations encountered in the real world. For example, it is difficult to infer how sorting cards in
the Wisconsin card sorting test is related to everyday situations and what circumstances in daily
life would require the abilities measured by the Wisconsin card sorting test (Burgess et al.,
2006). According to Burgess et al., assessing executive functions in real world settings would
provide a more accurate representation of the participant's impairments.
In response to the growing need for performance-based naturalistic assessments, a number of
tests have been developed to assess ED in the real world behaviour. The Multiple Errands Test
(MET) is one such measure. Administered in a real world setting (e.g. hospital complex,
shopping mall), it requires participants to complete everyday tasks (e.g. buy a birthday card) and
collect specific pieces of information (e.g. closing time of library) within the constraints of a set
of rules (e.g. you should not enter hospital treatment areas). Thus, one is able to learn about the
impact of ED in everyday life. This test was originally developed by Shallice and Burgess (1991)
who demonstrated that patients who performed relatively well on traditional neuropsychological
assessments measuring language, memory, perception and executive functions performed worse
than healthy controls on the MET in terms of errors committed, inefficiencies and abnormal
social behaviours. Shallice and Burgess (1991) argued that assessments such as the MET that
require participants to plan and perform multiple tasks over extended periods of time, without
consistent feedback from the examiner, reveal the impact of ED better than do traditional
neuropsychological tests of ED. In relation to the International Classification of Functioning,
Disability and Health‘s (ICF) framework of human functioning (World Health Organization,
44
2001), the MET measures functional performance in a real world environment and assesses at
the activity (previously known as disability) level (Chan et al., 2008). According to Chan et al.
(2008), the MET assesses planning and strategy allocation abilities.
To date, research on the MET has focused on understanding performance errors made by
participants in terms of (a) inefficiencies, where a more productive strategy could have been
used, (b) rule breaks, where a particular rule is broken, (c) interpretation failures, where task
instructions are misunderstood, (d) task failures, where any of the 12 tasks are not fully achieved
and (e) task omissions, where a particular task is excluded. Knight, Alderman and Burgess
(2002) reported that participants with severe acquired brain injury (traumatic brain injury, stroke,
tumors) committed significantly more rule breaks, had more errors, and completed significantly
fewer number of tasks than controls. In addition, Alderman and colleagues (2003) developed a
shopping mall version of the MET and found that brain-injured participants made three times
more total errors, broke more rules and were more likely to fail to complete tasks in comparison
to controls. They also developed a weighted scoring method to further analyze the results (see
Alderman et al. for more details). They reported two different patterns of failure on the MET in
the brain injured population: rule breakers and task failers. Rule breakers demonstrated problems
in monitoring their behaviours and were unable to understand and follow instructions, which
resulted in breaking task rules. In contrast, participants who were characterized as task failers
were unable to complete tasks because they failed to initiate tasks in the first place. Similarly,
Rand, Basha-Abu Rukran, Weiss, and Katz (2008), using a virtual version of the MET as well as
the MET in a real mall, found that participants with stroke made many types of errors in
planning, problem solving and multitasking, and were unaware of their errors and made more
social mistakes relative to controls. Dawson et al. (2009) developed a Baycrest version of the
MET (BMET) and also used the weighted scoring method. They reported that participants with
stroke committed more rule breaks and performed significantly worse on the number of tasks
completed. They also found that participants with TBI scored significantly worse on the
weighted error score and trended towards worse performance on tasks omitted and time to
completion.
The abovementioned studies provide an understanding of the performance errors in terms of
inefficiencies, interpretation failures, rule breaks, task omissions and problems in completing
tasks on the MET in the acquired brain injury population. Like the MET, other naturalistic
45
assessments of ED have also emphasized on studying performance errors such as the Cooking
Task (Chevignard et al., 2000) and the Rabideau Kitchen Evaluation-Revised (Neistadt, 1992),
both of which rely on kitchen performance. Understanding how well an individual performs on a
particular assessment is important and this is a traditional approach to scoring assessments of
cognition. However, we argue that to fully understand the impact of ED on participants'
performance in everyday situations, it is necessary to further examine the behaviours performed
while completing an assessment. This includes behaviours that are task specific as well as those
that the participants perform in relation to various environmental and contextual constraints.
This raises questions as to which behaviours should be scored, how to score these, and also how
to classify them. The ICF also emphasizes the importance of studying the environmental and
personal factors that may impact activity and participation in an individual (Vrankrijker, 2003).
To the best of our knowledge, thus far, no one has taken a theoretical approach to identifying and
categorizing behaviours, as we were unable to find relevant classification in the literature.
Hence, the main purpose of this exploratory study was to perform an in-depth analysis of
observable behaviours during the course of the MET. This would help investigate whether a
wide range of behaviours would also be important for successful test performance and allow
better discrimination between participants with stroke and healthy matched controls. The main
objective was to identify the behaviours performed by participants with stroke and healthy
controls (matched for age, education, gender) and determine differences between the two groups
in relation to their executive function.
Materials and Methods
Participants
This study was a secondary analysis of data collected for a previous study on BMET
performance (Dawson et al., 2009). Data consisted of videotapes of community dwelling adults
with stroke and healthy controls: 14 stroke survivors and 12 healthy controls matched for age,
gender and education (see Table 4.1 for a comparison of these variables). The participants with
stroke were divided into two groups on the basis of documented ED: 11 participants with stroke
(aged 47-77; mean=61.8; SD= ±11.5) and 3 participants with stroke-ED (aged 33-73;
mean=48.7; SD= ±21.4; see Table 4.1 for participant characteristics). The stroke-ED group
46
consisted of participants that demonstrated impairments in executive function, which was
defined as 1.5 standard deviations or more below age-corrected norms on two or more of the
following neuropsychological tests: FAS verbal fluency test (Benton, Hamsher, & Sivan, 1994;
Gladsjo, Shuman, Miller, & Heaton, 1999), Digits Backward (Wechsler, 1985), Trails B (Reitan
& Wolfson, 1985), and the Wisconsin card sorting test (Heaton, Chelune, Talley, Kay, & Curtiss,
1993). The participants in stroke-ED group were younger, more highly educated and had had
their strokes more recently compared to the participants in stroke group (p < 0.05). However,
there were no significant differences among the three groups on the basis of age, gender,
education, self-reported familiarity with the 1st floor of Baycrest ratings, the number of times
visiting Baycrest, and the number of rules remembered with and without cues (p > 0.20).
Table 4.1 Participant Characteristics
Total Participants
with Stroke (n=14)
Participants with Stroke (n=14) Total Controls
(n=12) Stroke-ED (n=3) Stroke (n=11)
Age (y) 59.0 ± 14.2 (33-80) 48.7 ± 21.4 (33-73) 61.8 ± 11.5 (47-77) 56.9 ± 16.5 (27-81)
Education (y) 15.1 ± 3.3 (7-19) 18 ± 1.0 (17-19) 14.3 ± 3.2 (7-18) 15.7 ± 3.5 (10-23)
Number (males:females) 8:6 3:0 5:6 7:5
Years post-Stroke 8.6 ± 6.0 (0.4-19.0) 2.7 ± 2.1 (0.5-4.7) 10.6 ± 5.9 (0.4-19.8) n/a
Familiarity with 1st
floor
(rating scale 1-10) 4.0 ± 3.3 (1-10) 5.7 ± 4.0 (1-8) 3.5 ± 3.1 (1-10) 3.3 ± 1.4 (1-6)
Number of times been to
Baycrest 1.7 ± 3.6 (0-10) 0.7 ± 1.2 (0-2) 2.0 ± 2.0 (0-10) 1.6 ± 1.4 (0-5)
Number of rules
remembered without cue 9.6 ± 2.4 (4-11) 9.0 ± 3.5 (5-11) 9.8 ± 2.2 (4-11) 10.4 ± 1.2 (7-11)
Number of rules
remembered with cue 10.1 ± 1.7 (6-11) 9.7 ± 2.3 (7-11) 10.3 ± 1.7 (6-11) 10.8 ± 0.6 (9-11)
NOTE: Values are Mean ± SD (range) or as otherwise indicated
Participants with stroke were recruited through local community agencies or from a list of
participants who had given consent to take part in future studies. They were included in the study
if they met the following inclusion criteria: (a) a minimum of 3 months post-injury, (b) were at
least 18 years of age or older, (c) were able to read, understand and speak English and (d) were
able to walk independently for at least half an hour. During the screening process, those who
scored above the cut-off of 16 on the Centre for Epidemiological Studies Depression Scale
(Radloff, 1977) were not included in the study. Also, those with conditions such as seizures or
leukemia were excluded from the study. Severity of stroke was difficult to determine for every
participant because of the time that had elapsed since the events. We were unable to obtain
47
health records and medical information on 5 out of 14 stroke survivors. Participants in both
stroke and stroke-ED groups had good language abilities. However, two participants in the stroke
group and two in the stroke-ED group had hemiparesis and walked with the help of a cane.
The twelve controls were recruited through friends and family members of the participants and
from the Baycrest volunteer pool and matched individually for gender, age (±5 years) and
education (±5 years). They met the same inclusion criteria as participants with stroke with the
exception of post-injury criterion and had to have a Mini-Mental Status Examination (MMSE)
score within the normative range based on age and education. All control participants also went
through neuropsychological assessment consisting of tests of attention, executive function,
memory, visuo-perception and visuo-constructional abilities and controls were required to be
within 2 standard deviations of age and education norms on each of these neuropsychological
tests (see Dawson et al., 2009 for more details on the neuropsychological assessment).
The study was conducted in accordance with human ethics standards and received ethics
approval from the joint Baycrest/University of Toronto Scientific and Ethics Review Committee.
All of the participants provided informed, written consent to participate in the study.
The BMET
The BMET, a version of the MET, was developed for use on the first floor of the Baycrest
Centre in Toronto, Canada. In this test, participants are required to complete 12 everyday tasks
while observing a set of 8 rules (see Appendix B and Dawson et al., 2009 for a list of tasks and
rules). The test required participants to access different areas of the first floor while using a map,
namely the gift shop, mailbox, information desk, cafeteria, parrot cage and library. Participants
were give a clipboard with the map of the first floor of Baycrest and task sheet listing the 12
tasks, which included buying and collecting items such as stamps, obtaining information such as
opening time of the gift shop, mailing something, meeting the examiner at a specific time and
location and telling the examiner when they have completed the test. Participants were also
provided with a watch, a pen, a bag to store their collected items, and a ten dollar bill for their
purchases.
A pre-test session was conducted with each participant to familiarize them with the tasks, rules,
and expectations, and also to answer any questions they had before starting the test. During this
48
session, participants were asked to memorize the 8 rules (three of which have two subparts for a
total of 11 rules). Participants were asked to freely recall each rule and were cued if they could
not do this. Both of these scores are shown in table 4.1. The test took about 60 minutes and was
videotaped and scripted to allow for scoring.
Coding procedure
Creating codes
The 26 BMET videotapes of were viewed and coded by the first author (SA) using event
recording software Behaviour Tracker version 1.5 (Behaviour Tracker, 2003). Behaviour Tracker
is an inexpensive software compatible to Windows-based operating systems, which allows
coding of the occurrence and frequency of multiple, concurrently occurring behaviours
performed by participants with stroke, stroke-ED and controls as they worked on the BMET (see
Chapter 3 for more detail on event recording).
A list of behaviours was identified and 66 separate codes for the Behaviour Tracker were created
to incorporate these behaviours. Behaviours were identified in the following manner:
1. Incorporating those previously documented in the score sheets of the BMET used by
Dawson et al. (2009);
2. By watching a subset of the videos (SA & DD) and determining additional behaviours
not previously coded, and by reading test administrators‘ written descriptions of
participants‘ test performance (SA). Several strategies recognized by Andre, Anderson,
Stuss and Dawson (2009) (who studied the same sample of participants) were also
utilized. Various rule breaks originally part of Dawson et al.'s (2009) study were also
included such as 'participant talks to the examiner' as well as examples of task specific
rule breaks such as 'participant goes to the second floor'. Interpretation failures that were
part of Dawson et al.'s (2009) work were also included such as 'participant does not tell
time at the parrot cage'. The Behaviour Tracker also allowed identification of dual-
tasking behaviours such as walking and looking at task sheet. This is important because
those with ED may have more problems with dual-tasking and have to stop to complete
certain behaviours.
49
Coding behaviours
To ensure all behaviours were captured, all videos were watched three times by the first author
(SA). Each participant's BMET written description was read and the video was watched prior to
coding to gain familiarity. The video was watched a second time and coding was performed. One
computer was used to code the video with both the video and the Behaviour Tracker software
running parallel to one another. Finally, each video was watched a third time to ensure
completeness and accuracy of coding.
Reliability of coding
Inter-rater reliability was evaluated by having a second rater code four of the 26 participant
videos using the Behaviour Tracker. The Intraclass Correlation Coefficients (ICCs) were
calculated using two-way random effects models, which makes the assumption that raters and
participants are random factors from a larger pool. Out of a total of 66 codes on the Behaviour
Tracker, the ICCs for 64 codes were substantial (> 0.6). Two codes had the ICC < 0.2, these
corresponded to two separate behaviours (see table 4.2 for inter-rater reliability results).
Table 4.2 Inter-rater Reliability
Number of Codes ICC Code Details
50 codes 1.0
14 codes 0.6-0.99
2 codes 0.1-0.19 - participant touches card
- examiner talks to participant
Results
Behaviour classification
Once all the videotapes were coded, behaviours were classified in the following way. As a first
step, behaviours were divided into two groups: (1) task specific behaviours, which referred to
those that were related to the completion of the 12 tasks on the BMET (for example 'participant
buys birthday card'), and (2) non-task specific behaviours, which referred to the behaviours that
occurred at any time during the test, but were not required or related to completing tasks on the
BMET, such as 'participant checks watch'. We organized the behaviours into these two groups to
analyze both task related as well as non-task related behaviours. We proposed that this would
provide an overall understanding of the impact of ED on an individual.
50
All the coded behaviours were then classified into one of the three categories within each group:
(1) relevant efficient behaviours, (2) relevant inefficient behaviours and (3) irrelevant
behaviours. Relevant behaviours were operationalized as those that were bearing upon and
pertinent to achieving a task. Irrelevant behaviours were those that were not applicable and/or
pertinent to achieving a task, which included apparent habitual behaviours such as 'licking a self-
adhesive stamp', as well as rule breaks such as 'talking to the examiner' and distracters such as
'watching television in the patient area'. Efficient behaviours were those that were performed
effectively and in the best possible manner to yield the most desirable result and inefficient
behaviours referred to those that were not performed effectively and/or did not yield the most
desirable result.
Table 4.3 provides an example of how certain behaviours were classified into their respective
categories. An example of a task specific relevant, efficient behaviour is 'writes down opening
time of gift shop on Friday'. This classification was made as this is a required task on the BMET.
An example of a task specific relevant, inefficient behaviour is 'asks for less than four stamps'.
One of the tasks on the BMET was to buy four local stamps in which it was found that some
participants asked for less than four stamps. This classification was made because this behaviour
would not yield the most desirable result since the participant asked for less than the amount
required. An example of task specific, irrelevant behaviour is 'reads message inside the card'.
This classification was made because this behaviour was not required to successfully complete
the task. It was categorized as an apparent habitual behaviour since people generally read
messages when they purchase cards. In order to complete the birthday card task efficiently, the
participant would have to go to the gift shop, go directly to the 0.99¢ card rack, find a birthday
card, and (if possible) check the back of the card to make sure the price is right and purchase the
card. However, reading the message, going through different cards and switching between card
racks would result in loss of time, when one of the rules provided on the task sheet stated that
they are to take as little time as possible to complete the exercise.
51
Table 4.3 Behaviour Classification
Relevant
Irrelevant Efficient Inefficient
Task Specific Behaviours Writes down opening time
of gift shop on Friday
Asks for less than four
stamps Reads message inside card
Non-task Specific
Behaviours Asks staff for help Asks non staff for help Asks examiner for help
We mention the example of 'asking for help' to highlight non-task specific behaviours. We
classified 'asking staff for help' as relevant efficient. Staff members included hospital staff such
as nurses, doctors, as well as cashiers and customer service representatives at the gift shop and
information desk. It would be relevant and efficient to ask staff members for help because they
would provide the most accurate and appropriate aid as they would be most aware of the
hospital. On the contrary, we classified 'asking non staff personnel (e.g. book vendors, charity
representatives) for help' as relevant inefficient because they may or may not know accurate
information that the participants were seeking. We classified 'asking help from the examiner' as
irrelevant because the rule list provided to the participants stated that speaking to the examiner is
not allowed unless it is part of the exercise.
A complete list of all the behaviours and their classification is provided in Appendix H. This
includes behaviours specific to the 12 tasks that were part of the MET as well as non-task
specific behaviours.
Related results
Descriptive analyses were conducted for each of the categories (task specific and non-task
specific: relevant efficient, relevant inefficient and irrelevant behaviours) to study the differences
in types and frequencies of behaviours for each of the three groups of participants. Differences
between mean behaviours of participants in each of the three groups (stroke-ED, stroke, controls)
for each of the six categories were analyzed using one-way analysis of variance (ANOVA). The
Tukey method was conducted as a post-hoc test to further analyze which groups were
significantly different than each other (Kafadar, 2003). The level of significance was set at ≤
0.10 due to the exploratory nature of the study. The effect size was calculated using eta squared,
a common effect size estimate for ANOVA in which 0.01 is a small effect, 0.09 is a medium
effect and 0.25 is a large effect (Levine & Hullett, 2002). The descriptive data were closely
52
examined for behaviours that had the most variation based on descriptive data (i.e. mean, SD).
All data were analyzed using the Version 17.0 of the SPSS.
The ANOVAs were conducted to compare the mean differences between stroke, stroke-ED and
control groups for all of the six behaviour categories. The main findings can be seen in Table 4.4.
In the task specific category, there was no difference in the number of relevant efficient
behaviours performed by stroke and stroke-ED groups compared to controls. The stroke group
demonstrated more irrelevant behaviours than controls, while the control group displayed more
irrelevant behaviours than the stroke-ED group. However, these differences were not statistically
significant and the effect size was also small for these two behaviour categories. The stroke
group performed more relevant inefficient behaviours than stroke-ED and control group, the
ANOVA analysis was significant F(2, 23) = 4.18, p < 0.03, the Tukey's test revealed that the
stroke group did this significantly more often than did the controls at the 0.05 level and the effect
size was large for this difference.
Table 4.4 Classification of participants' behaviours on the BMET. Differences in means, SD, range and p values
between stroke-ED, stroke and control groups for each behaviour category
Stroke-ED
(n=3)
Stroke
(n=11)
Controls
(n=12) p η
2
Task
Specific
Behaviours
Total Relevant
Efficient 20.67 ± 4.04 (17-25) 20.27 ± 5.83 (10-27) 21.00 ± 4.47 (14-28) 0.94 0.01
Total Relevant
Inefficient 4.33 ± 2.08 (2-6) 9.18 ± 5.53 (5-20) 4.42 ± 2.78 (2-11) *0.03 0.27
Total Irrelevant 1.33 ± 0.58 (1-2) 2.09 ± 2.34 (0-7) 1.58 ± 1.73 (0-6) 0.76 0.02
Non-task
Specific
Behaviours
Total Relevant
Efficient 35.33 ± 14.05 (22-50) 33.27 ± 22.33 (12-92) 35.67 ± 9.89 (23-58) 0.94 0.01
Total Relevant
Inefficient 31.33 ± 15.50 (20-49) 30.55 ± 15.40 (15-69) 25.67 ± 13.30 (12-52) 0.68 0.03
Total Irrelevant 8.33 ± 5.69 (2-13) 4.45 ± 4.13 (0-13) 3.08 ± 2.47 (0-8) *0.10 0.18
NOTE: Values are mean ± SD (range)
Significant at the p ≤ 0.10 for ANOVA analysis
η2 is the effect size
In the non-task specific category, there was no significant difference between the number of
relevant efficient behaviours performed by the stroke and stroke-ED groups compared to their
matched controls and the effect size was small. There were also no significant difference in the
number of relevant inefficient behaviours performed by stroke and stroke-ED participants
compare to controls and the effect size was small. The stroke-ED group performed more
irrelevant behaviours than the stroke and control groups, the ANOVA was significant F(2, 23) =
53
2.54, p < 0.10, the Tukey's test demonstrated that the stroke-ED group did this significantly more
often than the stroke group at the 0.05 level and the effect size was medium to large.
Further investigation of results
Three task specific and non-task specific behaviours that had the most variation were also
analyzed to further understand their significance when comparing the three groups (see Table 4.5
below for findings on specific behaviour analyses). These included: 'participant asks staff for
directions to a location', 'participants looks at/marks the map while walking' and 'participants
goes to the 0.99¢ card rack first'. The first two were compared on the basis of frequency of the
behaviour per participant in each group while the last compared the number of participants in
each group who performed the behaviour. One non-task specific behaviour was 'participant asks
staff for direction to a location' and the ANOVA was significant F(2, 23) = 4.11, p < 0.04, with
the Tukey's test showing that the stroke group did this significantly more often than did the
controls at the 0.05 level and the effect size demonstrated a large difference. Another non-task
specific behaviour was compared to understand the importance of dual-tasking abilities and was
labelled relevant efficient. This behaviour was 'walking and looking at/marking map'. The
ANOVA analysis was significant F(2, 23) = 3.69, p < 0.03, the Tukey's test demonstrated that
the stroke group did this significantly less often than did the controls at the 0.05 level and the
effect size revealed a large difference. Lastly, the number of participants who went to '0.99¢ card
rack first' were also compared. This behaviour was a task specific relevant efficient behaviour
related to buying a birthday card task. The ANOVA analysis was significant F(2, 23) = 4.28, p <
0.02, the Tukey's test revealed that the stroke group did this significantly less often than the
controls at the 0.05 level and the effect size was large for this difference.
Table 4.5 Specific behaviours findings. Differences in mean frequency, SD, range and p values between stroke-ED,
stroke and control groups on two behaviours
Specific Behaviours Stroke-ED
(n=3)
Stroke
(n=11)
Controls
(n=12) p η
2
Participant asks staff for
directions to a location 1.33 ± 2.31 (0-4) 3.27 ± 2.10 (0-7) 1.17 ± 1.40 (0-3) *0.04 0.26
Participant walking and
looks at/marks map 1.33 ± 2.31 (0-4) 0.73 ± 1.10 (0-3) 3.75 ± 3.67 (0-11) *0.03 0.24
NOTE: Values are mean ± SD (range)
Significant at the p ≤ 0.10 for ANOVA analysis
η2 is the effect size
54
It is interesting to note that compared to 9 out of 12 controls, only 2 out of 11 stroke participants
and 1 out of 3 participants in the stroke-ED group went to the 0.99¢ card rack first. Further, it
was observed that compared to 6 of the 12 control participants, 10 out of 11 stroke participants
and 1 of the 3 participants in the stroke-ED group asked staff for directions to a location. Also, it
was found that compared to 9 out of 12 controls, only 4 out of 11 stroke participants and 1
participant in the stroke-ED group were likely to attend to the map provided while walking
during the test.
'Stroke only' vs. 'control only' behaviours
Alderman et al. examined errors that were only observed in the brain injured group and those
that were committed only by the control group. This led to a more sensitive analysis of the errors
made on the MET. We also closely examined our data to isolate behaviours that were performed
only by participants with stroke, stroke-ED and those executed only by controls. We grouped
behaviours performed by participants in stroke and stroke-ED groups under 'stroke only'
behaviours. Table 4.6 lists stroke only and controls only behaviours. These behaviours will be
commented on in the discussion section.
Table 4.6 'Stroke only' vs. 'control only' behaviours
Behaviours demonstrated only by participants in the stroke group
Asks for less than four stamps
Asks for stamps at lotto booth
Puts stamp on urgent letter collected at information desk to mail
Licks self-adhesive stamp
Asks non-staff member for Mars bar price
Uses personal phone to call Katherine
Waits for Katherine to call back at payphone
Takes batteries from information desk
Behaviours demonstrated only by participants in the stroke-ED group
Asks for less than four stamps
Behaviours demonstrated only by participants in the control group
Buys Coke bottle instead of can
Picks up phone at information desk but does not use it
Tells examiner test is finished but continues to work on it
Discussion
The goal of this study was to investigate in more detail a wide range of behaviours performed by
participants with stroke (with and without neuropsychologically defined ED) and controls as they
worked on the BMET. To the best of our knowledge, this study was the first to employ an event
recorder to code behaviours to understand the impact of ED on everyday activities.
55
Researchers working with the MET have analyzed the errors committed by the participants
(tasks omitted, rules broken, tasks failed) as key markers of executive dysfunction. There have
also been some preliminary analyses of strategies used (Andre et al., 2009). We proposed that to
fully understand the impact of ED in everyday life, it is important to study the behaviours
observed during the completion of the MET, including those directly related to task completion
(e.g. 'participant buy 4 stamps'), as well as other behaviours that were less closely linked to a
specific task (e.g. 'participant performs casual self-talk') on the BMET. Our goal was to perform
an in-depth analysis of this fuller constellation of behaviours and to further classify them. As we
were unable to find relevant classification in the literature, we developed one based on the notion
that behaviours are undertaken as steps towards goal attainment (in this instance, completing the
tasks on the MET without breaking any rules). We first separated the behaviours into task
specific and non-task specific categories in which task specific behaviours were related to the
completion of the 12 tasks on the BMET and non-task specific behaviours, which occurred any
time during the test and were not related to a specific task. We further categorized the behaviours
as either relevant efficient, relevant inefficient or irrelevant. Relevant efficient behaviours were
those pertinent to achieving a task and yielded the most desirable result while relevant inefficient
behaviours were those that did not yield the most desirable result. In contrast, irrelevant
behaviours were those that were not related to achieving a task and included apparent habitual
behaviours, rule breaks and distracters.
The main findings of the study are as follows: the stroke group performed significantly more task
specific relevant inefficient behaviours than stroke-ED and control groups; participants in stroke
group performed significantly more non-task specific irrelevant behaviours than stroke-ED and
control groups; there were no significant differences in the number of task specific relevant
efficient and irrelevant behaviours performed between participants in stroke, stroke-ED and
control groups; and there were no significant differences between the three groups on the number
of non-task specific relevant efficient and relevant inefficient behaviours performed. Each of
these findings is discussed below.
There were no significant differences between the three groups on the number of relevant
efficient task specific and non-task specific behaviours. Compared to the previous studies on the
MET by Knight et al. and Alderman et al. who recruited participants with stroke that were either
inpatients or outpatients at their rehabilitation centers, this study included participants who were
56
living in the community. Smaller differences may have been obtained in the current study
compared to those of Knight et al. and Alderman et al. because the current individuals in stroke
and stroke-ED groups were well adapted to their communities. Our stroke group was on average
almost 10 years post-stroke and had been community-dwellers further adding to the likelihood
that they would have well-developed community living skills. Although the stroke-ED group had
had their stroke much more recently, they were both younger and had higher education. Each of
these factors might have benefited their performance.
There were no significant differences on task specific irrelevant behaviours and non-task specific
relevant inefficient behaviours between the stroke, stroke-ED and control groups. This may be
because we had a limited number of behaviours in these two categories that would discriminate:
there were eight behaviours in the task specific irrelevant category and five behaviours in the
non-task specific relevant inefficient category. Also, three out of eight task specific irrelevant
behaviours were committed by one participant only (i.e. participant with stroke).
Frequency of task specific relevant inefficient behaviours and non-task specific irrelevant
behaviours differed between stroke, stroke-ED and control groups. Participants with stroke
performed on average almost two times the number of relevant inefficient behaviours than
controls did. For example, to complete the birthday card task, several of these participants
purchased a card from the rack that contained regular priced cards as opposed to the ones from
the 0.99¢ card rack. This allowed them to complete the task, however not efficiently. Another
task on the BMET required participants "to collect something from the information desk and do
what is necessary". As shown in Appendix B, the task sheet provided the name of the examiner
at the bottom of the page labelled by an asterisk. When participants with stroke went to the
information desk, a number of them did not ask using the name provided and instead simply
asked if there was something available for the examiner. Also, one participant with stroke asked
if there was something available to collect for himself. Some were eventually able to realize that
they need to ask for something using the specific name of the examiner provided, however this
indicated a lack of planning when initiating these tasks.
Lastly, there were significant differences between the three groups on the number of non-task
specific irrelevant behaviours, which included apparent habitual behaviours, rule breaks and
distracters. Post hoc analysis revealed that the stroke-ED group performed more non-task
57
specific irrelevant behaviours than the stroke group. It appeared that participants in the stroke-
ED group did not seek alternative ways to perform tasks compared to controls and may have not
developed compensatory strategies like the stroke group, which may explain the high number of
non-task specific irrelevant behaviours. This finding may also indicate problems with inhibition
and the inability to monitor one's actions in participants in the stroke-ED group. It was found that
participant in the stroke-ED group were also distracted by the environment and would stop to
look at paintings and what was on the television in the patient waiting area. Furthermore, both
stroke and stroke-ED groups had a high frequency of task-related as well as casual conversations
with the examiner. It is natural and automatic to want to speak with the examiner, especially
when one had questions. However, this was a rule break and it highlighted the inability of these
participants to prevent themselves from breaking rules and altering their behaviours to fulfill the
requirements of the test at hand. This lack of self-control, and the inability to self-monitor and
inhibit oneself has been well documented as problems following ED (Alexander & Stuss, 2003;
Minassian, Perry, Carlson, Pelham, & DeFilippis, 2003).
Further investigation of results
Frequency of use of three specific behaviours differed between the stroke, stroke-ED and control
groups: going to the 0.99¢ card rack first, asking staff for directions to a location, and walking
and looking/marking map. Going to the 99¢ card rack first can only be understood in the group
of behaviours required to complete the birthday card task. If the participant went to the 0.99¢
card rack first, we categorized this as a task specific relevant efficient behaviour. There may be
several reasons for this. First and foremost, the participants are only allowed to spend $7.50 to
complete all tasks. This meant the fastest and most effective way to complete this task would be
to go straight to the 0.99¢ card rack and choose a birthday card to purchase. Secondly, the
participants are told that they are to finish the entire test in as little time as possible without
rushing excessively. If participants go straight to the 0.99¢ card rack, they do not have to spend
time comparing prices and looking for the cheapest card. Finally, since all participants in all
three groups completed the birthday card task, this meant that it would not be enough to only
look at task completion when assessing this task as it would not tell us enough about how the
participants came about completing it. We found that most controls went straight to the 0.99¢
card rack, however very few participants in stroke group and only one of the three participants in
stroke-ED group did this. They instead went to the other card rack which had regular priced
58
cards and picked up several cards and compared prices before choosing one to purchase, which
meant they spent more time in achieving this task. This is still a correct way to complete the task
without breaking any rules, however, it is not the most effective way. Some participants in stroke
and stroke-ED groups did eventually go to the 0.99¢ card rack after asking for a less expensive
card or when they noticed the rack themselves after having compared prices and cards at the
other card rack. The 0.99¢ card rack was located on the far right while the other card rack was
located closer to the entrance and came into the view first when participants entered the gift
shop. It may be the case that participants in stroke and stroke-ED groups were less likely to
consider alternatives and once they saw the other card rack, they did not go through the trouble
of looking around the store for other options.
Almost all of the participants in the stroke group and all of stroke-ED and control participants
asked staff for task-related help, however compared to controls, participants with stroke more
frequently asked staff for directions to a location. All the participants were provided with a map
(see Appendix B) which showed all the locations on the first floor of Baycrest and where
required items could be purchased and information could be obtained. It is interesting to note that
participants with stroke relied on asking for help more often than consulting the map (see below).
This may be a strategy that had proved beneficial to them in the past, which may have made
them more likely to use it. At the same time, this behaviour may also be indicating a tendency
towards disinhibition (Alderman et al., 2003) in which it is difficult for these participants to
inhibit themselves from using the same strategy and consider alternatives. Asking for help has
been closely examined by previous studies that used the MET. Both Knight et al. (2002) and
Alderman et al. (2003) reported that acquired brain injured participants used and relied on this
strategy more than their controls did. Alderman et al. (2003) also found that participants
classified as task failers benefited more from asking for help than did participants classified as
rule breakers. However, we did not observe such a pattern in our participants with stroke and
stroke-ED.
When it came to looking at/marking the map while walking, participants in stroke and stroke-ED
groups did this much less frequently compared to their controls. Knight et al. (2002) also
reported that their control group looked at the map more often. Looking at/marking the map
while walking would be beneficial for completing the entire BMET because it not only helps
with navigation to different locations, it also save time since two different behaviours are being
59
performed at once. Accordingly, this behaviour suggests different dual-tasking abilities in the
three groups with the control group demonstrating the strongest dual-tasking abilities. As
mentioned above, participants in stroke and stroke-ED groups relied heavily on asking the staff
for directions to a location, and this strategy may have led them to not bother looking at the map.
In addition, participants were provided the map on a clipboard but the map was placed behind
the task sheet and the participants had to manually flip over the task sheet in order to look at the
map. This may be another reason why stroke and stroke-ED participants were less likely to
consult the map. However, it is important to note that a few participants with stroke and stroke-
ED had hemiparesis and walked with the help of a cane (i.e. 2 participants in stroke group and 2
participants in stroke-ED group) and this may have hampered their ability to flip over to the map
and mark it while walking.
'Stroke only' vs. 'controls only' behaviours
Alderman et al. separated errors committed by brain injured participants from those made by
controls and looked at them individually. We also wanted to see if there were certain stroke only
and controls only behaviours in our sample. It was found that eight different behaviours were
performed by participants in stroke group only, one behaviour by a participant in stroke-ED
group only, while three behaviours were performed by controls only. On the basis of our
classification, three out of the eight behaviours performed by participants in stroke group were
irrelevant: licking self-adhesive stamp, waiting for Katherine to call back at payphone and taking
batteries from the information desk. The other five were classified as relevant inefficient
behaviours and all three behaviours performed by the controls were also relevant inefficient.
Performing relevant inefficient behaviours did help the participants in stroke and stroke-ED
groups achieve tasks, however they were not performed in the most efficient way. The three
irrelevant behaviours are particularly interesting as they highlight problems in inhibition and lack
of flexibility in these participants. Both licking a self-adhesive stamp and waiting for Katherine
to call back at payphone were very unusual behaviours. At the same time, taking batteries from
the information desk was also an atypical behaviour. During debriefing the participant said that
she took the batteries because she was trying to complete the task which required her to collect
something from the information desk and do what is necessary.
60
Bottari and Dawson (2011) reported that controls also make errors that may be qualitatively
similar to those observed in the brain injured population, especially those with ED. In this study,
we found that controls also performed behaviours that seemed out of the ordinary. For example,
'picking up the information desk phone and not using it', and 'telling the examiner that they have
completed the test but then continuing to work on it', both suggest to a certain extent a lack of
planning and task setting, impulsivity and distractibility, all of which are characteristics of ED
(Alexander & Stuss, 2003). In addition, a Coke bottle is usually more expensive than a can and
purchasing it would leave less money left to buy other items required in the test (for example
birthday card, stamps). This may highlight a lack of monitored spending, which is another
common problem related to the loss of monitoring abilities that may occur following ED.
Recently, Bottari and Dawson (2011) analyzed whether clinicians were able to correctly attribute
if specific isolated errors committed on the BMET were made by neurological participants or
healthy controls. They found that only 55.6% of errors were attributed to the correct population
(Bottari & Dawson, 2011). This speaks volume to the notion that a detailed analysis of the
behaviours committed during the natural course of complex behaviour in an everyday setting is
essential to make inferences about a participant's well being.
Future Directions
Future research using this behaviour classification may determine that participants in the stroke
and stroke-ED groups would commit fewer task specific and non-task specific relevant efficient
behaviours and more task specific and non-task specific relevant inefficient an irrelevant
behaviours than controls. Some of these differences, however were not observed in this study
possibly due to the limitations mentioned below.
Study limitations
The study is not without limitations. First of all, the study included a sample size that was both
small and convenient and has previously been used for at least two other studies. For this reason,
similar limitations can be found throughout these studies. Participants with stroke were several
years post-injury and had been living in the community for many years. As a result, these
participants may have adapted to their environments, which may have also impacted their
performance on the BMET. The second limitation of this study is that lesion and stroke-severity
data were not available for all participants with stroke, which meant that some may have had
61
strokes too mild to exhibit everyday problems. Another limitation of this study is that the quality
of the audio and videotapes was average. It was sometimes difficult to understand what the
participants were saying especially when they were speaking with a staff or a non-staff member.
This is because of the background noise that was also captured in the videos. In addition,
sometimes the videographer would film the participants from behind, which hindered our ability
to code certain behaviours such as 'checking watch' or 'looking at task sheet or map'.
Conclusions
This exploratory study informed us that it is important to examine a wide range of behaviours, in
addition to assessing performance on a priori behaviours, in order to better understand the impact
of ED. Accordingly, our study included behaviours that represented inefficiencies, interpretation
failures, rule breaks as well as other behaviours, which were not previously documented. To the
best of our knowledge, this study was the first to employ an event recorder in order to document
these behaviours. We presented a methodology to characterize these behaviours using our
classification. We found that participants in stroke group performed significantly more task
relevant inefficient behaviours than stroke-ED and control groups, while the stroke-ED group
performed significantly more non-task specific irrelevant behaviours than the stroke-ED and
control groups. In addition, we found that participants with stroke were significantly more likely
to ask staff for directions to a location, and significantly less likely to go to the 0.99¢ card rack
first and use the map while walking in comparison to controls. These results can be taken a step
further in improving and establishing a behaviour classification system to better characterize ED
and its impact on everyday activities. Moreover, these results can also play an important role in
refining the BMET as a performance-based naturalistic assessment of ED.
62
Chapter 5 Discussion
The main purpose of this study was to further our understanding of the impact of executive
dysfunction (ED) on everyday activities in people with stroke. This is important because ED not
only affects many aspects of daily life such as preparing a meal or shopping for groceries, it can
also have devastating effects on people's ability to achieve successful community re-integration
and social wellbeing (Grafman et al., 1996; Green, Kern, Braff, & Mintz, 2000). As described in
Chapter 2, there are few ecologically valid assessments of ED and our knowledge of how ED
impacts performance of everyday activities is very limited (Burgess et al., 2006). The Multiple
Errands Test (MET) (Shallice & Burgess, 1991) is one such assessment that captures the
situations of everyday life and provides an opportunity to examine participants' performance as
they purchase certain items and collect specific information in a naturalistic setting like a
hospital or a shopping mall. This thesis was undertaken to further examine the behaviours
performed by people with stroke and matched controls as they completed the Baycrest Multiple
Errands Test (BMET), a site specific version of the MET (Dawson et al., 2009). This would
allow for a better discrimination between people with stroke and controls and further expand our
understanding of the impact of ED in everyday life. An event recorder was selected as the
method that would be best suited to identify a wide range of behaviours as they were occurring
in a naturalistic setting. To the best of my knowledge, this methodology has not been previously
used in relation to naturalistic assessment of ED. I have developed a classification system to
analyze the behaviours performed and categorized them as either task specific or non-task
specific and into the following three categories: relevant efficient, relevant inefficient or
irrelevant behaviours. I found that participants with stroke performed significantly more task
specific relevant inefficient behaviours and non-task specific irrelevant behaviours than controls.
I also found significant discrimination between participants with stroke and controls on a number
of specific behaviours such as 'going to the 0.99¢ card rack first', 'asking staff for directions to a
location' and 'walking and looking at/marking map'. These results highlighted the importance of
performing a detailed analysis of behaviours, which would serve as a valuable measure of ED in
everyday life.
63
This chapter brings together the findings from the study I undertook with the literature review
provided in Chapter 2. First, it describes the theories of executive functions, followed by the
importance of real world assessments and behaviour analysis, and suggestion to improve the
BMET for future clinical and research use. Also, limitations of my study and suggestions on how
to improve future research on the impact of ED on everyday performance is discussed. In the
end, summary and conclusions to the entire thesis is presented.
Theories of executive functions
Six theories and models of executive functions were reviewed in Chapter 2: Duncan's theory of
goal neglect, the adaptive coding model, Mesulam's default mode, Grafman's structured event
complex framework, Norman and Shallice's supervisory attentional system and Stuss and
colleagues' fractionation of the supervisory system. Duncan's theory of goal neglect (Duncan,
1986; Duncan, Emslie, Williams, Johnson, & Freer, 1996) emphasizes the importance of goals in
human behaviour and suggests that damage to the prefrontal lobes can have an impact on goal
formulation, goal selection and goal monitoring (Turner & Levine, 2004). This is because the
main function of the prefrontal lobes is to organize and govern actions in accordance with
desired goals. In the adaptive coding model, the prefrontal lobes are viewed as a global, adaptive
unit (Duncan & Miller, 2002). Duncan and Miller (2002) suggested that the prefrontal lobes may
not have defined regions that mediate specific functions; instead the prefrontal lobes function
more generally and adapt to solve various task demands and cognitive problems. Mesulam
(2002) proposed that the main role of the prefrontal lobes is to overcome the default mode in
which actions are driven by automatic reactions and immediate need to achieve satisfaction
without consideration of contextual feedback and experience. This is achieved with the help of
executive processes that allow the individual to consider alternatives and act in more flexible
ways (Turner & Levine, 2004). In contrast, Grafman (1995) explained that different regions of
the prefrontal lobes store different features about a set of events or actions and damage to these
regions would lead to impairments in everyday life (Grafman et al., 1996). Norman and Shallice
(1986) suggested that the prefrontal lobes have a supervisory role that is involved when attention
is required for planning, decision making, multitasking, and in situations that are novel and
require inhibition of habitual responses. These abovementioned theories are valuable; however
this study was primarily influenced by Stuss and colleagues' fractionation of the supervisory
64
system (Stuss, Shallice, Alexander, & Picton (1995); Suss, 2006; Stuss & Alexander, 2007)
(which stems from Norman and Shallice's supervisory attentional system).
Stuss and colleagues conducted a series of lesion and neuroimaging studies and demonstrated
evidence of fractionation within the supervisory system (Stuss & Alexander, 2007; Stuss et al.,
1995). Two of the frontal processes identified by Stuss and colleagues were most relevant in this
study: task setting and monitoring. Task setting, which is related to the left lateral prefrontal
lobe, refers to the ability to set a stimulus-response relationship requiring formation of a
criterion, and continuous adjustment and organization of schemata necessary to complete tasks
(Stuss & Alexander, 2007). Monitoring, which is related to the right lateral frontal lobe, refers to
the ability to check the task over time to ensure quality control by keeping track of timing of the
activity, detecting occurrences of errors, and modulating actions to overcome discrepancies
(Stuss & Alexander, 2007). Having the knowledge of these two processes helped identify and
note specific behaviours. For example, behaviours related to task setting which have not been
noted in previous studies using the BMET included 'going to the 0.99¢ card rack first' as it
highlighted participants' ability to set a stimulus-response relationship in which they recognized
the need to buy a card (i.e. formation of a criterion) and spend as little as possible (i.e.
organization of schema) to complete the birthday card task. In contrast, problems in task setting
abilities were demonstrated when participants 'asked for less than four stamps' to purchase as it
demonstrated that they were unable to form a criterion correctly since they failed to understand
that the task required participants to purchase four stamps. Behaviours that reflected monitoring
included 'checking watch' as it highlighted the participants' ability to understand the timing
aspect of monitoring, while problems in monitoring were seen in behaviours such as 'arriving too
early or too late at the parrot cage' because these participants were unable to recognize the
importance of keeping track of time to ensure that they are able to accurately complete the task
of "arriving at the parrot cage in 10 minutes". It is important to note that some behaviours that
were observed were a combination of task setting and monitoring abilities, for example, when
participant 'told the examiner that s/he finished the test but continued to work on it' or when
participant 'called Katherine and waited for her to call back at payphone'. These behaviours
demonstrated problems in task setting as well as monitoring since the participants who
performed these behaviours were unable to set an appropriate stimulus-response relationship at
the beginning of the task and adjust their behaviours accordingly while they carried out the task.
65
It is also unclear as to how some of the irrelevant behaviours that were observed such as 'licking
self-adhesive stamp' or 'taking batteries from the information desk' would fit with Stuss and
colleagues' division of task setting and monitoring abilities.
It was nonetheless necessary to understand these theories and models of executive function since
a significant part of my study was devoted to the development of a classification system to better
understand the behaviours performed during the BMET. These theories, in particular Stuss and
colleagues' division of task setting and monitoring abilities, provided a basis as I was unable to
find relevant classification in the literature. However, as mentioned above, there was a lack of
agreement with Stuss and colleagues' model since I observed behaviours that were likely a
combination of difficulties in task setting and monitoring.
The importance of real world assessments and behaviour
analysis
In Chapter 2, three traditional measures were presented and critiqued on the basis of their
ecological validity (the degree to which findings in a test is related to those observed in a real-
world setting (Chaytor and Schmitter-Edgecombe, 2003)). Also, nine different real world,
performance-based assessments and questionnaires, in addition to the MET were described and
the need for them in both research and clinical setting was discussed. The findings in this study
support the need for real world, performance-based assessments as significant differences in the
types and frequencies of behaviours performed between participants with stroke and those
committed by controls were observed. The situations encountered in the BMET do relate to a
significant extent with those found in the real world (see results on ecological validity in Dawson
et al., 2009) as they include tasks such as shopping and collecting information in a naturalistic
setting. The authors of this test argue that in order to examine the impact of ED, it is important to
have participants work on multiple tasks over extended periods of time, without continuous
feedback from the examiner (Shallice & Burgess, 1991).
Although the MET is based on the principle that individuals with damage to the prefrontal lobes
may be specifically impaired in everyday situations that require executive abilities such as
planning and multitasking (Bottari & Dawson, 2011; Chan, Shum, Toulopoulou, & Chen, 2008),
this study also highlights the value of the MET over and above the assessment of ED. The
66
detailed analysis of behaviours allowed the examination of apparent habitual behaviours that
were performed by participants with stroke and controls such as 'licking the self-adhesive
stamps', 'casual self-talk' and 'casual talk with staff/non-staff/parrot/others'. It can be argued that
these behaviours may have occurred because the BMET encourages participants to carry out the
tasks in any order and the instructions are intentionally designed to be undefined and ill-
structured to observe how participants would resolve these situations using their own judgment
and experiences. These behaviours were also likely to occur as the situations encountered in the
BMET were like those that occur in everyday life. In addition, the authors of another real world
performance-based assessment namely, the Cooking Task (Chevignard et al., 2000) suggested
that real world assessments should include distracting materials to ensure an ill-structured and a
more lifelike environment. Since the MET occurs in a naturalistic, real world setting like a
hospital or a shopping mall, distracters are already present in the environment. In this instance,
behaviours that were observed related to distracters were 'looking at paintings', 'looking at
volunteer display rack', 'looking at patient area', 'looking at television in patient area' and
'drinking water from water fountain near information desk'. Examining these behaviours may
help understand the problems in distractibility, inhibition and the inability to self-monitor one's
actions.
This study also highlights the importance of undertaking a more detailed analysis of behaviours
performed, which has not been done previously. This can be emphasized by discussing the
birthday card task of the BMET as an example. All participants (controls and those with stroke)
bought a birthday card successfully. While this would typically be scored on the BMET as
completing the task, this more in-depth analysis showed that participants with stroke did not
complete the task as efficiently and smoothly as control participants. Using an event recorder
assisted in taking into account the behaviours performed in relation to this task and my
classification further helped understand them in the context of BMET and real world. It was
found that compared to controls, participants with stroke were more likely to purchase the
birthday card from the rack that contained regular priced cards as opposed to the one at the 0.99¢
card rack. In addition, some participants looked at several cards and read messages inside cards
before choosing one to purchase, while others asked the staff for a cheaper card. Therefore, if
behaviours are examined in a more detailed manner, it may highlight problems in various aspects
of executive functioning such as lack of planning, decision making or monitoring abilities.
67
By performing an in-depth analysis of behaviours, we can understand which characteristics of EF
need to be targeted during rehabilitation. For example, participants that demonstrated more
inefficient behaviours related to the lack of monitoring abilities can be trained to pay close
attention to checking their behaviours for accuracy and adjusting them to better fit the demands
of an ongoing task. Moreover, external aids such as the use of a watch can be stressed to ensure
quality control of the timing of activities. Similar behaviour analyses could also be performed for
other real-world assessments such as the Cooking Task (Chevignard et al., 2000) or the
Instrumental Activities of Daily Living Profile (Bottari, Dassa, Rainville, & Dutil, 2009a;
2009b), which would further inform rehabilitation techniques.
Improving the BMET
Although the purpose of the study was not to identify possible refinements to the BMET, the
results of our work have implications for both the clinical and research utilization of the MET
and BMET. During the course of this study, it became apparent that many participants (stroke
and control) misunderstood two rules and had trouble with three of the tasks. The three tasks
however, did not discriminate between participants with stroke and controls in this study, as well
as in previous study (Dawson et al., 2009) that used the same sample of participants. This section
discusses clarification of these tasks and rules to further enhance the ecological validity of the
MET and to improve discrimination between healthy controls and people with ED to highlight
the impact of ED on everyday tasks.
One rule that was misunderstood was "You should not go back into an area you have already
been in". This rule was included in the MET to encourage participants to plan their routes so that
they could not simply do the test task by task as they were not allowed to use areas such as the
gift shop or the information desk more than once. However, some participants understood this
rule to mean they were not to use the same hallway more than once. Some tried to navigate to
different locations on the main floor using different hallways and walked into prohibited areas. If
this rule can be reworded to specify what it is intending, it would ensure that all participants are
completing the test without being misunderstood. It can be reworded as "You may not enter the
same location you have already been in (e.g. Resident's library)" as it may have been that the
word 'area' that had caused the misunderstanding. Also, providing an example may further help
clarify the rule for the participants.
68
The task of 'collecting something from the examiner from the information desk and doing what is
necessary' also caused confusion in some participants. A few thought that they could collect
anything from the information desk and one participant with stroke took batteries. Others had a
hard time 'doing what is necessary with the letter collected' and kept the letter with them. Still,
others gave the letter to the examiner at the end of the test even though it said 'urgent' on the
envelope, however these behaviours did not discriminate between participants with stroke and
controls. This task seems more artificial considering the other tasks on the BMET because in the
real world if the envelope was to be delivered to the examiner urgently, it would be highly
unlikely to have been sitting at the information desk. The intent of the task, at least in part, seems
to be to provide an interruption to the planning participants might have done up to that point in
the test. Thus, the task may be modified to look more like an interruption that would occur in a
real world setting, which in turn may improve the ecological validity of the test. For example, the
examiner could hand the participant a note stating, "We forgot to tell you this earlier, but you
also need to buy a bag of chips." These types of interruptions occur in everyday life when we
receive a call or a text message from our spouses or parents telling us to also purchase a specific
item which they had forgotten to include on the grocery list. This would allow us to analyze the
participants' ability to reorganize their plans in the face of the interruption, and their task setting,
decision making, and monitoring abilities, all of which are important characteristics of ED. I
would hypothesize that control participants would have less difficulty adjusting to the
interruption relative to people with ED.
Another task that caused problems for the participants working on the BMET was 'meeting the
examiner at the parrot cage 10 minutes after they had started the test and telling the examiner the
time'. The instructions on this task were deliberately set to be a bit confusing to see how
participants would complete this task because if the instructions are too clear and concrete, the
test may not be sensitive to ED. Some participants rushed through the test thinking they had to
complete the entire exercise in 10 minutes to meet the examiner at the parrot cage. Also, the rule
of 'taking as little time as possible to complete the exercise' may have further strengthened this
misconception. Others took their time working on the test and met the examiner at the end of the
exercise while paying no particular attention to the time that had elapsed. Some met the examiner
at the parrot cage but did not state the time. It was surprising to see this confusion in both
participants with stroke and controls especially when they linked the task with the rule during
69
briefing. Some rewording of the task and rule might prevent misunderstandings related to this
task and better discriminate between control participants and people with ED in future. Also, this
task can perhaps be modified to be more representative of everyday situations by providing the
participants with an envelope of some sort and asking them that they would have to meet
someone at the parrot cage at a specific time to deliver it, as this may enhance the ecological
validity of the test.
The task requiring participants to report the number of entrances/exits on the main floor of
Baycrest caused problems during coding. Some participants asked a staff member to tell them
the number of entrances/exits and this was easy to code, however it was difficult to code whether
others used the map to count them. The examiner may have also missed this behaviour if the
participant had done this while walking and were away from the examiner. Perhaps modifying
the task to report which exit is closest to a particular location may eliminate this inconsistency.
Further, this would be more representative of a real-world situation than reporting the number of
entrances/exits on the main floor.
Making these suggested changes to the BMET would further make the assessment more
representative of everyday life and allow for better discrimination between controls and people
with ED. This is because these tasks caused misunderstanding in both participants with stroke
and controls, which may have impacted their performance. This may also explain why Bottari
and Dawson (2011) found that only 55.6% of errors in the standard scoring of the MET were
attributed to the correct population by clinicians who watched short video clips of participants
with acquired brain injury and controls demonstrating single performance errors. Bottari and
Dawson (2011) emphasized that clinicians need to be more cautious and aware of the risk of
misinterpreting single behaviours demonstrating errors in the real world. This study took into
account a wide range of behaviours and highlighted the importance of performing an in-depth
analysis of behaviours in order to better understand the impact of ED. In addition, Rand, Basha-
Abu Rukran, Weiss, and Katz (2008) suggested that BMET may also be used to assess ED
following an intervention or a rehabilitative training. Although Rand et al. (2008) were
deliberating on the virtual version of the MET, improvement in EF may also be examined using
the BMET. This is especially the case since Rand et al. (2008) found that real-world version of
the MET was better than the virtual one because it allowed observing social mistakes such as
ignoring that there was a line up ahead when paying for items purchased.
70
Limitations
One of the main limitations of this study was that it was a secondary analysis that used
participant videos not originally meant for the purpose of this study. This resulted in difficulty
characterizing some behaviours due to limitations of the audio and video quality. In several
instances, it was difficult to understand what the participants were saying when they were
speaking to the staff and non-staff members. This was important because I wanted to
discriminate between casual and task related conversations as it would highlight planning, task
setting, multitasking and monitoring abilities. Similarly, it was especially difficult to comprehend
when participants engaged in self-talk as the built-in microphone in the camera was not always
close to the participant to capture what was said. There was also a lot of background noise which
was captured in the videos (for example, piano being played for the patients near the gift shop,
people talking amongst themselves in the gift shop or the cafeteria, parrot squawking) and some
times, the participants spoke too softly, which further limited our ability to examine their talking
behaviours. Moreover, the videographer would sometimes film from behind the participants
because s/he would not be able to keep up with them or someone passing by would come in front
of the camera. This would restrict our ability to code various behaviours. However, it was
understood that some of these difficulties would be expected while filming a video in a
naturalistic, uncontrolled setting like the main floor of a hospital. The study used a relatively
small sample size, particularly the stroke-ED group. A small sample size has a greater
probability that the results obtained may be due to chance (i.e. the magnitude of the results may
be overestimated) or that a type II error was made (i.e. failing to reject the null hypothesis when
the null hypothesis is false) (Hackshaw, 2008). There was also lack of lesion data and medical
histories available. This meant that we could not examine the impact of ED in relation to lesion
location, which would have helped strengthen the association between the categorization of
certain behaviours and theories of executive functions, for example, the relationship between
damage to the left lateral frontal lobe and poor task setting behaviours. In addition, as mentioned
in chapter 4, it may be that some participants with stroke had conditions too mild to demonstrate
ED. The participants with stroke were many years post-stroke and may have developed
compensatory strategies that helped them in their daily life. This, in turn may have impacted
their performance on the BMET.
71
This study was unique to use an event recorder to code the behaviours captured in the videos.
However, there were several limitations of the Behaviour Tracker. The Behaviour Tracker
permitted one configuration file to include a maximum of 37 events. As a result, we had to create
more configuration files and run multiple instances of Behaviour Tracker using separate
configuration files. Due to this limitation, once the coding was completed the spreadsheets from
each instance had to be merged manually into a single spreadsheet. In addition, although
Behaviour Tracker allowed coding of duration of specific behaviours, considerable variability in
coding duration between computers and platforms was found (Windows or MAC). Thus, the
reliability of the duration coding was very low. As a result, the duration of behaviours was not
included in the study. If it had been possible to code the duration of behaviours accurately,
perhaps we would have been able to see more discrimination among the participants with stroke
and controls. Another limitation of the Behaviour Tracker was that if a behaviour was missed,
during the original coding of behaviours, we had to manually enter this behaviour once the data
was exported to the spreadsheet. The software did not allow entering the missed behaviour in the
editor mode.
Since this was a secondary analysis, some of the abovementioned limitations could not be
avoided. Ideally, participants with stroke that would be recruited would have documented ED
which would be separated on the basis of the number of years post-stroke as participants with
stroke that had documented ED in this study had their stroke more recently than the participants
in the stroke group without neuropsychologically defined ED. These participants would be
matched with their controls on age, gender and education like this study. It would be feasible to
include more participants and controls (i.e. a higher 'n') to support the study. The idea of filming
participants as they complete the BMET would still be important as it would permit further
analyses to be performed. However, the use of more sophisticated equipment would be
warranted to enhance accuracy of behaviour coding. For example, head cameras and eye tracking
technology, which detects a person's gaze would be used to enhance audio and video quality, and
eliminate background noise. This would also eliminate the need of a videographer and examiner
to walk closely behind the participant and permit the examiner to take notes from a distance and
only come close to the participant when it is part of the test (for example, when he/she needs to
give an interrupting message).
72
Future directions
Future research should address the limitations described earlier in this chapter to further our
understanding of the impact of ED on everyday activities in participants with stroke, especially if
the BMET or some variation of the MET is used as the measure of ED. Modification to the rules
and tasks of the BMET would allow the test to be more representative of everyday life. In
relation to this study, future research should consider refining the behaviour classification we
proposed and as mentioned earlier, this type of detailed behaviour analysis can also be
incorporated in other naturalistic assessments of ED such as the Cooking Task (Chevignard et
al., 2000) or the Instrumental Activities of Daily Living Profile (Bottari et al., 2009a; 2009b).
This type of behaviour analysis also has important clinical implications as information obtained
from this study can pave way for the development of more targeted rehabilitation programs.
Summary & conclusions
This research study explored a wide range of behaviours performed by participants with stroke
and their matched controls as they completed the BMET to further understand the impact of ED
on everyday activities. An event recorder was used to code the occurrences and frequencies of
behaviours. To the best of my knowledge, this methodology has not been previously used in
relation to naturalistic assessment of ED following stroke. A classification system was developed
to characterize these behaviours. It was found that participants with stroke performed
significantly more task specific relevant inefficient behaviours and non-task specific irrelevant
behaviours compared to controls. Specific behaviours were also analyzed and it was found that
participants with stroke were significantly more likely to ask staff for directions to a location,
and significantly less likely to go to the 0.99¢ card rack first and use the map while walking in
comparison to controls. In summary, this study served as an example to highlight the importance
of carrying out an in-depth analysis of behaviours performed. These results are promising and
support the need for future investigation of a wide range of behaviours, in addition to assessing
performance errors, inefficiencies, interpretation failures and rule breaks on the BMET to better
understand the impact of ED on everyday life.
73
References
Alderman, N., Burgess, P. W., Knight, C., & Henman, C. (2003). Ecological validity of a
simplified version of the multiple errands shopping test. Journal of the International
Neuropsychological Society, 9, 31-44.
Alexander, M. D., & Stuss, D. T. (2003). Executive disorders. In J. Noseworthy (Ed.),
Neurological Therapeutics, Principles and Practice, Vol. 2 (pp. 2808-2809). London:
Martin Dunitz.
Allain, P., Le Gall, D., Etcharry-Brouyx, F., Aubin, G., & Emile, J. (1999). Mental
representation of knowledge following frontal-lobe lesion: dissociations on tasks using
scripts. Journal of Clinical and Experimental Neuropsychology, 21, 643-665.
Anderson, C. V., Bigler, E. D., & Blatter, D. D. (1995). Frontal lobe lesions, diffuse damage, and
neuropsychological functioning of traumatic brain-injured patients. Journal of Clinical
and Experimental Neuropsychology, 17, 900-908.
Andre, M., Anderson, N. D., Stuss, D. T., & Dawson, D. R. (2009). Executive dysfunction and
strategy use during completion of the Multiple Errands Test. Neuroscience Meeting
Planner Program No. 675.17. Chicago, IL: Society for Neuroscience, 2009. Online.
Barceló, F., Escera, C., Corral, M. J., & Periañez, J. A. (2006). Task switching and novelty
processing activate a common neural network for cognitive control. Journal of Cognitive
Neuroscience, 18, 1734-1748.
Barkley, R. (1997). Behavioural inhibition, sustained attention, and executive functions:
Constructing a unifying theory. Psychological Bulletin, 121, 65-94.
Baum, C. M., Connor, L. T., Morrison, T., Hahn, M., Dromerick, A. W., & Edwards, D. F.
(2008). Reliability, validity, and clinical utility of the Executive Function Performance
Test: a measure of executive function in a sample of people with stroke. The American
Journal of Occupational Therapy, 62(4), 446-455.
74
Baum, C. M., & Edwards, D. (1993). Cognitive performance in senile dementia of the
Alzheimer‘s type: The kitchen task assessment. American Journal of Occupational
Therapy, 47, 431-436.
Bechara, A., Damásio, A. R., Damásio, H., Anderson, S. W. (1994). Insensitivity to future
consequences following damage to human prefrontal cortex.Cognition 50 (1-3), 7-15.
Behaviour Tracker. (2003). Behaviour Tracker (Version 1.5). Available from
http://www.behaviourtracker.com/
Bennett, P. C., Ong, B., & Ponsford, J. (2005). Measuring executive dysfunction in an acute
rehabilitation setting: Using the dysexecutive questionnaire (DEX). Journal of the
International Neuropsychological Society, 11, 376-385.
Benton, A., Hamsher, K, & Sivan, A. (1994). Multilingual Aphasia Examination (3rd ed.). San
Antonio, TX: Psychological Corporation.
Bogod, N. M., Mateer, C. A., & MacDonald, S. W. S. (2003). Self-awareness after traumatic
brain injury: A comparison of measures and their relationship to executive functions.
Journal of the International Neuropsychological Society, 9, 450-458.
Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2009a). The criterion- related validity of the
IADL Profile with measures of executive functions, indices of trauma severity and
sociodemographic characteristics. Brain Injury, 23, 322-335.
Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2009b). The factorial validity and internal
consistency of the Instrumental Activities of Daily Living Profile in individuals with a
traumatic brain injury. Neuropsychological Rehabilitation, 19, 177-207.
Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2010). The IADL Profile: Development, content
validity, intra- and interrater agreement. Canadian Journal of Occupational Therapy,
77(2), 90-100.
Bottari, C., Swaine, B., &Dutil, E. (2007). Interpreting activity of daily living errors for
treatment and discharge planning: The perception of occupational therapists. Journal of
Head Trauma Rehabilitation, 22(1), 26–30.
75
Bottari, C., & Dawson, D. R. (2011). Executive functions and real-world performance: How
good are we at distinguishing people with acquired brain injury from healthy controls?
OTJR: Occupation, Participation and Health, 31(1), S61-S68.
Bowie, C. R., & Harvey, P. D. (2006). Administration and interpretation of the Trail Making
Test. Nature Protocols, 1(5), 2277-2281.
Braswell, D., Hartry, A., Hoornbeek, S., Johansen, A., Johnson, L., Schultz, J., … Sohlberg, M.
M. (1992). Profile of executive control system. Puyallup, WA: Association for
Neuropsychological Research and Development.
Bryan, J., & Luszcz, M. A. (2000). Measurement of executive function: Considerations for
detecting adult age differences*. Journal of Clinical and Experimental
Neuropsychology, 22(1), 40-55.
Burgess, P. W. (2000). Strategy application disorder: the role of the frontal lobes in human
multitasking. Psychological Research, 63, 279-288.
Burgess, P. W., Alderman, N., Evans, J., Emslie, H., & Wilson, B. A. (1998). The ecological
validity of tests of executive function. Journal of the International Neuropsychological
Society, 4, 547-558.
Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates, L. M., Dawson, D. R., et al.
(2006). The case for the development and use of "ecologically valid" measures of
executive function in experimental and clinical neuropsychology. Journal of the
International Neuropsychological Society, 12, 194-209.
Burgess, P. W., & Robertson, I. H. (2002). Principles of the rehabilitation of frontal lobe
function. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe function (pp.
557-571). New York, NY: Oxford University Press.
Burgess, P. W., Veitch, E. J., Costello, A. de L., & Shallice, T. (2000). The cognitive and
neuroanatomical correlates of multitasking. Neuropsychologia, 38, 848–863.
Boyd, T. M. & Sautter, S. W. (1993). Route-finding: A Measure of Everyday Executive
Functioning in the Head-injured Adult. Applied Cognitive Psychology, 7, 171.181.
76
Cabeza, R., & Nyberg, L. (2000). Imaging cognition II: an empirical review of 275 PET and
fMRI studies. Journal of Cognitive Neuroscience, 12, 1-47.
Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H. (2008). Assessment of executive
functions: Review of instruments and identification of critical issues. Archives of Clinical
Neuropsychology, 23, 201-216.
Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological
tests: A review of the literature on everyday cognitive skills. Neuropsychology Review,
13(4), 181-197.
Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of
executive functioning assessment. Archives of Clinical Neuropsychology, 21, 217-227.
Chevignard, M., Pillon, B., Pradat-Diehl, P., Taillefer, C., Rousseau, S., Le Bras, C., & Dubois,
B. (2000). An ecological approach to planning dysfunction: Script execution. Cortex, 36,
649-669.
Chevignard, M. P., Taillefer, C., Picq, C., Poncet, F., Noulhiane, M., & Pradat-Diehl, P. (2008).
Ecological assessment of the dysexecutive syndrome using execution of a cooking task.
Neuropsychological Rehabilitation, 18(4), 461-485.
Cicerone, K., Levin, H., Malec, J., Stuss, D., & Whyte, J. (2006). Cognitive rehabilitation
interventions for executive function: Moving from bench to bedside in patients with
traumatic brain injury. Journal of Cognitive Neuroscience, 18, 1212-1222.
Dawson, D. R., Anderson, N., Burgess, P., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009).
Further development of the Multiple Errands Test: Standardized scoring, reliability and
ecological validity for the Baycrest Version. Archives of Physical Medicine &
Rehabilitation, 90(11), S41-S51.
Dawson, D. R., Anderson, N., Burgess, P. W., Levine, B., Rewilak, D., Cooper, E. K., Farrow,
S., Koscik, K. M., Krpan, K. M., Lo, A., Peer, M., & Stuss, D. T. (2005a). The ecological
validity of the Multiple Errands Test—Hospital Version: Preliminary findings. Poster
77
presented at Meeting of the International Neuropsychological Society, St. Louis, USA,
February.
Demakis, G. J. (2003). A meta-analytic review of the sensitivity of the Wisconsin Card Sorting
Test to frontal and lateralized frontal brain damage. Neuropsychology, 17, 255-264.
Duncan, J. (1986). Disorganization of behaviour after frontal lobe damage. Cognitive
Neuropsychology, 2, 271-290.
Duncan, J. (1995). Attention, intelligence and the frontal lobes. In M. S. Gazzaniga (Ed.), The
cognitive neurosciences (pp. 721-733). Cambridge, MA: MIT Press.
Duncan, J. (2001). An adaptive coding model of neural function in prefrontal cortex. Nature
Reviews Neuroscience, 2, 820-829.
Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal
lobe: The organization of goal-directed behaviour. Cognitive Psychology, 30, 257-303.
Duncan, J., & Miller, E.K. (2002). Cognitive focus through adaptive neural coding in the primate
prefrontal cortex. In D.T. Stuss & R.T. Knight (Eds.), Principles of frontal lobe function
(pp. 278–291). New York, NY: Oxford University Press.
Duncan, J., & Owen, A. M. (2000). Common regions of the human frontal lobe recruited by
diverse cognitive demands. Trends in Neuroscience, 23, 475-483.
Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., Herzog, H., Ahmed, A., … Emslie, H. (2000). A
neural basis for general intelligence. Science, 289, 457-460.
Dutil, E., Bottari, C., Vanier, M., & Gaudreault, C. (2005). ADL Profile: Description of the
instrument (H. Scott & C. Bottari, Trans. 4th ed., Vol. 1). Montreal: Les Éditions
Émersion.
Dutil, E., Forget, A., Vanier, M., & Gaudreault, C. (1990). Development of the ADL Profile: An
evaluation for adults with severe head injury. Occupational Therapy in Health Care, 7, 7-
22.
78
Elliot, R. (2003). Executive functions and their disorders. British Medical Bulletin, 65, 49-59.
Franzen, M. D., and Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in
neuropsychology. In Sbordone, R. J. & Long, C. J. (Eds.), Ecological Validity of
Neuropsychological Testing (pp. 91-112). Delray Beach, FL: GR Press/St. Lucie Press.
Garlinghouse, M. A., Roth, R. M., Isquith, P. K., Flashman, L. A., & Saykin, A. J. (2010).
Subjective rating of working memory is associated with frontal lobe volume in
schizophrenia. Schizophrenia Research, 120, 71-75.
Gilbert S. J.,& Burgess, P. W. (2008). Executive function. Current Biology, 18(3), R110-R114.
Gillen, G. (2008). Managing executive function impairments to optimize function. Cognitive and
perceptual rehabilitation optimizing function. Cognitive and Perceptual
Rehabilitation: Optimizing Function (pp. 245-283). St. Louis: Mosby Elsevier.
Giovagnoli, A. R. (2001). Relation of sorting impairment to hippocampal damage in temporal
lobe epilepsy. Neuropsychologia, 39, 140-150.
Gladsjo, J., Schuman, C., Miller, S., & Heaton, R. (1999). Norms for letter and category fluency:
Demographic corrections for age, education, and ethnicity. Odessa, FL: PAR.
Godbout, L., Grenier, M. C., Braun, C. M., & Gagnon, S. (2005). Cognitive structure of
executive deficits in patients with frontal lesions performing activities of daily
living.Brain Injury, 19, 337-348.
Gonzalez-Hernandez, J. A., Cedeno, I., Pita-Alcorta, C., Galan, L., Aubert, E., & Figueredo-
Rodriguez, P. (2003). Induced oscillations and the distributed cortical sources during the
Wisconsin card sorting test performance in schizophrenic patients: New clues to neural
connectivity. International Journal of Psychophysiology, 48, 11-24.
Gonzalez-Hernandez, J. A., Pita-Alcorta, C., Cedeno, I., Bosch-Bayard, J., Galan-Garcia, L.,
Scherbaum, W. A., …Figueredo-Rodriguez, P. (2002). Wisconsin Card Sorting Test
synchronizes the prefrontal, temporal and posterior association cortex in different
frequency ranges and extensions. Human Brain Mapping, 17, 37-47.
79
Goverover, Y., Kalmar, J., Gaudino-Goering, E., Shawaryn, M., Moore, N. B., Halper, J.,
...DeLuca, J. (2005). The relation between subjective and objective measures of everyday
life activities in persons with multiple sclerosis. Archives of Physical Medicine and
Rehabilitation, 86,2303-2308.
Grafman, J. (1995). Similarities and distinctions among current models of prefrontal cortical
functions. Annals of the New York Academy of Sciences, 769, 337-368.
Grafman, J., Schwab, K., Warden, D., Pridgen, A., Brown, H. R.,& Salazar, A. M. (1996).
Frontal lobe injuries, violence, and aggression: A report of the Vietnam Head Injury
Study. Neurology, 46, 1231-1238.
Green, M. F., Kern, R. S., Braff, D. L., & Mintz, J. (2000). Neurocognitive deficits and
functional outcome in schizophrenia: Are we measuring the ―right stuff‖? Schizophrenia
Bulletin, 26(1), 119-136.
Heaton, R., Chelune, G., Talley, J., Kay, G., & Curtiss, G. (1993). Wisconsin Card Sorting Test
manual: revised and expanded. Odessa, FL: Psychological Assessment Resources.
Henry, J. D. & Crawford, J. R. (2004a). A meta-analytic review of verbal fluency performance
following focal cortical lesions. Neuropsychology, 18, 284-295.
Hughes, C., & Graham, A. (2002). Measuring executive functions in childhood: Problems and
solutions. Child and Adolescent Mental Health, 7, 131-142.
Igarashi, K., Oguni, H., Osawa, M., Awaya, Y., Kato, M., Mimura, M., … Kashima, H. (2002).
Wisconsin card sorting test in children with temporal lobe epilepsy. Brain &
Development, 24, 174-178.
Josman, N., & Birnboim, S. (2001). Measuring kitchen performance: What assessment should
we choose? Scandinavian Journal of Occupational Therapy, 8, 193-202.
Jurado, M. B., & Rosselli, M. (2007). The Elusive nature of executive functions: A review of
our current understanding. Neuropsychological Review, 17, 213-233.
Kadafar, K. (2003). John Tukey and Robustness. Statistical Science, 18(3), 319-331.
80
Katz, N., Tadmor, I., Felzen, B., & Hartman-Maeir, A. (2007). Validity of the Executive
Function Performance Test in Individuals with Schizophrenia. Occupational Therapy
Journal of Research, 27, 44-51.
Keil, K., & Kaszniak, A. W. (2002). Examining executive function in individuals with brain
injury: A review. Aphasiology, 16(3), 305-335.
Kingstone, A., Smilek, D., Birmingham, E., Cameron, D., & Bischof, W. F. (2005). Cognitive
ethology: Giving real life to attention research. In J. Duncan, L. Phillips, & P. McLeod
(Eds.), Measuring the mind: Speed, control and age (pp. 341–357). Oxford, UK: Oxford
University Press.
Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for
studying human cognition. British Journal Psychology, 99(Pt. 3), 317-340.
Knight, C., Alderman, N., & Burgess, P. W. (2002). Development of a simplified version of the
multiple errands test for use in hospital settings. Neuropsychological Rehabilitation,
12(3), 231-255.
Koechlin, E., Corrado, G., Pietrini, P., & Grafman, J. (2000). Dissociating the role of the medial
and lateral anterior prefrontal cortex in human planning. Proceedings of the National
Academy of Sciences, 97, 7651-7656.
Kvavilashvili, L. & Ellis, J. (2004). Ecological validity and the real-life/laboratory controversy in
memory research: A critical and historical review. History & Philosophy of Psychology,
6, 59-80.
Lamberts, K. F., Evans, J. J., & Spikman, J. M. (2010). A real-life, ecologically valid test of
executive functioning: The executive secretarial task. Journal of Clinical and
Experimental Neuropsychology, 32(1), 56-65.
Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of
effect size in communication research. Human Communication Research, 28(4), 612-625.
Levine, B., Turner, G. R., & Stuss, D. T. (2008). Rehabilitation of frontal lobe functions. In D. T.
Stuss, G. Winocur & I. H. Robertson (Eds.), Cognitive Neurorehabilitation, 2nd
81
Edition: Evidence and Application (pp. 464-486). Cambridge: Cambridge University
Press.
Lewis, M. W., Babbage, D. R., & Leathem, J. M. (2011). Assessing executive performance
during cognitive rehabilitation. Neuropsychological Rehabilitation, 21(2), 145-163.
Lezak, M. D. (2004). Executive function and motor performance.In M. D. Lezak, D. B.
Howieson & D. W. Loring (Eds.), Neurological Assessment. New York: Oxford
University Press.
Lezak, M. D., Howieson, D. B., & Loring, D.W. (2004). Neuropsychological assessment.4th
Edition. New York: Oxford University Press.
Lie, C. H., Specht, K., Marshall, J. C., & Fink, G. R. (2006). Using fMRI to decompose the
neural processes underlying the Wisconsin Card Sorting Test. Neuroimage, 30, 1038-
1049.
Malloy, P., & Grace, J. (2005). A review of rating scales for measuring behaviour change due to
frontal systems damage. Cognitive and Behavioural Neurology, 18, 18-27.
Manchester, D., Priestley, N., & Howard, J. (2004). The assessment of executive functions:
Coming out of the office. Brain Injury, 18(11), 1067-1081.
Manly, T., Hawkins, K., Evans, J. J., Woldt, K., & Robertson, I. H. (2002). Rehabilitation of
executive function: Facilitation of effective goal management on complex tasks using
periodic auditory alerts. Neuropsychologia, 40(3), 271-281.
Marcotte, T. D., & Grant, I. (2010). Future directions in the assessment of everyday functioning.
In T. D. Marcotte & I. Grant (Eds.), Neuropsychology of everyday functioning (pp. 457-
461). New York, NY: Guilford Press.
Marcotte, T. D., Scott, J. C., Kamat, R., & Heaton, R. K. (2010). Neuropsychology and the
prediction of everyday functioning. In T. D. Marcotte & I. Grant (Eds.), Neuropsychology
of everyday functioning (pp. 5-38). New York, NY: Guilford Press.
82
Mesulam, M. (2002).The human frontal lobes: Transcending the default mode through
contingent encoding. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe
function (pp. 8-30). New York, NY: Oxford University Press.
Minassian, A., Perry, W., Carlson, M., Pelham, M., & DeFilippis, N. (2003). The Category Test
perseveration, loss of set, and memory scales. Three New Scales and Their Relationship
to Executive Functioning Measures. Assessment, 10(3), 213-221.
Miyake, A., Emerson, M. J., & Friedman, N. P. (2000). Assessment of executive functions in
clinical settings: Problems and recommendation. Seminars in Speech and Language, 21,
169-183.
Miyake, A., Friedman, N., Emerson, M., Witzki, A., & Howerter, A. (2000). The unity and
diversity of executive functions and their contributions to complex ―frontal lobe‖ tasks: A
latent variable analysis. Cognitive Psychology, 41, 49-100.
Mohlman, J. (2005). Does executive dysfunction affect treatment outcome in late-life mood and
anxiety disorders. Journal of Geriatric Psychiatry and Neurology, 18(2), 97-108.
Monchi, O., Petrides, M., Petre, V., Worsley, K., & Dagher, A. (2001). Wisconsin Card Sorting
revisited: Distinct neural circuits participating in different stages of the task identified by
event-related functional magnetic resonance imaging. The Journal of Neuroscience, 21,
7733-7741.
Mukhopadhyay, P., Dutt, A., Kumar Das, S., Basu, A., Hazra, A., Dhibar, T., … Roy, T. (2008).
Identification of neuroanatomical substrates of set-shifting ability: Evidence from
patients with focal brain lesions. Progress in Brain Research, 168,95-104.
Nagahama, Y., Fukuyama, H., Yamauchi, H., Katsumi, Y., Magata, Y., Shibasaki, H., …
Kimura, J. (1997). Age-related changes in cerebral blood flow activation during a card
sorting test. Experimental Brain Research, 114, 571-577.
Neistadt, M. E. (1992). The Rabideau Kitchen Evaluation – Revised: An assessment of meal
preparation skill. Occupational Therapy Journal of Research, 12, 242-253.
83
Norman, D. A., & Shallice, T. (1986). Attention to action: Willed and automatic control of
behaviour. In R.J. Davidson, G.E. Shwartz & D. Shapiro (Eds.), Consciousness and self
regulation: Advances in research and theory (pp. 1-18). New York, NY: Plenum.
Nyhus, E. & Barceló, F. (2009). The Wisconsin Card Sorting Test and the cognitive assessment
of prefrontal executive functions: A critical update. Brain and Cognition, 71, 437-451.
Pineda, D. A., & Merchan, V. (2003). Executive function in young Colombian adults.
International Journal of Neuroscience, 113, 397-410.
Proctor, A., Wilson, B., Sanchez, C., & Wesley, E. (2000). Executive function and verbal
working memory in adolescents with closed head injury (CHI). Brain Injury, 14(7), 633-
647.
Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical
neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA
division 40 members. Archives of Clinical Neuropsychology, 20, 33-65.
Rabin, L. A., Roth, R. M., Isquith, P. K., Wishart, H. A., Nutter-Upham, K. E., Pare, N., …
Saykin, A. J. (2006). Self- and informant reports of executive function on the BRIEF-A
in MCI and older adults with cognitive complaints. Archives of Clinical
Neuropsychology, 21, 721-732.
Radloff, L. (1977). The CES-D scale: A self-report depression scale for research in the general
population. Applied Psychological Measurement, 1, 385-401.
Rand, D., Rukan, S. B., Weiss, P. L. T., & Katz, N. (2008). Validation of the Virtual MET as an
assessment tool for executive functions. Neuropsychological Rehabilitation, iFirst, 1-20.
Reitan, R., & Wolfson, D. (1985). The Halstead-Reitan Neuropsychological Test Battery.
Tucson, AZ: Neuropsychology Press.
Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, I. (1996). The structure of normal
human attention: The Test of Everyday Attention. Journal of the International
Neuropsychological Society, 2(6), 525-534.
84
Rogers, R. D., Andrews, T. C., Grasby, P. M., Brooks, D. J., & Robbins, T. W. (2000).
Contrasting cortical and subcortical activations produced by attentional-set shifting and
reversal learning in humans. Journal of Cognitive Neuroscience, 12, 142-162.
Roth, R.M., Isquith, P.K., & Gioia, G.A. (2005).BRIEF-A: Behaviour Rating Inventory of
Executive Function - Adult Version. Lutz, FL: Psychological Assessment Resources.
Royall, D. R., Lauterbach, E. C., Kaufer, D., Malloy, P., Coburn, K. L., & Black, K. J. (2007).
The cognitive correlates of functional status: A review from the Committee on Research
of the American Neuropsychiatric Association. Journal of Neuropsychiatry and Clinical
Neurosciences, 19, 249-265.
Salthouse, T., Atkinson, T., & Berish, D. (2003). Executive functioning as a potential mediator
of age-related cognitive decline in normal adults. Journal of Experimental Psychology:
General, 132, 566-594.
Shallice, T. (2002).Fractionation of the supervisory system. In D. T. Stuss & R. T. Knight (Eds.),
Principles of frontal lobe function (pp. 261-277). New York, NY: Oxford University
Press.
Shallice, T., & Burgess, P. (1991). Deficits in strategy application following frontal lobe damage
in man. Brain, 114, 727-741.
Shallice, T. & Evans, M.E. (1978). The involvement of the frontal lobes in cognitive estimation.
Cortex, 14, 294–303.
Shilling, V.M, Chetwynd, A., & Rabbitt, P.M.A. (2002). Individual inconsistency across
measures of inhibition: An investigation of the construct validity of inhibition in older
adults. Neuropsychologia, 40, 605-619.
Sirigu, A., Zalla, T., Pillon, B., Grafman, J., Agid, Y., & Dubois, B. (1996). Encoding of
sequence and boundaries of scripts following prefrontal lesions. Cortex 32, 297-310.
Spielberger, C. D., Gorsuch, R.L., and Lushene.R.E. (1970). Manual for the State-Trait
Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.
85
Spooner, D. M., & Pachana, N. A. (2006). Ecological validity in neuropsychological assessment:
A case for greater consideration in research with neurologically intact populations.
Archives of Clinical Neuropsychology, 21, 327–337.
Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A compendium of Neuropsychological Tests:
Administration, Norms and Commentary. Third Edition. New York: Oxford University
Press.
Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and
integration. Journal of the International Neuropsychological Society, 12, 261-271.
Stuss, D. T. (2007). New approaches to prefrontal lobe testing. In B Miller & J Cummings
(Eds.), The Human Frontal Lobes: Functions and Disorders, 2nd Edition (pp. 292-305).
New York: Guilford Press.
Stuss, D. T., & Alexander, M. P. (2000). Executive functions and the frontal lobes: a conceptual
view. Psychological Research, 63, 289-298.
Stuss, D. T., & Alexander, M. P. (2007). Is there a dysexecutive syndrome? Philosophical
Transactions of the Royal Society B, 362, 901-915.
Stuss, D. T., Alexander, M. P., Floden, D., Binns, M. A., Levine, B., McIntosh, A. R., …
Hevenor, S. J. (2002). Fractionation and localization of distinct frontal lobe processes:
Evidence from focal lesions in humans. In D. T. Stuss & R. T. Knight (Eds.), Principles
of frontal lobe function (pp. 392-406). New York, NY: Oxford University Press.
Stuss, D. T., Levine, B., Alexander, M. P., Hong, J., Palumbo, C., Hamer, L., … Izukawa, D.
(2000). Wisconsin Card Sorting Test performance in patients with focal frontal and
posterior brain damage: Effects of lesion location and test structure on separable
cognitive processes. Neuropsychologia, 38, 388-402.
Stuss, DT, Floden, D, Alexander, MP, Levine, B, & Katz, D (2001). Stroop performance in focal
lesion patients: Dissociation of processes and frontal lobe lesion location.
Neuropsychologia, 39, 771-786.
86
Stuss, D. T., Shallice, T., Alexander, M. P., & Picton, T. W. (1995). A multidisciplinary
approach to anterior attentional functions. Annals of the New York Academy of Sciences,
769, 191-211.
Turner, G. R. & Levine, B. (2004). Disorders of Executive Functioning and Self Awareness. In
J. Ponsford (Ed.), Cognitive and Behavioural Rehabilitation: From Neurobiology to
Clinical Practice (224-268). New York: Guilford Press.
Vrankrijker, K. De. M. W. De. (2003). The long way from the International Classification of
Impairements, Disabilities and Handicaps (ICIDH) to the International Classification of
Functioning, Disability and Health (ICF). Disability and Rehabilitation, 25 (11/12), 561-
564.
Vuilleumier, P., & Driver, J. (2007). Modulation of visual processing by attention and emotion:
windows on causal interactions between human brain regions. Philosophical
Transactions of the Royal Society B: Biological Sciences, 362, 837-855.
Wechsler, D. (1985). The Wechsler adult intelligence scale—revised. New York, NY:
Psychological Corporation.
Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans, J. J. (1996). Behavioural
Assessment of the Dysexecutive Syndrome. Bury St. Edmunds, UK: Thames Valley Test
Company.
Wilson, B. A., Cockburn, J., & Baddeley, A. D. (1985). The Rivermead Behavioural Memory
Test. Reading, UK: Thames Valley Test.
Wilson, B. A., Evans, J. J., Emslie, H., Alderman, N., & Burgess, P. W. (1998). The
development of an ecologically valid test for assessing patients with dysexecutive
syndrome. Neuropsychological Rehabilitation, 8(3), 213-228.
Wolf, T. J., Stift, S., Connor, L. T., Baum, C., and The Cognitive Rehabilitation Research Group.
(2010). Feasibility of using the EFPT to detect executive function deficits at the acute
stage of stroke. Work, 36, 405-412.
87
Wood, J.N., & Grafman, J. (2003). Human prefrontal cortex: Processing and representational
perspectives. Nature Reviews Neuroscience, 4(2), 139-147.
World Health Organization. (2001). International Classification of Functioning, Disability and
Health. ICF, Geneva.
Yantz, C. L., Johnson-Greene, D., Higginson, C., & Emmerson, L. (2010). Functional cooking
skills and neuropsychological functioning in patients with stroke: An ecological validity
study. Neuropsychological Rehabilitation, 20(5), 725-738.
88
Appendix A: Behaviour Tracker Modes
(a) Configuration Mode
(b) Record Mode
89
(c) Editor Mode
(d) Viewer Mode
90
Appendix B: BMET Participant Package
Instructions
In this exercise you should complete the following three tasks:
1. You should do the following 6 things:
Collect something for the examiner* from the Main Information Desk (at the
Khedive Entrance) and do what is necessary
Buy 4 local stamps (considered 1 item)
Buy a birthday card
Buy a can of Coca-Cola
Telephone (Name) at (Phone
Number) and say where you are, who you are, and what time it is
Mail something to Dr. Dawson** at the University of Toronto.
2. You must meet me at the parrot cage 10 minutes after you have started the exercise and tell me
the time
3. You should obtain the following information and write it down in the spaces below:
What is the closing time of the resident‘s library on a Thursday?______________
What is the opening time of the gift shop on a Friday?______________________
What is the price of a Mars Bar?_______________________________________
How many entrances/exits are there on the main floor of Baycrest?____________
Tell me when you have completed the exercise.
While carrying out this exercise you must obey the following rules:
Rules
You should carry out all these tasks but may do so in any order
You should spend no more than $7.50
You should stay within the limits of the main floor of the hospital
You should not enter any of the hospital treatment areas or ―staff only‖ areas
You should not go back into an area you have already been in
You should buy no more than 2 items in the gift shop
Take as little time to complete this exercise without rushing excessively
Do not speak to us unless this is part of the exercise
*Your examiner is: **Dr. Dawson
University of Toronto
------------------------------ 500 University Ave., Suite 160
Toronto, Ont., M5G 1V7
91
Baycrest Centre for Geriatric Care
Toronto, Ontario
92
Appendix C: Meta
Appendix D: Metb
93
Appendix E: Metc
Appendix F: Metd
94
Appendix G: List of Behaviours Observed During the BMET
Beh
avio
urs
ob
serv
ed d
uri
ng
co
mple
tio
n o
f e
ach
tas
k o
n t
he
BM
ET
1. Buy a Birthday Card
1. goes to 0.99¢ card rack first
2. picks up 0.99¢ card
3. checks price (looks at back of 0.99¢ card)
4. checks price (looks at back of card from other card rack)
5. picks up card from other rack
6. touches card (at 0.99¢ card rack)
7. touches card (at other rack)
8. looks at other card rack at gift shop
9. puts card in bag
10. leaves cue to exchange card
11. checks message (look inside 0.99¢ card)
12. checks message (looks inside other rack card)
13. looks at other racks/display in gift shop
2. Buy 4 stamps
1. puts stamp in bag
2. buys 4 stamps
3. asks for stamps at lotto booth
4. asks for less than four stamps
3. Mars bar price
1. looks at Mars bar rack
2. writes down price of Mars bar
3. asks staff for Mars bar price
4. asks staff if they have chocolate bars
5. asks non-staff for Mars bar price
4. Opening Time of Gift
Shop on Friday
1. writes down opening time of gift shop on Friday
2. asks staff for opening time of gift shop on Friday
5. Buy Coke
1. buys Coke/diet Coke can
2. buys Coke/diet Coke bottle
3. puts Coke in bag
6. Collect something from
Information Desk and do
what's necessary
1. goes to Information Desk
2. asks specifically using name (Erin/Stephanie/Adrienne)
3. gives urgent letter to examiner
4. asks for something for the examiner
5. picks up something other than envelope
6. goes through Information Desk content
7. asks for something for him/herself
8. opens urgent envelope
9. puts envelope in bag
10. takes batteries
7. Mail Something to Deirdre
Dawson
1. writes in card outside of gift shop
2. addresses the envelope
3. puts stamp on card
4. mails card
5. mails using mailbox at Bathurst entrance
6. writes in card at gift shop counter
7. picks up something other than card to mail
8. mails something other than card to Deirdre
9. mails using mailbox outside of Apotex entrance or at Brain
Health Center
10. puts stamp on urgent letter collected at Information Desk to mail
11. licks self-adhesive stamp before putting on envelope
8. Phone Katherine and tell 1. gives name, time, location to Katherine
95
your name, time and location 2. calls using payphone at Bathurst entrance/near Information Desk
3. calls using courtesy phone at Information Desk/security
4. gives less than 3 pieces of information to Katherine
5. calls using payphone at Apotex or Brain Health Center or
courtesy phone at Apotex
6. uses personal phone
7. picks up Information Desk phone but doesn't use it
8. calls Katherine and waits for her to call back
9. Closing time of Library on
Thursday
1. writes down closing time of library on Thursday
2. asks staff for closing time of library on Thursday
3. goes to library on 2nd floor
10. Arrive at Parrot Cage in
10 minutes
1. goes to parrot cage
2. tells examiner time at parrot cage
3. arrives between 9-11 minutes at parrot cage
4. arrives late/early at parrot cage
5. stops to wait at parrot cage
6. does not tell time at parrot cage
11. Tell Examiner when Task
is Finished
1. tells examiner when finished test
2. tells examiner test is finished but continues to work on it
12. Total # of Entrances/Exits 1. asks staff for total entrances/exits
Beh
avio
urs
ob
serv
ed a
ny
tim
e du
rin
g t
he
com
ple
tion
of
the
BM
ET
1. Directional Behaviours
1. moves his head around after looking at map
2. moves his head around
3. asks staff for directions
4. goes outside Khedive/Apotex entrance
2. Talking Behaviours
1. asks for task-related help from staff
2. task-related self-talk
3. asks for task-related help from non-staff
4. talks to the examiner
5. causal talk with examiner
6. casual self-talk
7. casual talk with staff/non-staff/others
3. Monitoring Behaviours 1. checks watch
4. Other Observed Behaviours
1. stops and looks at painting
2. stops and looks at volunteer display rack next to cafeteria
3. drinks water from water fountain near Information Desk
4. looks at patient area/hall room
5. looks at the TV in patient area
6. puts map/task sheet clipboard in bag
5. Stopping Behaviours
1. stops to look at signage/billboard
2. stops to look at/mark task sheet
3. stops to look at/mark map
4. stops
6. Walking Behaviours 1. walking and looks at/mark task sheet
2. walking and looks at/mark map
96
Appendix H: Behaviour Classification
Task Specific Behaviours
Relevant Behaviours Irrelevant Behaviours
(includes apparent
habitual, distracters and
rule breakers) Efficient Inefficient
1.Buy a
birthday card
1.goes to 0.99¢ card rack first
2.picks up 0.99¢ card
3.checks price (looks at back of
0.99¢ card)
4.checks price (looks at back of
card from other card rack)
5.puts card in bag
1.picks up card from other rack
2.touches card (at 0.99¢ card rack)
3.touches card (at other rack)
4.looks at other card rack at gift
shop
5.leaves cue to exchange card
1.checks message (look
inside 0.99¢ card)
2.checks message (looks
inside other rack card)
3.looks at other
racks/display in gift shop
2.Buy 4 stamps 1.puts stamp in bag
2.buys 4 stamps
1.asks for stamp at lotto booth
2.asks for less than 4 stamps
3.Mars bar
price
1.looks at Mars bar rack
2.writes down price of Mars bar
3.asks staff for Mars bar price
1.asks staff if they have chocolate
bars
2.asks non-staff for Mars bar price
4.Opening time
of Gift Shop on
Friday
1.writes down opening time of
gift shop on Friday
2.asks staff for opening time of
gift shop on Friday
5.Buy Coke 1.buys Coke/diet Coke
2.puts Coke in bag
1.buys Coke/diet Coke bottle
6.Collect
something from
the Information
Desk and do
what is
necessary
1.goes to Information Desk
2.asks specifically using name
(Erin/Stephanie/Adrienne)
3.gives urgent letter to examiner
1.asks for something for the
examiner
2.picks up something other than
envelope (interpretation failure)
3.goes through Information Desk
content
4.asks for something for
him/herself
1.opens urgent envelope
2.puts envelope in bag
3.takes batteries from
Information Desk
7.Mail
something to
Deirdre
Dawson
1.writes in card outside of gift
shop
2.addresses the envelope
3.puts stamp on card
4.mails card
5.mails using mailbox at
Bathurst entrance
1.writes in card at gift shop counter
(socially inappropriate)
2.picks up something other than
card to mail (interpretation failure)
3.mails something other than card
to Deirdre (interpretation failure)
4.mails using mailbox outside of
Apotex entrance or at Brain Health
Center
5.puts stamp on urgent letter
collected at Information Desk to
mail (interpretation failure)
1.licks self-adhesive stamp
before putting on envelope
(habitual)
8.Phone
Katherine and
tell your name,
time and
location
1.gives name, time, location to
Katherine
2.calls using payphone at
Bathurst entrance/near
Information Desk
3.calls using courtesy phone at
Information Desk/Security
1.gives less than 3 pieces of
information to Katherine
2.calls using payphone at Apotex or
Brain Health Center or courtesy
phone at Apotex
3.uses personal phone to call
Katherine
4.picks up Information Desk phone
but doesn't use it
1.calls Katherine and waits
for her to call back
97
9.Closing time
of the library on
Thursday
1.writes down closing time of
library on Thursday
2.asks staff for closing time of
Library on Thursday
1.goes to library on second floor
10.Arrive at
Parrot Cage in
10 minutes
1.goes to parrot cage
2.tells examiner time at parrot
cage
3.arrives between 9-11 minutes
at parrot cage
1.arrives late/early at parrot cage
2.stops to wait at parrot cage
3.does not tell time at parrot cage
(interpretation failure)
11.Tell
examiner when
test is finished
1.tells examiner when finished
test
1.tells examiner test is finished but
continues to work on it
12.Total
entrances/exits
1.asks staff for total
entrances/exits
Non-task Specific Behaviours
Relevant behaviours Irrelevant Behaviours
(includes apparent habitual,
distracters and rule
breakers) Efficient Inefficient
1.Directional
behaviours
1.moves his head around after
looking at map
2.moves his head around
3.asks staff for directions to a
location
1.takes the elevator to go up
(rule break)
2.goes outside
Khedive/Apotex entrance (rule
break)
2.Talking
behaviours
1.asks staff for task-related help
2.task-related self-talk
1.asks non-staff for task-related
help
1.talks to the examiner (rule
break)
2.causal talk with examiner
(rule break)
3.casual self-talk (habitual)
4.casual talk with staff/non-
staff/others/parrot (habitual)
3.Other
observed
behaviours
1. checks watch 1.stops and looks at painting
(distracter)
2.stops and looks at volunteer
display rack next to cafeteria
(distracter)
3.drinks water from water
fountain near Information
Desk (distracter)
4.looks at patient area/hall
room (distracter)
5.looks at TV in patient area
(distracter)
6.puts map/task sheet
clipboard in bag (habitual)
4.Stopping
behaviours
1.stops to look at signage
2.stops to look at/mark task sheet
3.stops to look at/mark map
4.stops
5.Walking
behaviours
1.walking and looks at/mark
task sheet
2.walking and looks at/mark
map
