This article was downloaded by: [RMIT University] On: 07 September 2013, At: 02:24 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Neuropsychological Rehabilitation: An International Journal Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/pnrh20 Evidence-based practice guidelines for instructing individuals with neurogenic memory impairments: What have we learned in the past 20 years? Laurie A. Ehlhardt a , Mckay Moore Sohlberg b , Mary Kennedy c , Carl Coelho d , Mark Ylvisaker e , Lyn Turkstra f & Kathryn Yorkston g a The Teaching Research Institute-Eugene, Western Oregon University, Monmouth, USA b University of Oregon, Eugene, USA c University of Minnesota, Minneapolis, USA d University of Connecticut, Storrs, USA e College of Saint Rose, Albany, New York, USA f University of Wisconsin, Madison, USA g University of Washington, Seattle, USA Published online: 04 Jun 2008. To cite this article: Laurie A. Ehlhardt , Mckay Moore Sohlberg , Mary Kennedy , Carl Coelho , Mark Ylvisaker , Lyn Turkstra & Kathryn Yorkston (2008) Evidence-based practice guidelines for instructing individuals with neurogenic memory impairments: What have we learned in the past 20 years?, Neuropsychological Rehabilitation: An International Journal, 18:3, 300-342, DOI: 10.1080/09602010701733190 To link to this article: http://dx.doi.org/10.1080/09602010701733190
Transcript
This article was downloaded by: [RMIT University]On: 07 September 2013, At: 02:24Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK
NeuropsychologicalRehabilitation: An InternationalJournalPublication details, including instructions for authorsand subscription information:http://www.tandfonline.com/loi/pnrh20
Evidence-based practiceguidelines for instructingindividuals with neurogenicmemory impairments: Whathave we learned in the past 20years?Laurie A. Ehlhardt a , Mckay Moore Sohlberg b , MaryKennedy c , Carl Coelho d , Mark Ylvisaker e , LynTurkstra f & Kathryn Yorkston ga The Teaching Research Institute-Eugene, WesternOregon University, Monmouth, USAb University of Oregon, Eugene, USAc University of Minnesota, Minneapolis, USAd University of Connecticut, Storrs, USAe College of Saint Rose, Albany, New York, USAf University of Wisconsin, Madison, USAg University of Washington, Seattle, USAPublished online: 04 Jun 2008.
To cite this article: Laurie A. Ehlhardt , Mckay Moore Sohlberg , Mary Kennedy , CarlCoelho , Mark Ylvisaker , Lyn Turkstra & Kathryn Yorkston (2008) Evidence-basedpractice guidelines for instructing individuals with neurogenic memory impairments:What have we learned in the past 20 years?, Neuropsychological Rehabilitation: AnInternational Journal, 18:3, 300-342, DOI: 10.1080/09602010701733190
To link to this article: http://dx.doi.org/10.1080/09602010701733190
Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, orsuitability for any purpose of the Content. Any opinions and views expressedin this publication are the opinions and views of the authors, and are not theviews of or endorsed by Taylor & Francis. The accuracy of the Content shouldnot be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connectionwith, in relation to or arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Evidence-based practice guidelines for instructing
individuals with neurogenic memory impairments:
What have we learned in the past 20 years?
Laurie A. Ehlhardt1, McKay Moore Sohlberg2, Mary Kennedy3,Carl Coelho4, Mark Ylvisaker5, Lyn Turkstra6, and
Kathryn Yorkston7
1The Teaching Research Institute-Eugene, Western Oregon University,
Monmouth, USA; 2University of Oregon, Eugene, USA; 3University of
Minnesota, Minneapolis, USA; 4University of Connecticut, Storrs, USA; 5College
of Saint Rose, Albany, New York, USA; 6University of Wisconsin, Madison, USA;7University of Washington, Seattle, USA
This article examines the instructional research literature pertinent to teachingprocedures or information to individuals with acquired memory impairmentsdue to brain injury or related conditions. The purpose is to evaluate the avail-able evidence in order to generate practice guidelines for clinicians working inthe field of cognitive rehabilitation. A systematic review of the instructional lit-erature from 1986 to 2006 revealed 51 studies meeting search criteria. Studieswere analysed and coded within the following four key domains: PopulationSample, Intervention, Study Design, and Treatment Outcomes. Codingincluded 17 characteristics of the population sample; seven intervention par-ameters; five study design features; and five treatment outcome parameters.Interventions that were evaluated included systematic instructional techniquessuch as method of vanishing cues and errorless learning. The majority of the
Correspondence should be address to McKay Moore Sohlberg, Associate Professor/Communication Disorders and Sciences, 5285 University of Oregon Eugene, OR 97403
studies reported positive outcomes in favour of systematic instruction.However, issues related to the design and execution of effective instructionlack clarity and require further study. The interaction between the target learn-ing objective and the individual learner profile is not well understood. The evi-dence review concludes with clinical recommendations based on theinstructional literature and a call to clinicians to incorporate these methodsinto their practice to maximise patient outcomes.
Keywords: Evidence based practice guidelines; Memory impairment; Cogni-tive rehabilitation; Instruction; Errorless learning.
INTRODUCTION
This paper is one of a series of publications by the traumatic brain injury(TBI) Practice Guidelines Subcommittee established by the Academy ofNeurologic Communication Disorders and Sciences (ANCDS). The commit-tee was charged with generating evidence-based practice guidelines forcognitive-communication disorders resulting from TBI. This article examinesthe instructional research literature pertinent to teaching procedures or infor-mation to individuals with acquired memory impairments due to TBI orrelated conditions. The purpose is to evaluate the available evidence inorder to generate practice guidelines on instructional methods for cliniciansworking in the field of cognitive rehabilitation. (See www.ancds.org/practice.html for publications by this and other subcommittees.)
Why do clinicians need practice guidelines on instruction? Instruction iscentral to a variety of cognitive intervention approaches. Whether trainingthe use of compensatory systems or strategies or facilitating learning of factsor concepts, clinicians are charged with designing and delivering effectiveinstruction. Unfortunately, most rehabilitation professionals receive little train-ing in instructional theory and design. In a survey of medical speech-languagepathologists working in clinical settings throughout the United States, only one-third reported following an evidence-based instructional approach and nearlyhalf reported tallying data in their heads and making clinical decisions basedon intuition and experience (Lemoncello & Sohlberg, 2005).
Over the past 20 years, there has been mounting evidence that differentinstructional techniques can facilitate learning in individuals with memoryimpairment due to acquired neurological conditions. Also accumulating isevidence that neuronal plasticity can occur in response to structured input(e.g., Gonzalez-Rothi, 2001, 2006; Laatsch, Thulborn, & Krisky, 2004).Increasing our understanding and implementation of effective methods forstructuring input via instruction and training practices can facilitate experi-ence-dependent learning in cognitive-linguistic rehabilitation (Ducharme &Spencer, 2001; Sohlberg, 2006). In addition to improving rehabilitation
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 301
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
outcomes, there are pragmatic benefits to adhering to evidence-basedinstructional practices. In an era of reduced funding for cognitive rehabilita-tion, well-designed and delivered instruction may be key to facilitatingefficient and enduring positive outcomes (Sohlberg &Mateer, 2001; Ylvisaker,Hanks, & Johnson-Green, 2003).
Review of findings from previous meta-analyses
Special education possesses an abundant literature evaluating instructionaltechniques that is highly relevant to teaching people with acquired cognitiveimpairments. Several meta-analyses have attempted to parse out the mosteffective instructional practices and components within the special educationliterature (Adams & Engelmann, 2006; Kavale & Forness, 2000; Mastropieri,Scruggs, Bakken, &Whedon, 1996; Swanson, 1999, 2001; Swanson, Carson,& Sachse-Lee, 1996; Swanson & Hoskyn, 1998). Of particular interest to thecurrent paper are the series of meta-analyses by Swanson and colleagues(1996–2001). Following an extensive search yielding over 900 data-basedarticles, the authors selected 180 studies that met the inclusion criteria(e.g., comparison/control group; sufficient data to calculate effect sizes).These studies were categorised into one of four groups based on the instruc-tional techniques that were employed. The results indicated that theCombined Model that used both Direct Instruction and Strategy Instructiontechniques in concert produced the largest effect size. Strategy Instruc-tion alone, Direct Instruction alone, and Nondirect/Nonstrategy instructionshowed respectively smaller effect sizes. These meta-analyses are part of arich experimental literature within the field of special education that supportsthe use of explicit instructional techniques to effectively teach individualswith learning disabilities (Sohlberg, Ehlhardt, & Kennedy, 2005).
Strategy instruction is a specific instructional arena that special educationhas carefully evaluated. Graham and Harris (2003) conducted a meta-analysissupporting the effectiveness of Self-Regulated Strategy Development (execu-tive function/self-regulation scripts embedded within the curricular delivery)for a variety of students with and without disability. Similarly, Kim, Vaughn,Wanzek, and Wei (2004) provide a meta-analysis of graphic organisers usedwith a variety of students with and without disability. The burgeoning work inspecial education on current practices in strategy instruction has influencedthe brain injury field; however, this is an area that needs further attention.The current review of instructional practices limits its consideration of specificstrategy instruction due to the lack of research involving the brain injury popu-lation; clinicians and researchers are encouraged to examine the availablecross-disciplinary evidence. Executive function strategy and problem-solving interventions specific to the brain injury population are detailed in arecent practice guidelines manuscript (Kennedy et al., 2008 this issue).
302 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
The field of neuropsychology has built upon the special education researchand evaluated instructional design and teaching procedures in the populationof people with acquired neurogenic memory disorders. A meta-analysis eval-uated the effects of errorless learning and the method of vanishing cues across11 studies (Kessels & deHaan, 2003). This analysis revealed that error elim-ination strategies during the learning process produced a moderate effect sizeof .59 when compared to trial and error strategies. This analysis also showeda large effect size (.87) for studies specifically evaluating errorless learningand a small effect size (.27) for those evaluating the method of vanishingcues (.27); however, the relatively few studies in the latter category mayhave affected these results. Overall, this meta-analysis encourages moreevaluations of the application of errorless learning techniques to real worldsettings.
The current report provides a detailed review of the instructional literaturefrom 1986 to 2006 relevant to the treatment of the neurogenic population withacquired memory disorders. Because instruction represents a broad interven-tion domain applied to a wide variety of cognitive interventions, this paperdoes not utilise the standard classification system of practice standards,guidelines, and options (Miller et al., 1999); instead, it describes generalinstructional practices based on the research evidence.
METHODS
Writing Committee process
The ANCDS TBI Practice Guidelines Committee was formulated to generatepractice guidelines for working with persons living with the effects of trau-matic brain injury. The committee initially established a philosophy andrationale for the purpose and use of evidence-based guidelines (Kennedyet al., 2002) including identifying factors critical to scientific clinicaldecision-making (Ylvisaker et al., 2002). Priorities for domains of practiceguidelines were also established. To date, the committee has reviewed theliterature and generated evidence-based guidelines in the following clinicaldomains: use of direct attention training (Sohlberg et al., 2003); use of exter-nal aids to manage memory impairments (Sohlberg et al., 2007); use of stan-dardised assessment in traumatic brain injury (Turkstra et al., 2005);behavioural interventions (Ylvisaker et al., 2007); and executive functioninterventions (Kennedy et al., 2008 this issue). Each paper included Tablesof Evidence with agreed upon coding categories that allowed scrutiny andcomparison of research reports. Authors analysed the evidence in order todevelop clinical practice guidelines. Prior to submission, each paper wassent for external review to the ANCDS membership and to first authors
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 303
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
whose studies were analysed in the paper. Feedback was incorporated, and thepaper was submitted to the peer review process for the relevant journal. Wefollowed this protocol for the current guidelines paper. We received a total ofseven peer reviews. First author (and in one case, second author) reviewswere requested from researchers who had authored two or more papersincluded in the evidence base, comprising a total of six researchers. All sixreview solicitations were accepted and returned with comments, thus ensur-ing that the interpretation and coding of their studies was accurate.
The goal of this paper is to identify effective instructional practices in cog-nitive rehabilitation that the research literature supports with a reasonabledegree of certainty. In order to be inclusive in our search and evaluate theavailable evidence relevant to populations with functionally equivalentcognitive-linguistic impairments, we made the decision in this paper toinclude cross-population studies. Clients with traumatic brain injury, thetarget population of the guidelines subcommittee, represent a complex,heterogeneous group in which cognitive, communication, and behaviouralimpairments interact and are differentially affected by changing environ-ments. Because memory impairments following TBI are not a uniform con-dition specific to this diagnosis, it is important to consider other sources toverify treatment methods (Ylvisaker et al., 2002). Thus, the authors of thispaper reviewed literature that included the following aetiologies, all ofwhich commonly exhibit memory and/or learning impairments as a centralsymptom: acquired brain injury (TBI plus other neurogenic aetiologiessuch as cerebral vascular accidents, anoxic events, brain tumour, andneuro-infectious diseases), dementia, and schizophrenia. These categorieswere consistent with the breakdown in the research literature. The decisionto group a variety of aetiologies into an acquired brain injury category wasmade for two reasons. First, the review of the literature did not supportsystematic differences in the outcomes of intervention. Second, a numberof studies evaluated mixed participant groups, making it difficult to parseout aetiology-specific findings within the acquired brain injury population.Schizophrenia is not a neurogenic disorder; however, there is increasingevidence that this population experiences difficulties with new learning andmemory similar to people with acquired brain injury (McKenna, Clare, &Baddeley, 1995). Examination of the instructional literature for this popu-lation reinforces a number of instructional recommendations for peoplewith memory impairments from a broad spectrum of aetiologies.
Identifying, gathering, and extracting intervention studies
The following databases were searched for articles published between 1986and 2006: Academic Search Premier, Education Research Complete, ERIC,Medline, Psychology and Behavioural Sciences Collection, and PsycINFO.
304 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
We restricted our search to the past 20 years since this period included all ofthe primary instructional methods investigated with the target populations. Keycombinations of the following terms were used: brain injury, head injury,head trauma, brain trauma, dementia, schizophrenia, amnesia, memory,instruction, errorless learning, spaced retrieval, expanded rehearsal, distributedpractice, vanishing cues, trial and error, errorful, discovery learning, standardanticipation, backward-forward chaining, memory, executive function, dysex-ecutive, cognitive, computers, computer assisted instruction, and remediation.
The search methods initially yielded 2155 records narrowed to 857 recordsfor review. Hand searches of extant references (i.e., studies cited within anarticle) were also conducted. Abstracts or full articles were then reviewed,applying agreed upon selection criteria. Studies were included whose partici-pants had acquired memory impairment as their primary cognitive deficit dueto a variety of aetiologies, including acquired brain injury (ABI) (e.g., TBI,cerebrovascular accident, encephalitis), dementia (Alzheimer’s or vasculartype), or schizophrenia/schizoaffective disorder. Only studies that evaluatedthe application of instruction or training to the learning (or relearning) ofinformation or procedures were included. Studies also had to provide originalquantitative and/or qualitative (i.e., case study) data. When it was evidentthat two different studies (whether published separately or in the samearticle) evaluated the same participants, those participants were countedtwice. Studies were excluded that: (1) applied instruction or training pro-cedures to cognitive impairments other than memory (e.g., use of errorlesslearning to treat anomia); (2) reported findings from a study reviewedearlier using the same participants; or (3) consisted of reviews of other articleswithout presenting original data. Forty-eight studies met the above criteriaand were selected for final review. We incorporated three more studiesbased on first author reviews for a total of 51 studies.
As part of the ANCDS practice guidelines developed for individuals withAlzheimer’s dementia, Hopper et al. (2005) conducted a systematic review ofthe spaced retrieval literature for this population; hence, those studies werenot included in this review. However, studies evaluating spaced retrieval inthe TBI population were identified and included in this review, as well asthose studies within the dementia population that were not included in thereview by Hopper and colleagues.
Reviewing and coding studies into Tables of Evidence
Tables of evidence were organised by identifying relevant parameters withinthe following four key domains: population sample, intervention, studydesign, and treatment outcomes. Coding included 17 characteristics of thepopulation sample; seven intervention parameters; five study design features;and five treatment outcome parameters (see Table 1 for a summary of the
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 305
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
TABLE 1Summary of the number and percentage of studies reporting populationcharacteristics, intervention parameters, design features, and outcomes
(N ¼ 51)
Number and (percent) of studies reported
Type of evidence
Class I 1 (2)
Class II 6 (12)
Class III 26 (51)
Class IV 15 (29)
Class III & IV 3 (6)
Demographic variables
Number 51 (100)
Age 50 (98)
Male:Female 41 (80)
Aetiology 51 (100)
Site of injury 20 (39)
Time post-onset 39 (76)
Neuropsychological tests 51 (100)
Premorbid IQ 16 (31)
Initial severity 12 (24)
Memory severity 41 (80)
Dual diagnosis/co-morbidity 20 (39)
Selection criteria 29 (57)
Treatment history 9 (18)
Medication 11 (22)
Education 29 (57)
Occupation 19 (37)
Living Situation 28 (55)
Intervention
Instructional approach 51 (100)
Additional components 24 (47)
Treatment targets I ¼ 33 (65) P ¼ 10 (20)
I & P ¼ 8 (17)
Treatment dosage 51 (100)
Treatment setting 25 (49)
Treatment provider 26 (51)
Measurement 51 (100)
Study design
Design 51 (100)
Statistics 41 (80)
Reliability/validity 7 (14)
Outcomes
Ecological validity 14 (27)
Immediate outcomes 100 (100)
Generalisation 22 (43)
Maintenance 30 (59)
I ¼ Information; P ¼ Procedures.
306 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
number and percentage of studies reporting population characteristics,intervention parameters, design features, and outcomes). The full Tables ofEvidence are presented in three appendices. Appendix A displays the codedpopulation sample characteristics, Appendix B displays the coded interven-tion characteristics, and Appendix C displays the coded parameters relatedto study design and outcomes. There are three possible types of coding.Some parameters list the relevant study content; other parameters contain a“1” indicating the study reported the parameter or a “0” indicating thestudy did not report it; and, as described later, intervention outcomes arecoded with þ, /, or – indicating positive, qualified, or negative findings,respectively. Each appendix groups the three diagnostic categories separately(ABI—38 studies; dementia—7 studies; and schizophrenia/schizoaffectivedisorder—6 studies) to allow for population-specific analyses. Studieswithin each of the three diagnostic groups are ordered chronologically inthe appendices
Clinical practice recommendations must strongly consider the strength ofthe research evidence. Therefore, in Appendix C, each study was also ratedaccording to the revised American Academy of Neurology (AAN) classifi-cation system (i.e., Class I–IV evidence) (American Academy of Neurology,2004; www.aan.org). In this system, Class I studies include prospective,randomised controlled clinical trials with masked outcome assessment in arepresentative population with qualifiers. Class II studies are prospectivematched group cohort studies with masked outcome assessment in a represen-tative population that meet Class I criteria or randomised controlled trials thatlack one criterion for Class I evidence. Class III studies include all other con-trolled trials where outcome assessment is independent of patient treatment ina representative population. Class IV is evidence from uncontrolled studies,such as case reports or expert opinion.
The first author coded all 51 intervention studies. Reliability coding withthe second author was conducted for 26 out of the total of 51 studies (51%)for all coding categories, except for five parameters: the type of instructionand four of the outcome components (immediate outcomes, ecological val-idity, generalisation and maintenance). The second author coded these fourparameters for all 51 studies. There were nine points of disagreementrelated to categorising types of instruction (e.g., discriminating betweenerrorless learning and systematic instructional packages), eight points of dis-agreement related to determining immediate outcomes (e.g., discriminatingbetween a “qualified” versus “negative” outcome), one point of disagreementrelated to the determination of ecological validity, and no points of disagree-ment related to generalisation and maintenance outcomes. Inter-raterreliability for these four parameters averaged 90%, with overall reliabilityat 98% for the 26 studies reviewed in their entirety. Disagreements wereeasily resolved by discussion, with components re-categorised accordingly.
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 307
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
RESULTS
Population sample characteristics
This section summarises relevant population sample characteristics acrossthe three population groups (see Appendix A for details).1
Number of participants, age, gender, aetiology, site of injury,time post-onset
Acquired brain injury. There were 451 experimental participants withmemory impairment, 42 control participants with cognitive disabilities, and163 non-disabled control participants across the 38 studies. Ages rangedfrom 18 to 78 years, with two studies involving children 8–11 years old.The experimental participants were predominantly adult males (.2:1). Theaetiology of memory impairment included a wide range of conditions: TBI,CVA (including anterior and posterior communicating artery haemorrhage),encephalitis, brain tumour, Korsakoff’s syndrome, anoxia, brain abscess,Parkinson’s disease, toxicity, encephalopathy, and cerebral infection. Siteof injury and/or imaging findings were reported in 17/38 (45%) of thestudies. Time post-onset (TPO) was reported in 31/38 (82%) of the studieswith the majority of participants at least one year post-onset and othersseveral years post-onset (e.g., 15 years). Only three studies reported on par-ticipants with symptom onset of less than one year (Dou, Man, Ou, Sheng,& Tam, 2006; Glisky & Delaney, 1996; Wilson, Baddeley, Evans, & Shiel,1994).
Dementia. Across the seven studies, there were 55 experimental partici-pants and eight non-disabled control participants, ranging in age from 65to 89 years. The participants were predominantly male, with Alzheimer’sdementia as the most common aetiology, followed by vascular dementia.Imaging findings were reported in 3/7 (43%) of the studies (Clare et al.,2002; Haslam et al., 2006; Winter & Hunkin, 1999). Symptom onset date wasreported in 2/7 (29%) of the studies, and ranged from 18 months to 5 yearsprior to the study (Clare et al., 2000; Haslam, Gilroy, Black, & Beesley, 2006).
Schizophrenia/schizoaffective disorder. Across the six studies, therewere203 participants with schizophrenia/schizoaffective disorder and memoryimpairment, 21 control participants with schizophrenia/schizoaffective
1To limit manuscript space devoted to citations in the results section, the first two authors
selected and cited the two most representative studies to illustrate certain evidence parameters.
For objective evidence parameters (e.g., demographics) with more than three studies, percen-
tages were reported without citations.
308 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
disorder without memory impairment, and 88 non-disabled controlparticipants. The experimental participants were predominantly male, andages ranged from 18 to 55 years, with the majority of participants in the30–40 year age range. Site of injury/imaging findings were not reported forthis population. Symptom onset date was determined by the date of firsthospitalisation relative to the start of the study, which ranged from 1 monthto 34.7 years prior.
Acquired brain injury. Neuropsychological test data were available in all38 studies; however, the type and number of measures used varied consider-ably. Premorbid IQ was reported in 10/38 (26%) of the studies. Initial sever-ity (length of coma and/or post-traumatic amnesia) was reported in 12/38(32%) of the studies and was not relevant to participants in several studiesgiven the aetiology (e.g., Korsakoff’s syndrome). Severity of memory impair-ment was reported in 33/38 (87%) of the studies. Of these, 18/33 (55%)reported the level of impairment as “severe”, “profound”, “marked”, “significant”,or “chronic, affecting everyday memory”. Ten out of 33 (30%) of the studiesincluded participants with a range of severity levels: “mild”, “moderate”, and“severe”, while in other studies, severity was implied on the basis of cut-offscores. Few studies clearly linked performance on standardised memoryassessments and screening tools to the described levels of memory impair-ment (e.g., Page, Wilson, Shiel, Carter, & Norris, 2006; Tailby & Haslam,2003). In addition, few studies described the specific type of memory impair-ment. Descriptors reported typically included “anterograde” or “episodic”memory impairments (e.g., Andrewes & Gielewski, 1999; Komatsu,Mimura, Kato, Wakamatsu, & Kashima, 2000).
Dual-diagnosis/co-morbidity data were reported in 17/38 (45%) of thestudies. In addition to the memory impairments described above, other cog-nitive impairments reported included: executive function and attentionimpairments; disorientation; anomia and other language impairments; visuo-perceptual disorders; motor speech impairments or motor slowing; diabetes;and/or seizures (e.g., Glang, Singer, Cooley, & Tish, 1992; Glisky, Schacter,& Tulving, 1986a, 1986b). For several of the remaining studies, participantswere excluded on the basis of a dual-diagnosis or co-morbidity. (Note: As sofew studies reported on vision, hearing, and motor status, these data wereincluded in this category when available.) Selection criteria were reportedin 16/38 (42%) studies.
Dementia. Neuropsychological test data were available across all sevenstudies. Premorbid IQ was reported in 6/7 studies (86%). Two of the seven
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 309
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
studies included only participants with minimal or mild dementia (Clare,Roth, Wilson, Carter, & Hodges, 2002; Clare et al., 2000), while the remain-ing studies included those with mixed severities or moderate–severe demen-tia only (e.g., Metzler-Baddeley & Snowden, 2005; Ruis & Kessels, 2005).Six of the seven studies (86%) described severity of memory impairmentwith participants having “significant”, “severe”, or “profound” memoryimpairments. Dual-diagnosis/co-morbidity data were provided in 3/7(43%) studies (Haslam et al., 2006; Metzler-Baddeley & Snowden, 2005;Winter & Hunkin, 1999). In addition to memory impairment, other cognitiveimpairments included general decline in intellectual ability, impaired concen-tration, disorientation, and word-finding difficulties (Haslam et al., 2006;Metzler-Baddeley & Snowden, 2005; Winter & Hunkin, 1999). Selection cri-teria were reported in 6/7 (86%) studies.
Schizophrenia/schizoaffective disorder. Neuropsychological testing wascompleted in all six studies; however, the number of measures used was oftenlimited, thus precluding detailed descriptions of participant profiles. PremorbidIQ was not reported for this population group. Severity of memory impair-ment was reported in only two (33%) of the studies and determined viacut-off scores (Kern et al., 2005; O’Carroll, Russell, Lawrie, & Johnstone,1999). Selection criteria were reported in 6/6 (100%) studies.
Acquired brain injury. Treatment history was reported in 9/38 (24%) ofthe studies. Medication status was rarely reported – 2/38 (1%) studies(Andrewes & Gielewski, 1999; Parkin, Hunkin, & Squires, 1998). Educationlevels were reported in 21/38 (55%) of the studies with the majority of theadult participants completing high school education, and many achievingsome level of post-secondary education as well. Occupational history wasavailable in 13/38 (34%) of the studies, and included reports of participantemployment post-intervention (e.g., Andrewes & Gielewski, 1999; Hillaryet al., 2003). Post-injury living situation was reported in 18/38 (47%)studies, and included home/community dwelling, assisted living centres,and long-term care residence.
Dementia. Treatment history was not reported. Medication status wasreported in 3/7 studies (43%) (Clare et al., 2002; Dunn & Clare, 2007;Metzler-Baddeley & Snowden, 2005). Education status was reported in 2/7(29%) studies (Haslam et al., 2006; Ruis & Kessels, 2005), with participantshaving completed up to 10 years of education. Occupational history wasreported in 4/7 (57%) studies, with participants having worked in a variety
310 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
of jobs (e.g., business owners, clerical, engineers) prior to their retirement.Living situation was reported in 6/7 (86%) studies, with the majority of par-ticipants living with a family member.
Schizophrenia/schizoaffective disorder. Treatment history was notreported. Medication status was reported in all six studies and includedanti-psychotic and neuroleptic medications. Education levels in all sixstudies were reported, with the majority of participants completing at leasthigh school education. Occupational history was reported in only 1/6(17%) studies (Kern, Liberman, Kopelowicz, Mintz, & Green, 2002).Living status was not consistently reported but was implied on the basis ofthe described treatment setting (e.g., in-patient hospital setting; outpatientclinic, community dwelling).
To summarise, the description of population characteristics varied widelywithin and between the three population groups. The majority of the studiesprovided sufficient information to develop a basic profile of study partici-pants; however, there were notable gaps in the literature. For example,while the majority of the studies included neuropsychological testing, clearlinkages between assessment data and the determination of both the typeand severity of memory impairment were limited. This trend in combinationwith the relative lack of information in other domains such as treatmenthistory and medication status limit the extent to which results can be gener-alised to individuals beyond those involved in the studies (i.e., externalvalidity).
Intervention
This section summarises the types of instructional methodologies, additionaltraining components, treatment targets, dosage, treatment settings and provi-ders, as well as outcome measurements across the 51 studies (see Appendix Bfor details).
Instructional methods
Two broad instructional categories emerged from the review: (1) systema-tic instructional methods (e.g., method of vanishing cues, errorless learning,and spaced retrieval); and (2) conventional methods (e.g., errorful learning/trial and error learning). The primary goal of errorless learning is to eliminateerrors during the acquisition phase of learning by providing models beforea client attempts a response; guessing is discouraged (Baddeley & Wilson,1994). The method of vanishing cues (MVC) is a form of chaining thatprovides the client with progressively stronger or weaker cues followingrecall attempts of the targeted information/procedures (Glisky, Schacter, &Tulving, 1986a, b). Spaced retrieval (i.e., expanded rehearsal) is a form of
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 311
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
distributed practice that provides individuals with severe memory impairmentpractice at successfully recalling information over expanded time intervals(Melton & Bourgeois, 2005). Spaced retrieval incorporates errorless learningbut follows a prescribed practice regimen. In general, these systematicinstructional methods emphasise explicit, carefully faded models/prompts.The conventional methods (i.e., trial-and-error learning, errorful learning,standard anticipation) emphasise recall of the targeted information or skillwithout prior models/prompts, with the trainer providing the models onlyfollowing errors.
Studies were grouped into the following six intervention categories andcomparison conditions:
1. Errorless learning (EL). The study evaluated errorless learning with nocomparison condition or group.
2. Errorless learning versus errorful learning (EL vs. EF). The studycompared EL vs. EF, a comparison group and/or an earlier EL trainingprotocol.
3. Method of vanishing cues (MVC). The study evaluated MVC with nocomparison condition or group.
4. Method of vanishing cues versus errorful or errorless learning (EF/EL). The study compared MVC with a comparison condition, such aserrorful learning or errorless learning.
5. Spaced retrieval/spaced presentations (SR). The study primarily eval-uated spaced retrieval or spaced presentation of information.
6. Systematic instructional packages. The study evaluated a combinationof systematic techniques integrated into an instructional package and/or used a staged-learning process (e.g., Phase 1 ¼ acquisition phase,Phase 2 ¼ application phase) with or without a comparison con-dition/group.
The target systematic instructional methodologies varied across the threeaetiology groups as described in the following section.
Acquired brain injury. The target systematic instructional methodologiesvaried for the ABI group, with comparisons of EL versus EF and systematicinstructional packages as the predominant methods (11 and 13 studies,respectively); one study evaluated EL without a comparison condition/group (Parkin et al., 1998). There were five studies evaluating MVC withno comparison and five with a comparison, while three studies focusedprimarily on spaced retrieval/spaced presentations.
312 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
Dementia. Three studies compared EL versus EF (Haslam et al., 2006;Metzler-Baddeley & Snowden, 2005; Ruis & Kessels, 2005), three evaluatedsystematic instructional packages (Clare et al., 2000, 2002; Dunn & Clare,2007), and one study evaluated EL alone (Winter & Hunkin, 1999).
Schizophrenia/schizoaffective disorder. Five studies compared ELversus EF, while one study evaluated two different systematic instructionalpackages with a control group (Young et al., 2002).
Across groups, documentation of specific training procedures for eachinstructional methodology varied. Several studies provided sufficient detail toallow for replication (e.g., Andrewes & Gielewski, 1999; Glisky & Schacter,1989; Hunkin, Squires, Aldrich, & Parkin, 1998).
Additional instructional components
Several studies emphasised instructional components the researchershypothesised would increase the participants’ active processing of thetargeted information/procedures, while keeping errors to a minimum.These components took the form of either a strategy (e.g., verbal elaboration,imagery, prediction-reflection, evaluative questioning/self-generation ofresponses) or an emphasis on stimulus manipulation (e.g., varied trainingexamples for the targeted information/task step or stimulus pre-exposure).
Acquired brain injury. Eight studies included a strategy component (e.g.,Evans et al., 2000; Tailby & Haslam, 2003), while 12 studies emphasisedstimulus manipulation (e.g., Glisky & Schacter, 1987; Stark, Stark, &Gordon, 2005).
Dementia. Almost half of the studies (3/7) included a strategy com-ponent (Clare et al., 2000, 2002; Metzler-Baddeley & Snowden, 2005),while none of the studies explicitly described stimulus manipulation.
Schizophrenia/schizoaffective disorder. One out of the six studies (17%)emphasised strategy instruction (Young, Zakzanis, Campbell, Freyslinger, &Meichenbaum, 2002), while four of the studies described stimulus manipu-lation (e.g., Kern et al., 2002, 2005).
Instructional targets
Instructional targets were grouped into two categories: information andprocedures. Information targets included: word lists, face–name associations,facts, object names, definitions, concepts, and curricula content (e.g., maths,reading). Procedures included a variety of multi-step tasks, such as: indexcard filing, word processing, data entry, programming electronic aids,
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 313
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
external memory aid use, and route finding. Familiarity of the instructionaltargets varied with both unfamiliar and/or familiar information and pro-cedures trained across the three population groups.
Acquired brain injury. Most of the studies (24/38; 63%) focused ontraining information, while 7/38 (18%) targeted procedures, and 7/38(18%) targeted both.
Dementia. The majority of the studies (6/7; 86%) targeted informationwhile one study (Clare et al., 2000) evaluated instruction that targeted bothinformation and procedures.
Schizophrenia/schizoaffective disorder. Half of the studies (3/6) tar-geted information and the other half taught procedures.
Treatment dosage
Treatment dosage refers to the frequency of the treatment sessions and theduration of treatment, both in terms of length of session and the total amountof time participants received treatment.
Acquired brain injury. Treatment frequency ranged from one sessiononly to daily sessions. Treatment duration was also quite varied, with sessionslasting a minimum of 30 minutes up to 2 hours with total duration, whenspecified, from one week up to several months.
Dementia. Treatment frequency ranged from a minimum of one trainingsession in each condition (2 total) up to 16 sessions. Total duration, whenreported, ranged from one week up to four weeks.
Schizophrenia/schizoaffective disorder. Treatment frequency rangedfrom one session only up to six sessions, with durations up to four weeks.
Treatment settings and providers
Treatment settings and treatment providers were not consistently reportedacross the three population groups (49% and 53%, respectively).
Acquired brain injury. Half of the studies (19/38) reported treatmentsettings. These studies described a range of settings, including laboratory orclinic settings, work-sites, and home-based treatment delivered via phone(Melton & Bourgeois, 2005). Treatment providers were reported in 20/38(53%) of the studies and included experimenters and family members, as
314 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
well as computer-delivered stimuli, common to the studies by Glisky andcolleagues (e.g., 1986–1995).
Dementia. Treatment settings and providers were reported in 1/7 (14%)of the studies (Metzler-Baddeley & Snowden, 2005) that described a combi-nation of clinic and home-based therapy delivered by both the experimenterand spouses.
Schizophrenia/schizoaffective disorder. Treatment settings and provi-ders were reported in 5/6 (83%) of the studies, with the majority conductedin hospital or clinic settings. Experimenters, supervisors and assistantsprovided the treatment.
Measurement
Different types of intervention outcome measurements were used acrossthe three population groups, with the majority of studies incorporatingmore than one type. The most frequently used measure was the number,percentage, or proportion of targets correctly recalled. A total of 39/51(77%) of the entire sample included this form of measurement with thepopulation-specific breakdown as follows:
Other outcome measures used across the three groups included documen-tation of error responses or intrusions (e.g., Glisky & Schacter, 1987;Komatsu et al., 2000; Leng et al., 1991), task completion times (e.g., Glisky,1995; Glisky & Schacter, 1987; Leng, Copello, & Sayegh, 1991), levels ofindependence (Andrewes&Gielewski, 1999), number of trials/sessions to cri-terion (e.g., Ehlhardt, Sohlbert, Glang, & Albin, 2005; Glisky & Schacter,1988; Turkstra & Bourgeois, 2005), behavioural checklists/questionnaires(e.g., Hunkin, Squires, Aldrich, & Parkin, 1998; Ownsworth & McFarland,1999; Squires, Hunkin, & Parkin, 1996) and standardised assessments (e.g.,Dou et al., 2006; Schmitter-Edgecombe, Fahy, Whelan, & Long, 1995;Winter & Hunkin, 1999). Qualifiers to these measurement systems frequentlyincluded whether the measure was implemented immediately following train-ing or after a delay, whether recall was free or cued, and whether a recognitiontask was also included. These qualifiers were particularly relevant to thosestudies in which the determination of the different memory systems (e.g.,implicit versus explicit memory) underlying performance outcomes was of
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 315
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
primary interest (e.g., Hunkin, Squires, Parkin, & Tidy, 1998; Tailby &Haslam, 2003). It was generally unclear whether or not evaluators werenaı̈ve to the training conditions when measuring outcomes; hence, for themajority of the studies, it was assumed that the trainer also served as theevaluator.
To summarise, the systematic instructional methods evaluated within andbetween aetiology groups varied, the majority of the studies targeting error-less learning or systematic instructional packages. The studies also varied inthe extent to which the treatment procedures and dosages were sufficientlydetailed to allow a naı̈ve trainer to replicate the treatment, which is importantfor establishing external validity. Treatment targets spanned a range of infor-mation and procedures. A number of studies evaluated word recall, whereasothers selected functional targets, such as: face–name recall, computer taskcompletion, and external memory aid use. The most common outcomemeasure was the number-percentage-proportion of targets correct whileother measures were specifically tailored to the treatment target (e.g., taskcompletion time for multi-step procedures). Lack of explicit information con-cerning the evaluators’ knowledge of training conditions limits assessment ofinternal validity. Design factors influencing internal validity will be discussedin the next section.
Study design
This section summarises the classes of evidence, research design, statistics,reliability-validity, and ecological validity (see Appendix C for details).
Classes of evidence
As previously stated, each study was rated according to the revisedAAN classification system (i.e., Class I–IV evidence) (American Academyof Neurology, 2004) across population groups as follows. Table 2 showsclassification of research evidence across the three population groups.
This section summarises the methodological variables coded, including:research design and experimental control conditions, statistics, reliability-validity, and ecological validity. These components are analysed separatelyfor each population group. For this review, the authors determined that eco-logical validity had been established when the intervention targets would beof direct benefit to the study participants in their daily lives (e.g., teaching theuse of an external memory aid or a vocational task the person would poten-tially utilise).
316 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
Acquired brain injury. Twenty-six of the 38 studies (68%) incorporatedexperimental control conditions. There were three between-groups designswith random assignment. Experimental comparison conditions included:baseline comparisons (Ownsworth & McFarland, 1999), memory notebooktraining versus support group intervention (Schmitter-Edgecombe et al.,1995), and computer versus therapist-delivered treatment versus controlgroup (Dou et al., 2006). There were 12 within-subject experimentalstudies. Comparison conditions included EL versus EF (e.g., Baddeley &Wilson, 1994; Evans et al., 2000), EL versus MVC or different treatmentor presentation conditions (e.g., Glisky et al., 1986a; Komatsu et al.,2000), and baseline comparisons (Ehlhardt et al., 2005). Eight studiesused combinations of both between and within-subject designs with com-parison conditions similar to the above. Several studies reported counter-balanced treatment conditions and stimuli as appropriate to the design.Three studies used combinations of between and/or within-subjectdesigns and case studies (Glang et al., 1992; Glisky & Delaney, 1996;Squires et al., 1996). Twelve studies did not use experimental controland included case studies with and without pre–post treatmentcomparisons.
TABLE 2Classification of research evidence
Number (percent)
Acquired brain injury
(N ¼ 38) Class I 0/38 (0)
Class II 1/38 (3)
Class III 21/38 (55)
Class IV 14/38 (37)
Combinations 2/38 (5)
Dementia
(N ¼ 7) Class I 0/7 (0)
Class II 0/7 (0)
Class III 5/7 (71)
Class IV 1/7 (14)
Combinations 1/7 (14)
Schizophrenia/schizoaffective disorder
(N ¼ 6) Class I 1/6 (17)
Class II 5/6 (83)
Class III 0/6 (0)
Class IV 0/6 (0)
Combinations 0/6 (0)
“Combination” refers to those studies that included various combinations of
Class I–IV designs.
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 317
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
Levels of statistical significance were reported in 29/38 (76%) studies withprocedures including t-tests, ANOVAs, general linear model analyses, as wellas non-parametric procedures. Statistics were not appropriate for selectedwithin-subject studies that used visual analysis to determine effectiveness(e.g., Ehlhardt et al., 2005; Glang et al., 1992). Only four (11%) of thestudies reported reliability and/or validity data related to measures (e.g.,Ehlhardt et al., 2005; Melton & Bourgeois, 2005). Ecological validity wasconsidered established in 12/38 (32%) of the studies (e.g., Andrewes &Gielewski, 1999; Glisky & Schacter, 1987).
Dementia. Four of the seven studies used within-subject designs thatincluded comparisons of EL vs. EF (Clare et al., 2000; Metzler-Baddeley &Snowden, 2005; Ruis & Kessels, 2005) and use of a control set of stimulusitems (Clare et al., 2002). Haslam and colleagues (2006) used bothbetween and within-subjects comparisons when evaluating EL vs. EF andcontrol group performance. Clare and colleagues (2000) used a combinationof experimental, within-subjects (stable baseline as control), and a case studydesign. Winter and Hunkin (1999) used a non-experimental, pre–post com-parison study of EL. Levels of statistical significance were reported in 5/6(83%) of the studies and included t-tests, ANOVAs, correlational analyses,and time-series analyses. Reliability and validity were not reported in anyof the studies. Ecological validity was established in 1/7 (14%) of studies(Clare et al., 2000), in which participants demonstrated improved face–name recall and use of memory strategies in naturalistic settings.
Schizophrenia/schizoaffective disorder. Three of the six studies (50%)used between-group experimental designs comparing EL vs. EF, with andwithout a control group, and EL vs. symptom management (Kern et al.,1996, 2002, 2005). The three other studies used between and within-subjectscomparisons (O’Carroll et al., 1999; Pope & Kern, 2006; Young et al., 2002).Young and colleagues (2002) compared two forms of systematic instruction:scaffolded instruction versus direct instruction, and included a control group.Four of the six studies (67%) used randomised assignment (Kern et al., 1996–2005; Young et al., 2002). Levels of statistical significance were reported inall of the studies and included t-tests, ANOVA, MANOVA, correlational pro-cedures and non-parametric tests. Reliability and validity were reported in 3/6 (50%) of the studies (Kern et al., 2005; Kern et al., 2002; Young et al., 2002)with one study including fidelity of implementation data (Kern et al., 2005).Ecological validity was incorporated in one of the six studies (social problemsolving—Kern et al., 2005).
To summarise, the research designs used across the three aetiology groupsincluded between-group and within-subject designs. These different studydesigns vary in the extent to which they establish external and internal
318 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
validity. Notably, few studies documented reliability and validity procedures,limiting the extent to which changes in the dependent variables could be con-fidently linked to the targeted intervention (i.e., internal validity). Specificoutcomes associated with the targeted interventions are reviewed in thenext section.
Treatment outcomes
This section summarises immediate outcomes and generalisation and main-tenance of results (see Appendix C for details).
We identified three different categories of immediate outcomes thatdescribed the research results reported in the literature. The immediate out-comes were coded as favourable (designated by the þ symbol in AppendixC evidence table) if the instructional method evaluated in the study wasreported to produce positive learning in the dependent variables and/or sig-nificantly stronger learning than a control instructional condition. Outcomeswere coded as qualified (designated by the/symbol) if the instructionalmethod was reported to produce variable learning on the dependent measures.Outcomes were coded as negative (using the – symbol) if the target instruc-tional method was found to produce limited or no advantage to learning in thedependent variables.2
Several studies, particularly those published in the 1980s and early 1990s,focused on the basic question of whether or not a particular systematicinstructional method worked and/or resulted in better performance thananother method (e.g., MVC – Glisky et al., 1986; EL vs. EF – Baddeley &Wilson, 1994). A trend in later studies emphasised the evaluation of enhance-ments to previously validated methods (e.g., EL with pre-exposure – Kalla,Downes, & van den Broeck, 2001; EL with self-generated responses – Tailby&Haslam, 2003). Accordingly, the outcomes of these later studies were codedby whether or not the established instructional method was again shown to beeffective as well as by the effectiveness of the hypothesised enhancement.
Immediate outcomes
The outcomes are analysed across the three population groups in order toevaluate global trends. A broad analysis of the aggregate immediate outcomesrevealed solid evidence supporting the use of systematic instructional tech-niques. A total of 41 out of 51 (80%) studies reported favourable learning
2Outcome decisions were based on statistically-analysed group data, individual participant
data, or both depending on each study’s research design and primary mode of analysis. The
authors acknowledge that for studies with group data only, individual variations in performance
masked by statistical analysis might have resulted in a different outcome assessment had
individual data also been available.
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 319
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
outcomes using the instructional methods of errorless learning, method ofvanishing cues, spaced retrieval, or systematic instructional packages.Favourable outcomes occurred in each population group with the leastrobust outcomes for the dementia population. In the ABI group, 34 out of38 (89%) studies reported favourable immediate outcomes, four reportedqualified outcomes, and one study that included children reported negativeoutcomes (Landis et al., 2006). Glang and colleagues (1992) also includedchildren, but reported positive findings. The two studies differed considerablyin terms of number of participants, severity levels, systematic instructionalmethod (errorless learning or systematic instructional package, respectively),and study design. In the dementia group, two out of seven studies (29%)reported favourable immediate outcomes (Clare et al., 2000; Winter &Hunkin, 1999), while four studies reported qualified outcomes and onestudy a negative outcome (Dunn & Clare, 2007). With the exception ofKern et al. (2002) who reported qualified findings, five out of the six (83%)studies with participants who have schizophrenia reported favourableimmediate outcomes (e.g., Kern et al., 2005; Pope & Kern, 2006).
Studies varied by type of instruction and outcome measures used to docu-ment treatment effects. The two studies evaluating EL only reported favour-able outcomes (Parkin et al., 1998; Winter & Hunkin, 1999). In the 20 studiescomparing EL with EF and/or a control group, 14 (56%) reported favourableoutcomes (e.g., Squires, Hunkin,&Parkin, 1997;Van der Linden,Meulemans,& Lorrain, 1994; Young et al., 2002), with five studies reporting qualifiedfindings (e.g., Evans et al., 2000; Metzler-Baddeley & Snowden, 2005;O’Carroll et al., 1999) and one study reporting negative outcomes (Landiset al., 2006). All five studies evaluating MVC only reported favourable out-comes (e.g., Baddeley & Wilson, 1994; Glisky & Schacter, 1988; Gliskyet al., 1986b). In the studies that compared MVC to another instructional con-dition, three out of five reported favourable findings (Glisky et al., 1986a;Leng et al., 1991; Riley, Sotirious, & Jaspal, 2004) and two reported qualifiedfindings (Hunkin & Parkin, 1995; Thoene & Glisky, 1995). All three studiesevaluating spaced retrieval/spaced presentations reported favourable outcomes(Hillary et al., 2003; Melton & Bourgeois, 2005; Turkstra & Bourgeois, 2005).The majority of the studies (16/17; 94%) evaluating systematic instructionalpackages reported favourable outcomes (e.g., Andrewes & Gielewski, 1999;Schmitter-Edgecombe et al., 1995; Winter & Hunkin, 1999).
Generalisation
Generalisation outcomes were coded for those studies that evaluated trans-fer of training to non-trained targets and/or to natural settings. Studies werecoded as positive (þ) if any degree of generalisation was reported and codedas negative (–) if generalisation was measured but not achieved.
320 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
Acquired brain injury. Generalisation measures were taken in 19 out of38 (50%) treatment studies. Of those studies, 15/19 (79%) reported partialor 100% generalisation of treatment targets. For example, in the study byEhlhardt and colleagues (2005), generalisation for learning the e-mail pro-cedures was reported when participants were able to utilise a slightly morecomplex e-mail interface. Other studies reported generalisation from labora-tory to naturalistic settings, while in some cases training in the naturalenvironment also facilitated generalisation (e.g., Andrewes & Gielewski,1999; Glisky & Schacter, 1987).
Dementia. Generalisation was reported in one of the seven studies. Clareand colleagues (2000) reported generalisation of face–name recall to a natur-alistic setting for two of the six participants and use of memory strategies innew situations for one participant.
Schizophrenia/schizoaffective disorder. Generalisation measures weretaken in two of the six studies (33%). Kern and colleagues (2005) reportedthat the experimental group generalised trained social problem-solvingskills to novel problem-solving scenarios. Young and colleagues (2002)noted that the scaffolded instruction group demonstrated significant improve-ment on selected generalisation assessment measures (e.g., object sortingtask).
Maintenance
Maintenance outcomes were coded when re-assessment of trained targetsoccurred more than one day following the cessation of training since that wasthe minimum time reported by authors as “maintenance”.
Acquired brain injury. Maintenance checks were reported in 24 out of38 (63%) of the studies. All 24 studies reported partial or 100% maintenanceof treatment targets from a few days to nine months post-intervention(e.g., Andrewes & Gielewski, 1999; Thoene & Glisky, 1995).
Dementia. Maintenance checks were reported in two of the seven studies(Clare et al., 2002, 2000) and revealed generally positive findings with reten-tion of therapy gains up to six months post-intervention.
Schizophrenia/schizoaffective disorder. Maintenance checks werereported in four out of six (67%) studies. Results were mixed. Kern and col-leagues (2002) reported a decline in the performance of work-related tasksacross both the intervention groups at three months, while Kern and colleagues(2005) reported maintenance of social problem-solving skills at three months.
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 321
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
One notable difference between the studies was that the Kern et al. (2002)supplied two, 45–60 minute sessions and no opportunity to practise skillspost-training, whereas Kern and colleagues (2005) supplied six hours of train-ing over two days with the opportunity to practice skills post-training.
To summarise, these results suggest that systematic instructionalapproaches can produce durable/flexible acquisition of skills and infor-mation. Further, the results of those studies measuring generalisation under-score the importance of incorporating specific training components thatsupport “firm” skill and knowledge acquisition, maintenance, and generalis-ation to “real-life”, personally meaningful contexts.
DISCUSSION
Globally, the research evidence provides strong support for the effectivenessof systematic instruction. However, it is the details surrounding the designand execution of that instruction that lack clarity and require further study.Learning is complex, and the interaction between the target learning objectiveand the individual learner profile is not well understood. While this reviewdemonstrates that systematic instructional techniques are helpful for individ-uals with acquired memory impairments, identifying the most effective com-bination of variables pertaining to the training procedures, the treatment task,and participant characteristics requires dynamic assessment with each client.The research does, however, suggest several themes, or practice guidelinesthat can assist clinicians in designing, evaluating, and modifying their instruc-tion based on client performance.
Key training variables
Specificity of training. A dominant theme in the research is the need toreduce hyper-specificity of training conditions and increase effortful proces-sing in order to facilitate flexible learning and generalisation. Stimulus var-iance helps to reduce hyper-specificity. Strategy components (e.g., verbalelaboration) encourage effortful processing, which can lead to improvedattention, encoding, and organisation during acquisition as well as facilitateexplicit retrieval (Riley & Heaton, 2000; Riley et al., 2004). However,these components also run the risk of increasing recall errors. Maintaininga balance between constraining errors and encouraging effortful processingis thus critical to careful treatment planning (e.g., Komatsu et al., 2000;Tailby & Haslam, 2003).
Strategies. Studies were coded as having a “stimulus variation” com-ponent if there was an emphasis on training multiple exemplars (e.g.,Glisky & Schacter, 1987; Stark et al., 2005) or the stimulus was enhanced
322 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
in some other way (stimulus pre-exposure; Kalla et al., 2001). Studies werecoded as having a “strategy component” if the intervention included tech-niques such as verbal elaboration, imagery, or self-generated responses(e.g., Clare et al., 2000; Tailby & Haslam, 2003). At least 16 of the 51studies emphasised stimulus variability, with 15/16 (94%) of these reportingpositive findings. Of the 51 total studies, 12 included a strategy component.Of these, 9/12 (75%) reported positive findings, while three reported quali-fied outcomes. Interestingly, two of these latter studies (Evans et al., 2000;Thoene & Glisky, 1995) cited the use of imagery and/or verbal elaborationas contributing to better recall. Both of these are considered to contributeto more effortful processing compared to using EL or MVC alone.
Practice. Another critical training variable is providing sufficient prac-tice. There was a clear trend in this literature review suggesting that morepractice leads to more durable learning. For example, 14 of the 16 studiestargeting multi-step procedures (e.g., data entry, external memory aid use)reported favourable treatment outcomes and all but one prescribed high treat-ment dosages, ranging from at least six to 30 sessions or more (e.g., Andrewes& Gielewski, 1999; Hunkin, Squires, Aldrich & Parkin, 1998). The twostudies that that did not report clear positive results held less than four totaltraining sessions (Evans et al., 2000; Kern et al., 2002).
Spacing or distribution of practice trials (i.e., spaced retrieval/expandedrehearsal) is another key training variable, a finding well supported in theliterature for non-disabled individuals (Donovan & Radosevich, 1999) andfor those with memory impairment due to dementia (Hopper et al., 2005).Hopper and colleagues reviewed 15 studies in which all participants learnedsome or all of the target information taught using spaced retrieval. In thecurrent review, three studies targeted spaced retrieval or spaced presentationsfor individuals with TBI, all of which reported favourable outcomes (Hillaryet al., 2003; Melton & Bourgeois, 2005; Turkstra & Bourgeois, 2005). Further,several other studies in this review that included spaced retrieval/distributedpractice as part of their errorless learning or systematic instructionalpackage also reported favourable outcomes (e.g., Clare et al., 2000; Ehlhardtet al., 2005).
Both the frequency and distribution of practice trials contribute to masteringtargeted skills and information, which is essential if one is to use the skills/information in daily life. A “criterion for mastery” is the a priori determinationof the level of performance (e.g., accuracy, level of independence, time frame)indicative of such mastery. For example, a clinician and client might determinethat mastery has been reached when a data entry task can be performed with100% accuracy for three consecutive sessions. In this review, the majority ofthe studies did not establish a criterion for mastery since the primary researchquestions were concerned with whether a particular instructional method
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 323
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
worked and/or if it was significantly better than another method. In otherwords, information/skill mastery was not always the primary goal. Thatsaid, 13 of the 51 studies (25%) included a criterion for mastery. The majorityof these studies 11/13 (85%) reported positive outcomes, several of whichincluded high treatment dosages and distribution of practice trials as describedabove (e.g., Ehlhardt et al., 2005; Glisky & Schacter, 1989).
Task characteristics. A number of the studies remind us of the importanceof considering task characteristics when designing intervention. For example,multiple studies by Glisky and her colleagues (1986–1989) evaluating themethod of vanishing cues suggested that complex procedures can be learnedif they are broken into simple components, and if the knowledge relevant tocarrying out the procedures is explicitly trained. Evans and colleagues(2000) conducted nine experiments in three study phases to compare errorlessto errorful learning. Their results suggested that tasks and recall conditions thatfacilitated retrieval of implicit memory for learned material (e.g., learningnames given a first letter cue) benefit from errorless learning methods, whilethose that require explicit recall of novel associations (e.g., learning a routeor programming a device) do not benefit from errorless learning.3 Similarly,Thoene and Glisky (1995) showed a differential response to MVC based onthe type of task. They suggested that MVC was not beneficial for learning arbi-trary associations, whereas more explicit, mnemonic techniques were shown tobe more beneficial for such material.
Ecological validity. Of most interest to clinicians is the evidence support-ing the learning of ecological tasks. As previously stated, we defined instruc-tional targets as being ecologically valid if the target constituted informationor skills that the study participants would use in their own daily lives. Of the51 studies, 14 (27%) utilised ecologically valid tasks. Targets ranged fromface–name associations to recall of relevant people, training the use of exter-nal memory aids or computer tasks, as well as academic skills. Of thesestudies, 100% reported positive findings. This high percentage of positiveoutcomes supports Bradely, Kapur, and Evan’s (2003) assertion that task vari-ables, such as motivational and emotional significance, help memories to last.Specifically, the evidence suggests learning may be facilitated when tasks orinformation are inherently functional to an individual. Another interestingtrend noted in this subset of ecologically valid studies was that all reportedhigh treatment dosages (with a minimum of six treatment sessions), and allincorporated task analyses to train procedures and/or were explicit in identi-fying the target learning objective for the participant.
3Other authors would argue such procedural tasks rely on implicit rather than explicit
memory (e.g., Baddeley & Wilson, 1994; Page et al., 2006).
324 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
Key participant variables
No clear relationships between aetiology of memory impairment andresponsiveness to instructional methods were observed in this cross-population analysis. At this point, we can say that individuals with memoryimpairment from a variety of causes have been shown to benefit from sys-tematic instruction, suggesting that it may be fruitful to look across neuro-genic populations to evaluate instructional practices. However, as notedearlier, the outcomes were less clear for the dementia population; many ofthese studies had qualified treatment effects. Of the studies reporting qualifiedfindings across aetiology groups, most reported differential outcomes fordifferent memory severity levels (e.g., Evans et al., 2000; Riley & Heaton,2000). Riley and Heaton’s (2000) study provides a clear example. Theyshowed that participants with poorer memories may benefit from moregradual fading of cues while those with stronger memories may requiremore rapid fading. However, a number of studies (e.g., Page et al., 2006;Tailby & Haslam, 2003) showed benefits of errorless learning acrossmemory severity levels. The question of how severity of memory impairmentaffects candidacy for specific instructional practices remains equivocal.
The role played by an individual’s cognitive profile in domains other thanmemory (e.g., attention, executive functions, awareness) is equally unclear.There is some evidence that clients with frontal lobe damage and concomitantimpairments in awareness and executive functions may benefit less from themethod of vanishing cues and errorless techniques than patients with lessfrontal lobe dysfunction (e.g., Clare et al., 2002; Leng et al., 1991). Andrewesand Gielewski (1999) address this issue in their detailed report of positiveoutcomes following the training of vocational skills in a person with denseamnesia resulting from herpes encephalitis. They qualify their findings,noting that the participant’s spared executive functions, high premorbid intel-ligence, and intact semantic memory may have allowed her to benefit fromthe training and achieve successful return to work. What is evident acrossstudies is the need for therapists to employ instructional techniquesmatched to specific client cases and measure the ongoing effects of learningin order to adjust instructional practices as necessary. Experimental evalu-ation of candidacy issues is an important goal for future clinical research.
RECOMMENDATIONS
The instructional literature clearly demonstrates that structuring the mannerin which target information or procedures are introduced and reviewed canfacilitate learning. By implication, careful planning of how to train andinstruct people with damaged learning systems allows clinicians to optimiseexperience-dependent learning. The review of the best current evidence
EVIDENCE-BASED PRACTICE GUIDELINES FOR INSTRUCTION 325
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
suggests there are a number of key instructional practices that can promotelearning for individuals with acquired memory impairments. These include:
. Clear delineation of intervention targets and/or use of task analyseswhen training multi-step procedures.
. Constraining errors and controlling client output while acquiring new orrelearning information and procedures.
. Providing sufficient practice.
. Distributing practice.
. Use of stimulus variation (e.g., multiple exemplars).
. Use of strategies to promote more effortful processing (e.g., verbalelaboration; imagery).
. Selection and training of ecologically valid targets.
While there is substantial evidence to encourage clinicians to incorporateand evaluate the effects of these instructional techniques, we run the risk ofbeing overly simplistic if we look to the evidence to identify the single,right instructional practice(s). Learning is complex and we must recognisethat specific task, training, and learner characteristics will require differentcombinations of instructional practices. Riley and Heaton (2000, p. 147)eloquently remind clinicians and researchers that:
. . . the most effective [instructional] approach to teaching in any givenset of circumstances is likely to be a package that combines severalstrategies in an attempt to facilitate a range of key processes andwhich is tailored to the needs of the learning circumstances. Fromthis perspective, it is inappropriate to pose research questions aboutwhether one teaching strategy is more effective than another in someabsolute sense divorced from the circumstances.
We conclude with a call to clinicians to examine their instructional andtraining practices and to be deliberate and systematic in their treatment plan-ning. Clinicians typically do not view themselves as the designers and con-veyors of curricula. However, this is ultimately what must occur in order toimplement the instructional procedures supported in the literature. The useof instructional design and delivery principles that have been experimentallyvalidated are the key to successfully teaching or re-teaching procedures and/or knowledge to clients with memory impairments.
326 EHLHARDT ET AL.
Dow
nloa
ded
by [
RM
IT U
nive
rsity
] at
02:
25 0
7 Se
ptem
ber
2013
REFERENCES
Adams, G. L., & Engelmann, S. (1996). Research in direct instruction: 25 years beyond Distar,
Seattle, WA: Educational Achievement Systems.
American Academy of Neurology (AAN) (2004). Clinical practice guideline process manual.
Prepared for the Quality Standards Subcommittee and the Therapeutics and Technology
Assessment Subcommittee, by Edlund, W., Gonseth, G., So, Y., & Franklin, G. Retrieved
18 June 2006 at www.aan.org.
Andrewes, D., & Gielewski, E. (1999). The work rehabilitation of a herpes simplex encephalitis
patient with anterograde amnesia. Neuropsychological Rehabilitation, 9(1), 77–99.
Baddeley, A., & Wilson, B. A. (1994). When implicit learning fails: Amnesia and the problem
of error elimination. Neuropsychologia, 32(1), 53–68.
Bradley, V. A., Kapur, N., & Evans, J. (2003). The assessment of memory for memory rehabi-
litation. In P. W. Halligan & D. T. Wade (Eds.), Effectiveness of rehabilitation for cognitive
deficits (pp.115–134). NewYork: Oxford University Press.
Butters, M. A., Glisky, E., & Schacter, D. (1993). Transfer of new learning in memory-impaired
patients. Journal of Clinical and Experimental Neuropsychology, 15(2), 219–230.
Clare, L., Roth, I., Wilson, B., Carter, G., & Hodges, J. (2002). Relearning face–name associ-
ations in early Alzheimer’s disease. Neuropsychology, 16(4), 538–547.
Clare, L., Wilson, B. A., Carter, G., Breen, K., Gosses, A., & Hodges, J. R. (2000). Intervening
with everyday memory problems in dementia of the Alzheimer type: An errorless learning
approach. Journal of Clinical and Experimental Neuropsychology, 22(1), 132–146.
Donovan, J. J., & Radosevich, D. J. (1999). A meta-analytic review of the distribution of prac-
tice effect: Now you see it, now you don’t. Journal of Applied Psychology. 84(5), 795–805.
Dou, Z. L., Man, W. K., Ou, H. N., Sheng, J. L., & Tam, S. F. (2006). Computerized errorless
learning-based memory rehabilitation for Chinese patients with brain injury: A preliminary
