Memoria a Prendi Za Jel 2 Am

Language Learning ISSN 0023-8333

Relearning in the Elderly: Age-Related

Effects on the Size of Savings

Nienke van der HoevenUniversity of Groningen

Kees de BotUniversity of Groningen and University of the Free State, South Africa

This article reports on a study on learning new and relearning forgotten words ofFrench as a foreign language in young (mean age 22.4), middle-aged (mean age 50.3),and elderly speakers (mean age 76.0). The three age groups performed similarly onrelearning old words, but the younger learners were significantly better at learning newwords. Data from a questionnaire on contact with French and a working-memory testshowed that neither diminished language contact nor age-related decline in workingmemory can be seen as single factors explaining differences between learning andrelearning. The decline in older adults’ ability to learn new lexical information isdiscussed in terms of theories explaining age-related memory deficits and conceived ofas an age-related deficiency to form associations between unrelated concepts.

Keywords language attrition; relearning; vocabulary; aging; savings paradigm; French

Introduction

Over the past few decades, there has been a vast amount of research on age-related biological, psychological, and social changes that may affect humancognitive processes. Many older adults report a decline in one or more aspectsof cognitive functioning, although the extent to which any such effects play

We want to thank the four Language Learning anonymous reviewers who evaluated this manuscript

for their questions, feedback, and suggestions on previous versions of this work, and our colleague

Mik van Es for his invaluable help and support with the statistical analyses.

Correspondence concerning this article should be addressed to Nienke van der Hoeven, De-

partment of Applied Linguistics, Faculty of Arts, University of Groningen/Rijksuniversiteit

Groningen, P.O. Box 716, 9700AS Groningen, The Netherlands. Internet: N.Hoeven.van.

[email protected]

Language Learning 62:1, March 2012, pp. 42–67 42

C© 2012 Language Learning Research Club, University of MichiganDOI: 10.1111/j.1467-9922.2011.00689.x

van der Hoeven and de Bot Relearning in the Elderly

a role differs from one individual to the other. According to Burke (1999),for elderly people aspects of language production, in particular word retrievalfailures and occurrences of the tip of the tongue (TOT) phenomenon, tend tobe their most serious cognitive problems. Research into age-related changes inlanguage processing can therefore “play an important role in the developmentof theory in cognitive aging” (Burke & Shafto, 2008, p. 373). Interestingly,reduced lexical access, including TOT, is also common in both younger andolder bilingual speakers (Bialystok, Craik, & Luk, 2008; Gollan & Silverberg,2001), where it has been explained either as a result of the “frequency lag”(Gollan, Montoya, Cera, & Sandoval, 2008), that is, as a result of the fact thatbilinguals use each of their languages less than monolinguals, or as a resultof interference, that is, of competition from vocabulary items in the nontargetlanguage, which interfere during the retrieval of words from the target language(Bialystok, 2009; Green, 1998).

Many older people’s language problems are likely to be connected to an age-related decline in certain memory functions, because memory is one of the mainresources playing a role in the development of the language system (see de Bot &Makoni, 2005). Apart from the word-finding failures and the TOT phenomenonreferred to above, there is, for instance, evidence that because of a decline inmemory elderly speakers tend to use less complicated grammatical forms andsyntactic structures (Kemper, 1992; Kynette & Kemper, 1986; Lintsen & deBot, 1989). In our view, language, memory, and aging should be seen as closelyinterrelated concepts, so that studying one of them can improve insight intothe other two as well. In this respect, the role of language has been twofold.In the first place, in experimental studies, mainly in the field of experimentalpsychology, language has often been the means by which potential age-relatedeffects in certain memory functions, in particular working memory and episodicmemory, have been studied. Second, the fairly recent interest in the phenomenonof language attrition has made language itself the object of research; in this case,the focus is on processes that are fundamentally related to memory functions,such as long-term retention and consolidation processes. This study aims tocontribute knowledge that is relevant to these two fields by focusing on aquestion that is of interest to both: To what extent might memory processesthat are implicated in two language functions, namely the memorization andretrieval of lexical information from an attrited second language (L2), be subjectto age-related effects? More particularly, we will investigate if we can findevidence for an age-related advantage in relearning old over learning newlexical information. From the perspective of language attrition, we want to seeif and to what extent residual lexical information remains available after long

43 Language Learning 62:1, March 2012, pp. 42–67


periods of disuse, particularly in groups of elderly learners. Second, we hopethat potential differences between age groups in either learning or relearningmay provide us with better insights into memory-related processes, bearing inmind the suggestion by MacLeod (1988) that “relearning may tell us a gooddeal about memory in broader terms” (p. 209).

Language, Memory, and Aging

In an article on language, memory, and aging, MacKay and Abrams (1996)postulated three hypotheses that emerge from findings in linguistic research,two of which we will focus on in this article. The first one concerns the contrastbetween new versus old connections in the lexical network, addressing theobservations that “older adults exhibit differential decline in tasks that involvenew versus old learning” and “that they require more time than young adults toform the new connections for representing novel combinations of words” (pp.251–252). In the field of experimental psychology, this observation was laterexpressed in the Associative Deficit Hypothesis:

A significant cause of older adults’ deficient memory performance is theirdifficulty in merging different aspects of an episode into a cohesive unit.In a sense, the glue that links together the various aspects (attributes) of anepisode is not as efficient in old age. As a result, although each of thecomponents may be retained to a reasonable degree, the associations thattie the attributes-units to each other grow weaker in old age.(Naveh-Benjamin, 2000, p. 1185)

Second, MacKay and Abrams (1996) propose the Distributed-Deficits Hy-pothesis, which builds on the connectionist point of view (see, e.g., Anderson,1983) that mental phenomena, such as memory and cognition, can be describedas processes within interconnected networks of units, or “nodes”; according tothis hypothesis, age-related processes cause these nodes or the connectionsbetween them to “become generally defective, resulting in processing deficitsthat are distributed throughout the information processing networks” (p. 252).This hypothesis has been linked to a number of explanations for age-relatedmemory decline that all point to the same set of underlying reasons: reducedspeed of processing, reduced attentional capacity, smaller working-memorycapacity, and defective inhibitory processing.

Recent research in the fields of psychology and neuroscience has yieldedfindings that deepen insights into cognitive processes. Without denying that age-related differences in processing resources may cause differences in processing



efficiency, the approach toward cognitive aging has now become increasinglybased on the assumption that “age differences are specific to various memorystructures or systems, and the research goal is therefore to determine whetherthe different systems are affected differently by aging” (Smith, 1996, p. 243).Nowadays, the most widely accepted division of memory structures is the oneby Squire (2004), who divided the long-term memory system into a declara-tive or conscious component, which comprises episodic memory (also calledautobiographical memory, referring to conscious awareness of past events)and semantic memory (for world knowledge, i.e., for remembering informa-tion without recollection of the circumstances surrounding its learning), anda nondeclarative component (sometimes referred to as procedural memory)which is implicated in for instance procedural skills and habits, simple clas-sical conditioning, and nonassociative learning. So far, a number of studieshave investigated the extent to which the various memory systems are affectedby aging. One of the most comprehensive ones is a cross-sectional study ondistinctiveness and interrelationships among different memory systems in 345adults by Park et al. (2002). They found that episodic memory, speed of pro-cessing, short-term memory, and working memory decline to a similar degree,whereas, interestingly, semantic memory showed an increase across the lifespan. Summarizing the most relevant research into age-related deficits in declar-ative memory, Hoyer and Verhaeghen (2006) concluded that all cross-sectionalstudies of episodic memory have found considerable age-related effects, whichreflect “weakly formed associations at encoding, conceptualized as age-relatedassociative deficits or binding deficits, or the age-sensitive consequences ofattentional demands at either encoding or retrieval” (p. 215). This observationis in line with Naveh-Benjamin’s (2000) Associative Deficit Hypothesis, as itpoints to the same underlying cognitive problem: the decline in older adults’ability to form new associations between concepts.

The findings described above lead us, not surprisingly, to expect thatyounger participants will perform better on learning new lexical informationthan older learners. However, if we want to know to what extent lexical in-formation from an attrited language remains available after long periods ofdisuse, we will not only have to address the issue of how well older learnersmemorize or retrieve lexical information, but also how, after successful mem-orization in the past, these lexical items were retained over a long period oftime. For this concept, the process of consolidation is essential. The idea thatfresh memories need time to stabilize is far from new: Consolidation theoriesalready date from the end of the 19th century. Nowadays, neuroscientists dis-tinguish between two types of consolidation processes (Dudai, 2004; Medina,



Bekinschtein, Cammarota, & Izquierdo, 2008). Synaptic or cellular consolida-tion is accomplished within the first minutes to hours after encoding or practicehas finished. In this process, the hippocampus plays a crucial role. The secondtype, system consolidation, is what can follow synaptic consolidation; it is aslow process that can take months, years, or even decades and appears to resultin a permanent memory trace in the neocortex. The above information illus-trates how in fact the common division between short- and long-term memoryis an oversimplification of what happens when a lexical item is memorized. Itmight also explain why after a long period of time lexical information becomesless sensitive to age-related effects.

In this context of long-term retention of lexical knowledge, it is worth men-tioning the study by Bahrick (1984). He investigated the attrition of Spanishas an L2 learned in school, in a cross-sectional study with 773 participants.One of the main contributions of his study to the field of language attrition wasthat its focus of interest was a language learned through instruction, so that theparticipants’ language skills and knowledge were usually not as firmly estab-lished as in the case of a language acquired under naturalistic circumstances.Bahrick’s main conclusion was that:

[a] portion of the knowledge acquired in Spanish classes is lost within afew years after training, but the remainder is immune to further losses forat least a quarter of a century, and much of that content survives for fiftyyears or longer. (pp. 110–111)

However, because of the study’s cross-sectional setup, the teaching methodsused for older and younger participants may have differed. Moreover, in spiteof Bahrick’s detailed listings of variables representing all types of languagecontact that his participants might have been subject to, the extent to whichthese variables play a role can never be precisely established.

Relearning

There are two reasons why applying the relearning paradigm to the field oflanguage, memory, and aging can contribute to our knowledge about bothmemory processes and language attrition. First, as it concerns an experiment,such as the one to be reported here, which reveals to what extent the participantshave an advantage in relearning old over learning new lexical information, itwill give an indication of how much is still left of seemingly forgotten words.The idea behind the relearning paradigm is that there is a certain depth toforgetting, and that as such it is a sensitive test that can detect knowledge



that cannot be recognized by other memory tests. Second, an experiment withdifferent age groups might provide even more insight into memory functions,as differences in either learning or relearning scores between age groups mightimply an age-related development in the memory function(s) implicated inthat particular task. Before reporting on our own experiment, we will give anoverview of previous research involving the relearning paradigm.

The relearning paradigm was introduced by 19th-century German psy-chologist Hermann Ebbinghaus. In his 1885 book, Uber das Gedachtnis [OnMemory], he reports on experiments in which the only participant, himself,memorized 169 lists of 13 nonsense syllables until he could reproduce themcorrectly. Next, after periods varying between 20 minutes and a month, he re-learned a list he had learned previously and additionally learned a similar, newlist of nonsense syllables. The relearning, which we will call a savings effect,reflects the advantage of relearning old, forgotten words over new words, andin his experiment it was operationalized as the difference between the numberof trials needed to learn a previously learned list and a new one. In the begin-ning, forgetting took place rather quickly, but the rate of forgetting decreasedas a function of time. This observation led to the assumption that informa-tion, once it has been transferred to long-term memory, is never completelylost.

A few decades later, Ebbinghaus’s relearning paradigm was used for acase study. In 1907 the Swiss psychiatrist Edouard Claparede reported on aseries of experiments he conducted on a 47-year-old patient with Korsakoffsyndrome, who displayed the classical symptoms of the syndrome: “she hasretained all of her previous memories intact. She can [. . .] correctly name thecapitals of Europe, but she does not know her whereabouts nor how long shehas been there” (Claparede, 1907, cited in Nicolas, 1996, p. 1194). FollowingEbbinghaus’s relearning paradigm, Claparede administered a test on his patientin which she repeatedly had to memorize a number of words. Eventually, hefound a learning advantage for old over new words that persisted even after8 months. From this experiment and other evidence, he concluded that herunconscious or implicit memory was still (partly) intact, because she appearedto preserve some ability to acquire new procedural and semantic memories,although she could not consciously recall them.

Not until the 1970s did a number of experimental psychologists give newimpetus to research on the relearning paradigm. Nelson and colleagues (Nelson,1971; Nelson & Rothbart, 1972) investigated the properties of the informationstored in the memory trace and found that part of it is acoustic in nature.Nelson also concluded that relearning served a trace-strengthening function:



The relearning trial resulted in an increased overall amount of memory strength,which would be sufficient for successful retrieval (Nelson, 1978). In two differ-ent studies, MacLeod investigated the relearning effect in the case of translationequivalents (1976) and savings for pictures and words (1988). Surprisingly, hisexperiments showed that savings was only detected by recall, and not by recog-nition tests, which made him conclude that “relearning facilitates the retrieval ofinformation, rather than (or perhaps in addition to) increasing its trace strength”(MacLeod, 1988, p. 209).

Savings in Language Attrition Research

A decade ago, the savings paradigm was for the first time applied to languageattrition research. de Bot and Stoessel (2000) compared the learning scoreson a Dutch language test of two participants, German adults who had spokenDutch as children during a 4-year stay in the Netherlands, with those of a groupof control subjects who had not been exposed to Dutch before. On average,the participants scored better on the relearning task than the control group.They also administered a within-subjects test, consisting of old words takenfrom the original Dutch list of high-frequency words and a number of Dutchlow-frequency words. Both subjects appeared to have a significant relearningadvantage for old over new items, confirming that residual lexical knowledgewas still present in memory. The study by de Bot and Stoessel was the first ofa series of studies of savings in vocabulary relearning of languages acquiredunder natural exposure. They are reviewed by Hansen, Umeda, and McKinney(2002), who also conducted their own study on language attrition and savingsof Japanese and Korean as L2s. Investigating a large sample of 304 nativespeakers of English after their return to North America from a stay in Japanor Korea lasting from 18 to 36 months, Hansen et al. found a considerablesavings effect. They suggested that subsequent research into relearning shouldaddress the question of which variables affect the size of savings, for instance,the original proficiency in the attrited language and age. A study that focuseson the role of a potential variable is the one by Schneider, Healy, and Bourne(2002). In two experiments, they gave groups of non-French-speaking collegestudents (64 students in the first and 48 students in the second experiment)three trials of training on French-English vocabulary pairs and subsequentlytested them on these pairs. In a second session 1 week later, participants wereretested and then retrained on the same pairs, in both translation directions.The researchers found that the size of savings during relearning increasedwhen during the first session students had been trained in the more difficult



English-French translation direction or had not been pretrained on Frenchwords. This made them propose that “any manipulation that increases thedifficulty of [the conditions of] a learning task might reduce the loss across aretention interval” (p. 439).

More recently, research into savings was extended to the relearning ofvocabulary of languages acquired under classroom conditions. de Bot, Martens,and Stoessel (2004) reported on a series of experiments in which they comparedthe learning scores of newly acquired L2 words versus words students of an L2are likely to have acquired in the past. In all three tests, there appeared to be asignificant savings effect for the old over the new words.

The last study involving the relearning paradigm that we want to mentionhere is an experiment by Bowers, Mattys, and Gage (2009), in which a group of7 native English speakers tried to relearn the phonology of the language (Hindior Zulu) they had learned in childhood and from which they had afterward beencompletely separated. At first there was no evidence that the participants hadretained any knowledge of their childhood language, but after some practiceparticipants under 40 years of age appeared to have regained sensitivity tophoneme contrast in the language concerned. However, in participants over40 years of age there were no signs of relearning. Likewise, young controlparticipants who had not been exposed to Hindi or Zulu before showed nolearning. The writers concluded that even when adults have not retained anyexplicit memory of their childhood language, they may still have preservedtraces of implicit knowledge of that language.

The Present Study

In the experiment described in this article, we have again applied the relearningparadigm to the vocabulary of an L2 acquired originally under classroomconditions, investigating if and to what extent residual information remainsavailable after long periods of disuse. In contrast to earlier experiments, groupsof elderly learners are involved as well. Our main research question is whetherthe savings effect is age related or, in other words, how much different agegroups differ in the advantage they have in learning old over new words. Toassess this effect, we will operationalize “savings” as the score for old minusthe score for new words and compare the mean values for savings betweenthree different age groups (old, middle aged, and young), the hypothesis beingthat the differences between the groups will be significant.

We would like to emphasize that in our experiment the score for new wordswill function as a kind of control condition; basically, we might seem more



interested in the question of how older and younger learners perform on therelearning task, but this study involves one more variable than earlier studieson savings, that is, age. Moreover, our (cross-sectional) setup implies that thethree age groups differ not only in age, but also in the amount of time thathas elapsed since they left secondary school, and possibly in other respectsas well. Thus, in the interpretation of our results, the use and particularly thecomplexity of the extra variable, age, will require us to focus on the differencebetween the relearning and learning scores and also to analyze these differentcomponents separately in order to isolate the source of a potential differencein scores. Then, the reason why we have also included a group of middle-agedparticipants is that it will enable us to assess more accurately the extent to whichpotential differences in the size of savings between age groups are correlatedwith an expected age-related decline in working-memory capacity; after a firstdecline starting around age 20 and a period of relative stability, a second declinetypically sets in after middle age (see Park & Payer, 2006). Lastly, because wewant to assess potential effects of intervening variables, we measure whetherthere is a significant correlation between working memory, the vocabulary sizescore (i.e., the total amount of lexical knowledge that was found in the pretestto be still available for recognition), or the amount of contact with the attritedlanguage and either the savings effect or the scores for old or new words. Wealso measure whether the scores for working memory, vocabulary size, andamount of language contact differ significantly between the age groups.

There are three reasons why French was chosen as the test language for thisexperiment. In the first place, French was part of the Dutch curriculum duringthe years that the participants of the three age groups attended secondaryschool. Moreover, of the three foreign languages that are commonly taught atDutch secondary schools, it is least often used in Dutch society and it is alsoperceived to bear the least resemblance to Dutch. The last two characteristicsmake it highly subject to attrition and thus suitable for a study into savingseffects for lexical knowledge.

Method

DesignThe experiment consisted of two sessions with an interval of 2 weeks conductedby the first author, who was the experimenter. The main purpose of the studywas to assess the retention of lexical knowledge on the level of subthresholdmemory. The first session consisted of a pretest in which the amount of Frenchlexical knowledge was measured (vocabulary size), a learning session, and a



test of the words that had been memorized. Additionally, participants filled ina questionnaire on their language training and on contact with French. Duringthe second session the same words were tested as in the first session in orderto measure how many words that had been learned or relearned in the firstsession were still retained after 2 weeks. Additionally, a working-memory testwas administered: The participants were asked to repeat backward numbersequences, read out by the experimenter, of increasing length. A participant’sdigital span was defined as the maximum number of digits he or she successfullymanaged to repeat backward two consecutive times. This backward digital spantask has often been used to measure working memory, because, unlike simpleforward span tasks, it is supposed to require participants not only to load, butalso to process information (Park & Payer, 2006). Although Park et al. (2002)report that both the forward and backward digital span tend to be age insensitive,this observation is contradicted by numerous other findings (Gregoire & van derLinden, 1997; Hester, Kinsella, & Ong, 2004). Additionally, in a meta-analysisof 14 studies, Babcock and Salthouse (1990) found a greater age-related declinefor backward than for forward digital span. Another reason why we opted for thebackward digital span task was that we thought this test could be administeredrelatively easily and quickly and as such would not be too much of a strain forthe elderly participants. All the participants were tested individually, in person,in their own homes.

ParticipantsForty-five subjects participated in this experiment, with an equal division overthree age groups, that is: (a) 15 from the young group (referred to as group 1),20–30 years old (mean age = 22.4; SD = 2.2); (b) 15 from the middle-agedgroup (referred to as group 2), 45–55 years old (mean age = 50.3; SD = 2.6);and (c) 15 from the elderly group (referred to as group 3), 70–85 years old(mean age = 76.0; SD = 4.4). All the participants from the elderly group werestill living independently and leading active social lives. None of them usedmedication which could be assumed to affect their mental capacities, and neitherdid any one show during either of the two test sessions any signs of cognitivedecline. The participants were fairly equally divided over the two gender groups:From groups 2 and 3, 9 participants were female and 6 male, whereas fromgroup 1, 10 participants were female and 5 male. All participants were nativespeakers of Dutch; none of them were language students or graduates, andnone of them had lived in a French-speaking environment. As far as levelof education is concerned, they had all successfully completed a higher formof Dutch education, with between 4 and 6 years of experience in learning



French in a formal classroom setting. It was relatively difficult to find youngerparticipants with 4 years of experience in learning French, because, for thepast 6 years in all but the schools serving college-bound students (known in theNetherlands as vwo, or voorbereidend wetenschappelijk onderwijs), French hasbeen a mandatory subject only during the first 3 years of the secondary-schoolcurriculum. This also explains why on average the youngest group had a slightlyhigher level of secondary education than the older groups. All participants camefrom families with a middle to high socioeconomic status, with a fair divisionover the age groups. Because of the required 4 to 6 years of French tuition, theparticipants could not be selected at random, but instead were recruited fromthe experimenter’s personal social network.

MaterialsThe stimulus set consisted of 116 French words, either nouns or adjectives, takenfrom the vocabulary list compiled by Savard and Richards (1970), which formsthe basis of the materials used for French language teaching at Dutch secondaryschools. The words were divided over three lists. The first list consisted of60 highly frequent words that were assumed to have been included over the past70 years or so in the basic vocabulary to be learned by any Dutch pupil during thefirst years of high school. This assumption was not only based on the judgmentof the experimenter, but also on that of an experienced teacher of French. Carewas taken that the words did not have any clearly recognizable cognates inDutch or in English because all the participants had some proficiency in thatlanguage. The words on this first list were randomly mixed with 20 Frenchpseudo-words that had been checked on verisimilitude by a Dutch graduateof French. The total number of words, either real or pseudo French words,therefore amounted to 80. The second list consisted of 40 and the third list of16 French words that were slightly lower on the frequency list than those on thefirst list, but which could still be assumed to belong to the basic vocabulary to belearned by high-school students. Readers will find the complete list of Frenchwords in Appendices S1 (real words) and S2 (pseudo-words) in the onlineSupporting Information. Next, for each participant an individual set of 40 cardswas produced, 20 of them containing the first 20 French words from the wordlistfor which the participant had failed to produce the correct translation. On theback of the cards were the Dutch translations. The other 20 cards contained thepseudo-words from the first list of the pretest, with a translation on the back,made up by the experimenter. The pseudo-words and their translations did notbear any obvious resemblance to existing French words. Just as during thepretest, the participants were not informed that these were pseudo-words. The



reason why pseudo-words instead of low-frequency words were used is that itcould not be taken for granted that all lower frequency words were completelyunknown to all participants. The list of low-frequency words used by de Botet al. (2004), for instance, contained a small number of words that were knownto the experimenter and therefore possibly also to the participants.

The reason why we did not test in a computerized setting is that thismethod was expected to present problems for some participants, in particularfor elderly people who were not familiar with computers. As a result, theywould have been at a disadvantage compared to participants from the youngerage groups. Another issue concerns our decision to use a translation test in thepretest, in which we assessed if a word was forgotten or not. We are aware thatthis is not the most sensitive test of memory, but to make up for this, we set notime limit on this pretest, encouraged participants to guess, and accepted neartranslations. Moreover, we were concerned that the alternative, a recognition ora forced-choice test, might affect test results. Lastly, we assumed that this wayof (pre)testing did not give one age group any advantage over the other two.Theoretically, it cannot be ruled out that this assumption was wrong, but thereis no evidence from postexperiment interviews that in this respect there was adifference among the age groups.

The questionnaire (see Appendix S3 in the online Supporting Information)contained questions regarding the participants’ secondary education, training,and grades in French; knowledge of additional Romance languages; and recentcontacts with the French language, inquiring how often they (1) visited aFrench-speaking country; (2) spoke French; (3) read a French book, magazine,or newspaper; or (4) watched French films or television programs. As regardsknowledge of Romance languages, 10 participants from group 1 had beentaught another Romance language (Latin, Spanish, or Italian), compared to 8participants from group 2 and 6 from group 3. The answers to the four questionsabout language contact (see Appendix S3, questions 8–11) were evaluated (aanswers yielding 0, b answers yielding 1, and c and d answers yielding 2 points)and incorporated in the variable “language contact” with eight levels, level 1(0 points) indicating no recent contact and level 8 (8 points) indicating amaximum degree of contact.

ProcedureAll the participants were tested by the experimenter in their own homes, indi-vidually and in person. The test language was Dutch. They were first presentedwith a list of 60 French words, mixed with 20 pseudo-words, with the requestto provide the Dutch translations of the words they still remembered. In this



pretest they were encouraged to guess if they felt unsure about a translation,and there was no time limit. Next they filled in a questionnaire, while the ex-perimenter listed which words were correct and selected the cards containingwords for which no correct translation had been produced. If this number waslower than 20, the participant was presented with a second list, which con-tained 40 slightly less frequent words. If after they had filled this one in thetotal number of words they had not translated correctly was still below 20, theywere presented with a third list of 16, still less frequent words. In all cases,this resulted in an individual list of 20 unrecognized French words for everyparticipant. Finally, the cards with the 20 words that had not been recognizedwere mixed with the cards containing the pseudo-words, which had alreadybeen presented in the pretest. The participants were asked to study the wordsone by one and to memorize the Dutch translation on the back of the card for8 seconds in total. After they had finished, they were asked to count backwardfrom 100 by steps of 7. Next, they were asked to describe in detail how theytraveled from their home to a public building at some distance, for instance, thecentral station. Finally, they were asked to count backward from 40 by stepsof 3 and they were engaged in some conversation. These tasks were meant todistract the participants, so that they had to empty their working memory. Afterabout 10 minutes, the cards were presented to them in a random order and theywere asked to provide the translations.

During the second session, 2 weeks later, each participant was presentedwith the same words that he/she had memorized in session 1, but in a newrandom order, and asked to provide the translations. Finally, at the end of thesecond session, the participants’ working memory was tested by means of thebackward digital span task.

Results

In this section, we report the descriptive statistics and inferential results ofour experiment. Readers interested in the raw results for our main variablescan find the individual scores listed in Appendix S4 in the online SupportingInformation. To answer our main research question, if the age groups differin the advantage they have in learning old over new words, we present theaverage number of words remembered correctly, the savings effect (the scoresfor old minus new words), and standard deviations in Table 1. The scores overtwo sessions are plotted in Figure 1, and the scores for savings (scores for oldminus new words) for the age groups, over 2 sessions, are plotted in Figure 2.



Table 1 Average test scores (and standard deviations) per age group (N = 15 each)

Age Group 20–30 yrs 45–55 yrs 70–85 yrs

Session 1 Old words 12.9 (3.0) 13.1 (3.9) 12.4 (3.4)New words 7.1 (4.5) 4.4 (4.5) 2.9 (2.6)Savings effect 5.8 (3.2) 8.7 (3.8) 9.5 (3.1)

Session 2 Old words 11.0 (4.2) 9.6 (4.3) 10.9 (4.6)New words 4.4 (2.8) 1.3 (1.8) 1.1 (1.2)Savings effect 6.6 (3.0) 8.3 (3.8) 9.9 (4.0)

Note. Total maximum score possible was 20 for old words and 20 for new words.

Figure 1 Boxplot with scores for old and new/pseudo-words in two sessions, for threeage groups.

We first performed a repeated-measures analysis of variance (ANOVA)with age group (three levels) as the between-subjects factor and the scoreson savings in sessions 1 and 2 as the within-subjects factor. Session had nosignificant effect, F(2,42) = 0.22, p = .64. Neither was there a significant



Figure 2 Boxplot with scores for savings (old minus new words) in sessions 1 and 2.

interaction effect of session with age group, F(2,42) = 0.51, p = .61. For therest of our analyses we therefore used the mean values for savings over the twosessions.

An ANOVA1 with age group (3 levels) as independent variable and theaverage score for the old minus the new words (i.e., savings) in sessions 1 and2 as dependent variable showed that the age groups differed significantly,F(2,42) = 5.03, p = .01, eta-squared = 0.19. Tukey’s post hoc tests showedthat the difference between the young and the elderly group was significant(p = .01); the difference between the young and the middle-aged (p = .11) andbetween the middle-aged and the elderly (p = .55) was not significant.

To isolate the source of this effect, we performed one-way ANOVAs2 on theaverage word scores over the two sessions. For old words, the effect of age groupwas not significant, F(2,42) = 0.09, p = .92. For the new or pseudo-words, theeffect of age group was significant, F(2,42) = 7.33; p = .00. Tukey’s post hoctests showed that the young group differed significantly from the middle-aged(p = .02) and elderly groups (p = .00). The difference between the middle-agedand the elderly groups was not significant (p = .68).



Table 2 Average scores for working memory, vocabulary size, and language contact(and their standard deviations) per age group (N = 15 each)

Age group Working memorya Vocabulary sizeb Language contactc

20–30 years 6.7 (1.0) 24.1 (7.4) 1.5 (0.7)45–55 years 6.0 (1.0) 40.5 (19.1) 2.2 (1.4)70–85 years 4.9 (0.8) 64.5 (31.1) 2.4 (1.7)

aScore on the backward digital span task, out of a total possible score of 10.bMeasured as number of real French words recognized on the pretest, out of 116 total.cSelf-reported in the study questionnaire (see Appendix S3, questions 8–11), 0 =no recent contact, 8 = maximum degree of contact.

We also measured variables that could affect the scores on old or newwords. The average scores per age group for working memory as measured inthe backward digital span task, the average vocabulary size scores as measuredin the pretest, and the average value for the level of recent French languagecontact are presented in Table 2.

As regards working memory, a one-way ANOVA3 showed that the agegroups differed significantly in working memory scores, F(2, 42) = 13.63, p =.01; a Tukey’s post hoc test showed that only the difference between the elderlygroup and the middle-aged group (p = .01) and between the elderly and theyoungest group (p = .00) were significant. Correlation tests revealed significantpositive correlations between working memory scores and the average scorefor old words (r = .47) and new or pseudo-words (r = .54), but not betweenworking memory and the average score for savings, that is, the score for oldminus new words (r = −.01).

In order to analyze whether the age groups differed significantly in the scoreson vocabulary size, we used a (nonparametric) Kruskal-Wallis test, because theLevene test showed that the variances across the age groups were not equal(p = .00). The difference between the age groups was significant, H(2,42) =13.56, p = .01; further Kruskal-Wallis tests showed that the vocabulary size ofthe youngest group was significantly smaller than that of the two older groups(p = .00) and that of the oldest group significantly larger than that of the twoyounger groups (p = .00). A correlation test showed that the correlation betweenvocabulary size and the average score for old words was significant, r = 0.42,p < .01. A correlation test also revealed a significant positive correlationbetween the scores for vocabulary size and the average score for savings (scorefor old minus new words), r = 0.51, p < .01 (Figure 3).



Figure 3 Scatterplot showing the correlation between the vocabulary size score and theaverage score for old minus new words (savings).

As regards the variable indicating the amount of French language contact,an ANOVA4 showed that the differences between the age groups were not sig-nificant (p = 0.15). Correlation tests revealed a significant positive correlationbetween the scores for language contact and the scores for vocabulary size,r = .54, p < .01, but correlations between scores for language contact andscores for old words, new words, or savings were not significant.

Discussion

The main focus of this study was to assess the relationship between the factor ofage and the strength of the savings effect, that is, we addressed the question ofwhether there was a difference between three age groups in the advantage theymight have in relearning old over learning new lexical information. We found



that the difference between the elderly and the young group was significant.These results confirm the hypothesis that the size of savings is age related:Older learners experience a greater advantage in relearning old over learningnew words than younger learners. Isolating the source of this effect, we foundthat the three groups’ performance was very much similar in relearning oldwords, but that young learners were significantly better at learning new wordsthan older learners.

In order to analyze whether potential differences between age groups in theadvantage they have in relearning old over learning new lexical informationmight be caused by interfering effects from other variables, we also measuredthe three additional factors of working memory, vocabulary size (i.e., the totalamount of words recognized in the pretest), and the amount of target languagecontact. As regards language contact, we found that the difference between theage groups was not significant. Still, we found a significant positive correlationbetween language contact and vocabulary size. The correlation between vocab-ulary size and the scores for old words and savings (i.e., the score for old minusthe score for new words) may be (partly) mediated by the correlation betweenlanguage contact and vocabulary size. However, it is notoriously difficult toachieve a valid measurement of language contact, which is a complex con-struct (Li, Sepanski, & Zhao, 2006). In the first place, participants sometimesappeared to interpret questions regarding language contact differently. Second,even if a participant regularly went to France, it appeared that this did not nec-essarily mean that he or she regularly spoke French; for instance, particularlyyounger participants often reported on having stayed on French campgroundswithout any significant contact with the French language.

Our test results for working memory are in line with findings fromthe literature, in particular those from studies by Park et al. (2002), in thatonly the elderly differed significantly, and negatively, from the younger groups.There was also a significant positive correlation between working memory andthe test scores for old and new words, but the correlation between workingmemory and savings, which reflects the difference between the score for oldminus the score for new words, was not significant. Therefore, an age-relateddecline in working memory cannot be seen as the single explanatory factor forthe age-related difference in the size of savings. This is in line with expecta-tions, as this experiment addressed not only memorization, but also long-termretention of lexical knowledge. Still, working memory may play a mediatingrole in determining the strength of the savings effect.

Our results for the variable vocabulary size, operationalized as the totalnumber of words recognized in the pretest, appeared more interesting. Here,



again, the elderly group differed significantly from the middle-aged and younggroups; somewhat surprisingly, the vocabulary size score of the elderly washigher than that of the other groups. The difference between the middle-agedand young groups was not significant. There was also a significant positivecorrelation between vocabulary size and the savings effect, suggesting a signif-icant cohort effect of vocabulary size. Apparently, people with a large amountof lexical knowledge left find it easier to relearn old words than people withless lexical knowledge left. The reason for this might be that people withlarge amounts of lexical knowledge have large lexical networks and conse-quently many potential links with seemingly forgotten words, thus improvingthe chance that on relearning a word will again be activated. Another point ofdiscussion presents itself if we consider our findings against the backgroundof current work on bilingualism, particularly on the studies dealing with re-duced lexical access in bilingual speakers mentioned in the introduction. Theidea would be that the fact that the elderly participants have larger amounts ofvocabulary knowledge left, and therefore retain larger lexical networks, mightalso have a negative impact on their ability to learn new words. In line withthe interference account for reduced lexical retrieval in bilinguals, the addi-tional lexical links in elderly people’s networks might thus cause interferencefor forming new associates, which might provide an alternative account forNaveh-Benjamin’s (2000) Associative Deficit Hypothesis.

We acknowledge that the cohort effect of vocabulary size that we foundmay be seen as a threat to the results of our study. Still, if we want to determinewhich factor causes the difference between the age groups, we should analyzein which respects the age groups differ from each other: in relearning old orin learning new words, or both. As we have seen, the different age groupsperform very much the same in relearning, and the differences in savings areprimarily caused by an age-related decline to learn new words. The groupwith significantly higher vocabulary size scores than the other two, namely theelderly, also had the lowest scores for learning new words, and these were thescores that caused the difference in the size of savings.

It is remarkable that, in spite of the variation in vocabulary size, all groupshave surprisingly similar scores for relearning old words, while they differ inthe scores for new words, which showed no correlation with vocabulary size.Of course, it can never be established with any certainty how much importancewe must attach to this equality in the relearning scores (i.e., the scores for theold words) over the age groups. On the one hand, for the younger group theperiod of language tuition was much more recent, and the younger groups hadsignificantly higher scores on the working-memory test; on the other hand, the



older groups had a larger vocabulary size, which put them at an advantage.It remains to be seen if taken together these factors resulted, more or lesscoincidentally, in equal scores for the age groups, or if, because all participantshad to relearn old words at their own level, that is, taken from their ownpersonal lists compiled after the pretest, they performed more or less equallyin this respect.

As we have seen, another remarkable outcome of our study was that vocab-ulary size seems to correlate positively with age. In this respect, the differencebetween the youngest group and the elderly group was so considerable, that wemust conclude that after the end of formal education the average vocabularysize of the elderly, and probably that of the middle-aged group as well, was mostlikely larger than that of the youngest group, if we take it that after secondaryeducation the amount of language contact was similar, which according to theresults of our questionnaire seemed indeed to be the case. Still, the point isperhaps not primarily how often participants went to France, but rather thatwhen in France, elderly and middle-aged learners probably made better useof opportunities to (re)activate and increase their lexical knowledge than theyounger ones, because of their high proficiency, which is based on a large vo-cabulary size at finishing school. This phenomenon is an example of the wellknown Matthew effect, according to which the rich get richer, and the poor getpoorer. The high vocabulary size scores of the older groups are also in linewith findings in the literature about the lack of age-related decline in verbalknowledge (i.e., semantic memory), reported for instance by Bahrick (1984);as we have seen, some studies, for instance the one by Park et al. (2002), evenreported an increase for this factor across the life span. Moreover, also whenconsidered in the light of consolidation theories (e.g., Medina et al., 2008),our findings for vocabulary size are not as surprising as they might seem atfirst. Even when lexical information has been transferred from working mem-ory to long-term memory systems, final consolidation or system consolidationmay take months, years, or even decades. In this final stage of consolidation,(re)activation of the existing memory trace seems essential.

Although the approach of this study was quantitative, the fact that all par-ticipants were tested orally and individually makes it possible to present someanecdotal evidence regarding their learning strategies. Even though the partic-ipants were not aware that there were basically two different word types, theyoften seemed to use different strategies to memorize and retrieve the old (real)and the new (pseudo) words. Particularly elderly and middle-aged participantsoften remarked that they found it extremely difficult to memorize the newwords, as they could not see any connection between them. Participants who



indicated they had problems memorizing new words could not think of facilitat-ing strategies. Only two elderly and three middle-aged participants seemed toconsciously, and successfully, employ a strategy during word learning, namelyforming associations between words by using mnemonic aids. Several youngerparticipants appeared to use this strategy successfully. The main differencebetween younger and older participants using mnemonic strategies was thatduring the second session older participants had often forgotten the associa-tions they had previously made between words and their translations; one ofthem even exclaimed “where are my associations!” Younger participants of-ten remembered the associations from the first session successfully. On theother hand, elderly and middle-aged participants indicated that memorizingand retrieving old words was much easier than memorizing new ones, becausethey “just knew them,” as the old words were still “somewhere at the backof their minds,” even if at first sight the words were not familiar. Basically,relearning seemed to be a cognitively different task than learning, and for re-trieving old versus new words the participants seemed to rely on different strate-gies. This difference between relearning and learning strategies was less clearfor younger participants, who for both word types relied more on mnemonicaids.

Last but not least, we would like to remark upon the fact that the youngestparticipants, who left school most recently, had by far the lowest vocabulary sizescores. The disappointing results can partly be explained by the insignificantrole of French in modern Dutch society. Yet, this situation was rather similarwhen the middle-aged group attended secondary school. The results might alsobe attributed to recent changes in the French curriculum in the Dutch schoolsystem. The amount of time spent on French has been drastically reduced.Moreover, teaching methods have changed. Nowadays, far less time is spent onmemorizing words than in the past, when the emphasis in word learning wason repetition (for an overview of foreign language teaching methodology inthe Netherlands, see Wilhelm, 1997). Of course, tuition in a foreign languageinvolves much more than memorizing words. Still, the data from our experimentsuggest at least a better retention for the older participants.

Conclusion

We would like to conclude based on the evidence we have presented thatolder learners experience a greater advantage in relearning old over learningnew words than younger learners. Both quantitative findings and anecdotalevidence from the experiment point to an age-related decline in the ability to



learn new lexical information, which seems to be related to a decreased abilityin older adults to form new associations between concepts. This observation isin line with findings from the literature described in the background section ofthis paper, and particularly with the Associative Deficit Hypothesis by Naveh-Benjamin (2000). For the ability to relearn lexical information we found thatthere was hardly any difference between the age groups. We hope that thepaper has made it sufficiently clear that the increased savings effect for olderlearners does not imply that they have any advantage over younger participantsin their performance on the relearning task; the concept of savings has beendefined as the advantage of relearning old over learning new words, and thusoperationalized as the difference between the scores on the relearning andthe learning (the control) task. The fact that for older learners this advantageis greater than for younger learners, because in spite of the decline in theirability to memorize new words, they perform similarly on relearning forgottenwords, tells us something important about long-term retention of vocabulary.It suggests that an amount of lexical knowledge that is not forgotten soonafter memorization is preserved or consolidated in long-term memory, whereit becomes relatively insensitive or immune (cf. Bahrick, 1984) to further, age-related loss. Although, as we found in the pretest, its traces may not be strongenough for recall or recognition, they can still be detected by means of therelearning paradigm. At the same time, the pretest showed that the older groupshad a larger vocabulary size than the younger group, and this advantage mighthave facilitated the relearning process. The older groups probably had a largevocabulary size upon leaving secondary school, and this lexical knowledge wasprobably firmly consolidated in the years that followed, particularly when it wasregularly activated.

Finally, with respect to recommendations for further study, we must ac-knowledge that the age-related effect that we found was confused by a cohorteffect for vocabulary size. This was partly caused by the study’s cross-sectionaldesign: The three age groups might have differed not only in terms of theinterval between the testing and the participants’ secondary school period butalso in the amount of language contact and the teaching methods in the attritedlanguage. Perhaps in future studies such cohort effects can be limited by theuse of a more longitudinal design. Still, we hope that this paper has shownconvincingly that looking at age-related effects on relearning can help to pro-vide useful, unique insights into memory processes. At the same time, futureresearch into language attrition might benefit from contributions by psychol-ogists and neuroscientists, as better insights into cognitive functions may lead



to a better understanding of processes of memorizing and retrieving lexicalinformation.

Revised version accepted 1 February 2010

Notes

1 The Levene test for homogeneity of variance was not significant: p = 0.47. Thehistogram appeared to be rather normal.




References

Anderson, J. (1983). A spreading activation theory of memory. Journal of VerbalLearning and Verbal Behavior, 22, 261–295.

Babcock, R. L., & Salthouse, T. A. (1990). Effects of increased processing demands onage differences in working memory. Psychology and Aging, 5, 421–428.

Bahrick, H. P. (1984). Fifty years of second language attrition: Implications forprogrammatic research. Modern Language Journal, 68, 105–118.

Bialystok, E., Craik, F. I. M., & Luk, G. (2008). Cognitive control and lexical access inyounger and older bilinguals. Journal of Experimental Psychology: Learning,Memory, and Cognition, 34, 859–873.

Bialystok, E. (2009). Bilingualism: The good, the bad, and the indifferent.Bilingualism: Language and Cognition, 12, 3–11.

Bowers, J., Mattys, S., & Gage, S. (2009). Preserved implicit knowledge of a forgottenchildhood language. Psychological Science, 20, 1064–1069.

Burke, D. M. (1999). Language production and aging. In S. Kemper & R. Kliegl (Eds.),Constraints on language, aging, grammar and memory (pp. 3–28). Boston: Kluwer.

Burke, D. M., & Shafto, M. A. (2008). Language and aging. In F. I. M. Craik & T. A.Salthouse (Eds.), The handbook of aging and cognition (3rd ed., pp. 373–443).NewYork: Psychology Press.

de Bot, K., Martens, V., & Stoessel, S. (2004). Finding residual lexical knowledge: The“savings” approach to testing vocabulary. International Journal of Bilingualism, 8,373–382.

de Bot, K., & Makoni, S. (2005). Language and aging in multilingual contexts.Clevedon, UK: Multilingual Matters.



de Bot, K., & Stoessel, S. (2000). In search of yesterday’s words: Reactivating a longforgotten language. Applied Linguistics, 21, 364–368.

Dudai, Y. (2004). The neurobiology of consolidations, or, how stable is the engram?Annual Review of Psychology, 55, 51–86.

Ebbinghaus, H. (1885). Uber das Gedachtnis. Untersuchungen zur experimentellenPsychologie [On memory: Research into experimental psychology]. Leipzig,Germany: Duncker & Humblot.

Gollan, T. H., & Silverberg, N. B. (2001). Tip-of-the-tongue states in Hebrew-Englishbilinguals. Bilingualism: Language and Cognition, 4, 63–83.

Gollan, T. H., Montoya, R. I., Cera, C., & Sandoval, T. C. (2008). More use almostalways means a smaller frequency effect: Aging, bilingualism, and the weaker linkshypothesis. Journal of Memory and Language, 58, 787–814.

Green, D. W. (1998). Mental control of the bilingual lexico-semantic system.Bilingualism: Language and Cognition, 1, 67–81.

Gregoire, J., & Van der Linden, M. (1997). Effects of age on forward and backwarddigit spans. Aging, Neuropsychology, and Cognition, 4, 140–149.

Hansen, L., Umeda, Y., & McKinney, M. (2002). Savings in the relearning of secondlanguage vocabulary: The effects of time and proficiency. Language Learning, 52,653–678.

Hester, R. L., Kinsella, G. J., & Ong, B. (2004). Effect of age on forward and backwardspan tasks. Journal of the International Neuropsychological Society, 10, 475–481.

Hoyer, W., & Verhaeghen, P. (2006). Memory aging. In J. Birren, K. Schaie, & R.Abeles (Eds.), Handbook of the psychology of aging (6th ed., pp. 209–232).Amsterdam: Elsevier.

Kemper, S. (1992). Adults’ sentence fragments: Who, what, when, where and why.Communication Research, 19, 445–458.

Kynette, D., & Kemper, S. (1986). Aging and the loss of grammatical forms: Across-sectional study of language performance. Language and Communication, 6,65–72.

Li, P., Sepanski, S., & Zhao, X. (2006). Language history questionnaire: A Web-basedinterface for bilingual research. Behavior Research Methods, 38, 202–210.

Lintsen, T., & de Bot, K. (1989). Language proficiency in Dutch elderly people.Nijmegen, Netherlands: Department of Applied Linguistics.

MacKay, D., & Abrams, L. (1996). Language, memory and aging: Distributed deficitsand the structure of new-versus-old connections. In J. Birren & K. Schaie (Eds.),Handbook of the psychology of aging (4th ed., pp. 251–265). San Diego, CA:Academic Press.

MacLeod, C. M. (1976). Bilingual episodic memory: Acquisition and forgetting.Journal of Verbal Learning and Verbal Behavior, 15, 347–364.

MacLeod, C. M. (1988). Forgotten but not gone: Savings for pictures and words inlong-term memory. Journal of Experimental Psychology: Learning, Memory andCognition, 14, 195–212.



Medina, J., Bekinschtein, P., Cammarota, M., & Izquierdo, I. (2008): Do memoriesconsolidate to persist or do they persist to consolidate? Behavioural Brain Research,192, 61–69.

Naveh-Benjamin, M. (2000). Adult age-differences in memory performance: Tests ofan associative deficit hypothesis. Journal of Experimental Psychology: Learning,Memory and Cognition, 26, 1170–1187.

Nelson, T. O. (1971). Savings and forgetting from long-term memory. Journal ofVerbal Learning and Verbal Behavior, 10, 568–576.

Nelson, T. (1978). Detecting small amounts of information in memory: Savings fornon-recognized items. Journal of Experimental Psychology: Human Learning andMemory, 4, 453–468.

Nelson, T. O., & Rothbart, R. (1972). Acoustic savings for items forgotten fromlong-term memory. Journal of Experimental Psychology, 93, 357–360.

Nicolas, S. (1996). Experiments on implicit memory in a Korsakoff patient byClaparede (1907). Cognitive Neuropsychology, 13, 1193–1199.

Park, D., Lautenschlager, G., Hedden, T., Davidson, N., Smith, A., & Smith, P. (2002).Models of visuospatial and verbal memory across the adult life span. Psychologyand Aging, 17, 299–320.

Park, D., & Payer, D. (2006). Working memory across the adult lifespan. In E.Bialystok & F. I. M. Craik (Eds.), Lifespan cognition: Mechanisms of change(pp. 128–143). Oxford, UK: Oxford University Press.

Savard, J., & Richards, J. (1970). Les indices d’utilite du vocabulaire fondamentalfrancais [Indicators of usefulness of basic French vocabulary]. Quebec, Canada:Les presses de l’Universite Laval.

Schneider, V., Healy, A., & Bourne, L. (2002). What is learned under difficultconditions is hard to forget: Contextual interference effects in foreign vocabularyacquisition, retention and transfer. Journal of Memory and Language, 46, 419–440.

Smith, A. (1996). Memory. In J. Birren & K. Schaie (Eds.), Handbook of thepsychology of aging (4th ed., pp. 236–247). San Diego, CA: Academic Press.

Squire, L. (2004). Memory systems of the brain: A brief history and currentperspective. Neurobiology of Learning and Memory, 82, 171–177.

Wilhelm, F. (1997). Foreign language teaching policy in the Netherlands 1800–1970:A historical outline. In T. Bongaerts & K. de Bot (Eds.), Perspectives on foreignlanguage policy: Studies in honour of Theo van Els (pp. 1–21). Amsterdam:Benjamins.

Supporting Information

Additional Supporting Information may be found in the online version of thisarticle:

Appendix S1. Real French Words.



Appendix S2. Pseudo French Words.Appendix S3. Questionnaire.Appendix S4. Individual Test Results.

Please note: Wiley-Blackwell are not responsible for the content or functionalityof any supporting materials supplied by the authors. Any queries (other thanmissing material) should be directed to the corresponding author for the article.


Date post:	10-May-2017
Category:	Documents
Upload:	bastigueta
View:	222 times
Download:	0 times

Memoria a Prendi Za Jel 2 Am

Documents