Linguistic Society of America - UT Liberal Arts › frenchitalian › ...been amply documented (e.g....

Linguistic Society of America

Ultimate Attainment in Second Language AcquisitionAuthor(s): David BirdsongSource: Language, Vol. 68, No. 4 (Dec., 1992), pp. 706-755Published by: Linguistic Society of AmericaStable URL: http://www.jstor.org/stable/416851 .Accessed: 28/02/2014 14:52

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Linguistic Society of America is collaborating with JSTOR to digitize, preserve and extend access to Language.

http://www.jstor.org

This content downloaded from 128.83.77.101 on Fri, 28 Feb 2014 14:52:55 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=lsahttp://www.jstor.org/stable/416851?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsp

ULTIMATE ATTAINMENT IN SECOND LANGUAGE ACQUISITION

DAVID BIRDSONG

University of Texas at Austin On the prevailing view of ultimate attainment in second language acquisition, native

competence cannot be achieved by postpubertal learners. The present study offers con- vergent experimental evidence which suggests there are exceptions to this generalization. At the same time, early arrival in the host country-even if past puberty-correlates with attainment of native norms on a variety of measures. Also investigated are the loci of competence differences in the syntax of the target language (French). Contrary to findings by Coppieters 1987, experimental performance is not predicted by the status of a given linguistic variable as within or outside the theoretical domain of Universal Gram- mar.*

INTRODUCTION

1. The outcome of first language acquisition (L1A) is success: normal chil- dren acquire the grammar of the ambient language. Adult second language acquisition (L2A), on the other hand, results in varying degrees of success. Failure to acquire the target language grammar is typical. This contrast in ultimate attainment-universal success in LIA, variable success in L2A-has been amply documented (e.g. Johnson & Newport 1989, Sorace 1991a,b) and has been the centerpiece for arguments that L1A and L2A are profoundly different epistemological phenomena (Bley-Vroman 1989).

The facts of ultimate attainment in L2A also represent a basic point of reference for studying maturational effects. Investigations of critical or sensitive periods for language acquisition examine the end-point or steady-state competence of individuals who began L2 learning at various ages or stages of development.' The general failure of adults to acquire a second language is

* Portions of this paper were presented at the 1991 meetings of the Second Language Research Forum, the Applied Linguistics at Michigan State Conference, and the Boston University Con- ference on Language Development. The paper has profited from discussions with Alan Beretta, Randy Diehl, Lynn Eubank, Jim Lantolf, Richard Meier, Bill Rutherford, Bonnie Schwartz, An- tonella Sorace, and Lydia White. I wish to acknowledge the contributions of audiences at several universities: Concordia, Cornell, Florida, McGill, Southern California, and Texas. The comments of two anonymous reviewers for Language are also appreciated.

I extend sincere thanks to David Johns, who contributed to the data analysis; Michele Roberts, who transcribed many of the think-alouds; and John Bro and Dalila Ayoun, who assisted in data elicitation. The cooperation of all subjects is also greatly appreciated. I am especially indebted to Steve Pinker and to Susana Franck and Jacques Mehler of the Laboratoire de Sciences Cognitives et Psycholinguistique in Paris for their assistance during the data collection phase of the project.

This study was supported in part by a Research Development Award from the Division of Spon- sored Research, University of Florida, DSR 89021023.

1 There are conflicting views on whether it is appropriate to think of ultimate attainment in terms of a 'steady state' or an 'end point'. Hurford (1991:196) assumes that 'language is a finite bounded system which can be known in its entirety by a speaker'. However, it is indisputable that additional lexis, novel morphological derivations, and jargon/slang/idiomatic expressions accrue throughout an individual's lifetime. (This observation applies to both near-natives and natives.) Ultimate attainment is therefore not an end point in any simple sense, but an asymptotic curve approaching some theoretical point of absolute completeness.

706


http://www.jstor.org/page/info/about/policies/terms.jsp

ATTAINMENT IN SECOND LANGUAGE ACQUISITION

consistent with the notion of a critical period for language learning (Lenneberg 1967). Michael Long, in an extensive review of the literature on maturational constraints, views ultimate attainment in terms of a Popperian test of critical period hypotheses: 'the easiest way to falsify [claims for a critical period in L2A] would be to produce learners who have demonstrably attained native- like proficiency despite having begun exposure well after the closure of the hypothesized sensitive periods' (1990: 274). Thus, against the backdrop of general failure, one may still wish to establish the upper limits of adult L2A, that is, to determine 'whether the very best learners actually have native-like competence' (281).

The ultimate attainment issue implies two fundamental questions. The first is whether there are competence differences between native speakers (NS) of a language and those exceptional learners who appear to perform like natives, i.e. near-native speakers (NNS). Hyltenstam 1990 reports that highly proficient bilinguals, even if they began L2 learning in childhood, demonstrate nonnative grammatical and lexical features; however, Hyltenstam points out that design features of his study and the small sample size make it impossible to answer this question categorically. It is understandable that concerns for experimental design, procedural control, and prudent interpretation of results should be at the heart of the matter, since the relevant data, often in the form of elicited production or acceptability judgments, provide at best indirect evidence for grammatical competence. Long's review of studies to date reaches the tentative conclusion that grammars acquired by nonnative adults do not coincide with those of natives, but he challenges researchers to overcome the empirical ob- stacles that have precluded a definitive answer (1990:281).

The second question addresses the locus of differences between NS and NNS. In what grammatical domains do the groups appear to diverge? This

Under the standard generativist view of competence, it may be argued that internal representations of grammars can be conceived in terms of steady-state competence. In this sense, all adult native speakers of a language share the same linguistic competence and could be said to have reached the same point in ultimate attainment.

Clearly, these idealizations ignore facts of performance and interspeaker variation. On the indirect measures of competence available to us (primarily in the form of introspective judgments of acceptability), natives disagree with one another (see Ross 1979). Thus the putative 'ultimate attainment' of one group of native speakers may be different from that of another group of natives.

In addition, within groups of natives, variation in judgments is attested on all but the most basic sentence structures. Intragroup variation can be reduced, but usually not eliminated, by sampling from narrow, homogeneous social and educational classes. As a practical matter, comparison of the ultimate attainment of natives and nonnatives involves eliciting judgments from subjects that are similar along relevant social and educational dimensions. Such intergroup comparisons must take into consideration intragroup variability. Accordingly, statistical procedures such as ANOVAs and t-tests are used to compare the groups' central tendencies and variance in judgments for a given item. Note that the benchmark for ultimate attainment to be employed here is native speakers' judgments, not linguists' assignments of sentence grammaticality or prescriptivists' notions of correct and incorrect. (For discussion of disagreement between linguists and 'normals', see Ross 1979; for discussion of native speaker variability and grammatical competence, see Davies 1991.)

Note also that the scope of the present investigation is restricted to isolated aspects of target- language syntax, and does not address the acquisition of phonology, morphology, pragmatics, etc., which would be treated in an exhaustive study of ultimate attainment.

707



LANGUAGE, VOLUME 68, NUMBER 4 (1992)

question is of special interest to L2A theorists, whose predictions about L2 competence often originate in theoretical and descriptive accounts of target languages. In particular, theories of L2A derived from the principles and pa- rameters approach to grammar (Chomsky 1981) look to abstract properties of L2 learners' syntax for evidence that Universal Grammar (UG) is available to these learners (for an overview, see White 1989a). Relevant data are sometimes in the form of asymmetries in errors across linguistic structures. For example, the frequently-cited 1989 study by Johnson & Newport addresses the acquisition of twelve basic English structures, among them subject-auxiliary inver- sion in Yes-No questions, irregular verb morphology, canonical word order, and use of determiners in obligatory contexts. Nonnative subjects (native speakers of Korean and Chinese) made fewer errors in grammaticality judgments for canonical word order violations (e.g. *Martha a question asked the policeman) than for ungrammatical items involving determiners (e.g. *Tom is reading book in the bathtub). Johnson & Newport suggest cautiously that the pattern of errors across linguistic variables may be due in part to 'universal factors in learnability' and not to 'item difficulty or transfer from the first language' (1989:92). They refrain, however, from further generalizations about the linguistic domains of NS-NNS divergence, since their sample is limited.

At the center of discussion about these two basic questions is the Coppieters 1987 study. Coppieters compared the grammaticality judgments of 20 French natives with those of 21 near-native speakers of French who had acquired French as adults. These NNS were from a variety of Ll backgrounds and were observed to speak French at a proficiency level comparable to that of natives. The sentences judged were instantiations of nine linguistic variables, including semantic contrasts such as tense/aspect distinctions and syntactic conditions such as clitic raising. Coppieters observed that the performance of NNS diverged significantly from that of NS. However, the divergence was less dra- matic on those aspects of language 'normally covered by the term UG' (565), such as cliticization and raising of en, than for 'functional' distinctions such as passe compose vs. imperfect.

Thus, while attesting NS-NNS competence differences, Coppieters finds that the magnitude of differences is not uniform across all linguistic structures. The asymmetry appears to reflect a distinction motivated in linguistic research, namely, whether a given structure is susceptible to analysis in terms of UG theory. Though provocative, the results of the Coppieters study are not immune to criticism on both conceptual and methodological grounds. Because of the severity of these problems, to be discussed in detail below, a reexamination of ultimate attainment is warranted.

An additional motivation of the present study is to contribute to the ongoing debate on the maintenance of UG throughout maturation, i.e. its availability and mediating function in adult L2A (for a compilation of opposing viewpoints in this debate, see Eubank 1991). Empirical studies by Johnson & Newport (1991) and Schachter (1989) have shown that adult L2 learners appear unable to access fully the universal syntactic principle of subjacency (Chomsky 1981). In contrast, other research (for a selection, see Flynn & O'Neil 1988) suggests

708




that adults are able to demonstrate knowledge of linguistic structure that is unavailable from their knowledge of L1 or from target-language input.

A variety of arguments for the availability of UG in L2A has been advanced. For example, duPlessis et al. (1987) have shown that learners' hypotheses for linguistic form, even if inconsistent with facts of the target language or the L1, are nonetheless consistent with constraints on natural language given by UG. Another argument addresses the epistemology of UG-type structures versus that of functional features of language. Representations of UG structures in L2A are thought to be deductive consequences of innately-given knowledge, while acquisition of functional, peripheral, and language-specific features de- pends on induction from primary linguistic data. Flynn & Manuel (1991:131- 32) and Zobl 1992 cite studies which attest to adult L2 learners' apparent su- periority on judgments of UG-type structures relative to that for other types of linguistic features. These results suggest that UG is accessed by adult learners, and that its mediating effects surpass those available through domain- general learning. Thus, consistent with Coppieters 1987, the decline of language learning ability is apparently more pronounced in 'surface aspects of L2 language knowledge ... than [in] more abstract subsystems of principles and rules of UG' (Flynn & Manuel 1991:131). The present study further examines this putative asymmetry by comparing learners' ultimate attainment for formal, UG- type structures with that achieved on functional or language-specific features.

This investigation also represents a response to Long's challenge to produce credible evidence that NNS exposed after puberty to a second language can perform like NS of that language. My starting point is an attempt to replicate the competence differences found by Coppieters (1987). In the interest of re- liability and validity of results I depart in certain respects from Coppieters' experimental design, procedure, and methodology. The two studies are alike, however, in that the crucial data are NS and NNS acceptability judgments, supplemented with evidence from a concurrent think-aloud procedure.

The results of the acceptability judgment task suggest answers quite different from those offered by Coppieters. In a nutshell, NNS and NS in the present study are found not to diverge dramatically in their judgments. In fact, several NNS in this study fall squarely within the range of performance attested by NS. Moreover, what NNS-NS divergences there are do not appear to cluster into tidy patterns consistent with the notion that UG structures are somehow privileged relative to other types of target-language structures.

As a complement to these findings, I report data bearing indirectly on the source or causes of NNS-NS differences. Specifically, I document the relationship of a number of biographical variables (e.g. age of arrival in the host country and age of beginning study of the target language) to NNS-NS divergences. These results will be shown to be generally consistent with those of Johnson & Newport 1989 and Patkowski 1980.

In addition, I review briefly the results of two further tasks, which involve sentence- and word-level interpretation. Evidence from all three tasks con- verges on the conclusion that for some individuals ultimate attainment in L2A can be congruent with that of LIA.

709




THE COPPIETERS STUDY

2.1. Coppieters 1987 presents evidence suggesting that competence differences obtain between NNS and NS. His results are summarized in a table and a figure, reproduced below.

Figure 1 (Fig. 2, Coppieters 1987:554) displays cumulative deviance from NS 'prototypical norms', by individual NS and NNS subjects, across the 107 sentences tested. Coppieters' operational definition of a norm is in the form of a nominal statement of grammaticality for a given item, as established by a majority of NS in the experimental sample. Thus if more than half of NS reject an item, the norm for that item is 'ungrammatical'. Coppieters offers an affir- mative answer to the general question of NNS-NS differences, based on robust evidence (553):

'[The figure] inaicates that NS's and NNS's differed significantly in terms of the quantifiable judgments they expressed. The NS's varied from the norm on between 5% and 16% of the sentences on the questionnaire. They show up as a fairly compact group at the bottom of the figure. The NNS's varied on between 23% and 49% of the sentences: they appear in a separate and somewhat less compact group at the top. The NNS who is closest to the natives, Sp[eaker] 20, is just over 3 standard deviations away from the NS mean. Thus it seems fair to say that even he could not belong to the native group (p < .005).'

Evidence relevant to the question of locus of competence differences is provided in Table 1 (= Fig. 3, Coppieters 1987:554). In the leftmost column are displayed the linguistic variables, i.e. the types of structures tested. These structures are described in Appendix 1 of the present study. Just to their right, in parentheses, appear the numbers of tokens of each type. In the next two columns, Coppieters shows NS and NNS deviations from the prototypical NS norm for each of the nine linguistic variables. In the rightmost column are

SPEAKERS _ nnnative:

0 5 10 15 20 25 30 35 40 45 50

FIGURE 1 (= Coppieters Fig. 2 [1987: 554]). Cumulative divergences of Coppieters' native and near-native subjects from native norms, expressed as percentages of the 107 items tested.

710




SECTION (with no. of sentences) NSs NNSs NNSs - NSs Impf. vs. P.Comp. (5) 2% 41.5% 39.5% a vs. de + Inf. (2) 6% 40.7% 34.7% Place of the Adj. (14) 11.8% 38.1% 26.3% Il/Elle vs. ce (10) 10% 35.7% 25.7% Articles (8) 15% 34.9% 19.9% Obj. + Pred. (28) 14.3% 33.2% 18.9% Caus. & Clitics (24) 8.5% 27.3% 18.8%o de + Adj. (10) 6% 22.9% 16.9% A-over-A (6) 5% 19.4% 14.4%

TABLE 1. (= Coppieters Fig. 3 [1987:554]). Section-by-section analysis of deviation from the prototypical norm. The first column gives the percentage of deviation from the norm for the

NSs as a group, the second for the NNSs as a group. The third indicates the difference between native and near-native divergence.

expressions of the difference between NNS departures from the norm and NS departures from the norm.

It is on the basis of these data-and an informal analysis of think-alouds, to be discussed below-that Coppieters concludes that NS-NNS competence differences fall into two classes. For items which Coppieters considers represen- tative of UG-type structures (Object + Predicate, Causatives & Clitics, and A-over-A), the magnitude of NNS-NS differences-using data in the rightmost column-is small (average = 17.4%) when compared to differences on items Coppieters classifies as representing '"functional" or "cognitive" aspects of grammar' (565), i.e. Imparfait vs. Passe Compose, a vs. de + Infinitive, Ad- jective Placement, illelle vs. ce, Articles, and de + Adjective, for which the average deviance is 27.2%.

2.2. PROBLEMS WITH THE COPPIETERS STUDY. There are numerous procedural and methodological features of the Coppieters study that compromise its con- clusions, especially as concerns the locus of competence differences. First of all, it is unclear which of Coppieters' linguistic variables should be included under the UG umbrella. The author states that the scope of UG is 'formal areas of grammar' such as 'constraints on grammatical form, logical form, complex syntax, etc.' (565). Coppieters excludes the ce vs. illelle distinction from the domain of UG, as he offers a purely functional analysis for these French pro- nouns (555-57). Yet many of the factors underlying the distribution of ce and illelle are amenable to a formal syntactic analysis. In fact, Coppieters himself acknowledges Pollock's 1983 treatment of the contrast, which hinges on case and theta-role assignment. Similarly, the aide + Infinitive construction in French, for which Coppieters presents a semantically-based account, also ad- mits of formal structural analyses (reviewed in Mark Long 1974:Ch. 4). Lin- guistic theory is fluid and controversial; the consequent variability of the presumed scope of UG undermines Coppieters' claim that near-natives diverge less from natives on UG-type linguistic constructions. Thus if the alde + In- finitive variable and the ce vs. illelle variable are included in the purview of UG, then the average deviance for + UG items jumps from 17.4% to 22.5%, while the average for -UG items falls to from 27.2% to 25.7%, essentially neutralizing the putative difference.

711




Manipulating the +/- UG distinction in terms of Coppieters' percentage figures turns out to be something of a vacuous exercise, however, since many of the percentages are based on small numbers of exemplars, thus lowering their face validity. Recall that the percentages for NNS-NS deviance are derived from varying numbers of tokens of each linguistic variable. At one extreme of NNS-NS divergences are the variables Imparfait vs. Passe Compose (39.5% divergence) and a vs. de + Infinitive (34.7%), which are exemplified by 5 tokens and 2 tokens, respectively. At the other extreme is the A-over-A variable (14.4%), exemplified by 6 tokens. As numbers of tokens decrease, percentage figures become less valid indices of response patterns. It would be a simple matter, for example, to deflate or inflate the deviance for the ai vs. de + Infinitive category by changing (or adding, or subtracting) a single well- chosen item. Clark 1973 has demonstrated the dangers of inference from few tokens of a given category: there is no evidence that the findings will generalize beyond the items sampled and apply to the category as a whole.

A contingency in data elicitation further clouds the issue of locus of competence differences. For some 41 of the 107 items, subjects were given a choice between paired forms (e.g. Est-ce que tu {as sulsavais} conduire dans la neige? 'Did you {manage/know how} to drive in the snow?'). They were then asked by the experimenter to decide: (1) whether both forms were acceptable; (2) if so, whether a difference in meaning was entailed in the differing forms; (3) if so, 'what that difference consisted of in the framework of sentence interpretation' (550). These 41 items were tokens of five linguistic variables: the Im- parfait/Passe Compose distinction, il/elle vs. ce, prepositions a and de + Infinitive, prenominal vs. postnominal adjectives, and article use. The other elicitation procedure required subjects merely 'to provide straightforward well- formedness judgments' (550) for single (unpaired) sentences. This was done for 'most of the other 66 sentences' (550), representing four linguistic categories: the Object + Predicate construction, Causatives and Clitics, A-over-A constraint, and the de + Adjective construction. Reviewing the data in Table 1, above, one may discern a striking pattern. For the top five variables-just those variables tested by the first technique-NNS diverged dramatically from the NS (NNS-NS average = 29.2%) and the range of NNS-NS divergence is disparate (19.9%-39.5%). For those variables tested under the second procedure, on the other hand, the NNS-NS divergence is relatively small (average = 17.3%), with the range clustering tightly (14.4%-18.9%). Comparing these figures with the deviance figures for the +/- UG distinction (average for putatively -UG items = 27.2%, range = 16.9%-39.5%; average for putatively +UG items = 17.4%, range = 14.4%-18.9%), one notes that the magnitude and heterogeneity of NNS-NS divergence are somewhat better predicted by the procedural contingency than by Coppieters' version of the +/- UG distinction.

The issue of elicitation procedure has special significance for de + Adjective structure. Coppieters registers his surprise that NNS should diverge so mini- mally from NS (16.9%) on such a 'comparatively obscure' construction (561). It is an ad hoc move to invoke frequency as a predictor of divergence for this

712




structure alone. Left unmentioned is the fact that minimal divergences for this structure are an embarrassment for the + /- UG distinction as a general predictor of NNS-NS differences. As it happens, the de + Adjective construction was tested under the second procedure; for it and the other structures tested in this manner, NNS-NS divergences were small. There is no anomaly to account for if one allows for the possibility that the asymmetry in NNS-NS differences is an artifact of a procedural asymmetry. Coppieters is apparently aware of the procedural asymmetry, but does not attribute differential results to it (553-54).

Though one can only speculate about whether differences in procedure are in fact causally related to differences in results, it is nevertheless easy to see how there might be a relationship. The first procedure involved a three-part, fine-grained decision task in the context of a face-to-face interview with the experimenter. In the second procedure, subjects were asked merely to give nominal statements of well-formedness. In the latter instance, it is unlikely that subtle cues could be communicated by the experimenter to the subjects. How- ever, that likelihood necessarily increases in an elaborated discussion of form, meaning, and interpretation, as was the case in the first procedure.2 I offer this observation about the relative possibility of experimenter effects as a statement of objective fact about experimental procedure (see Heringer 1970, Labov 1975, Rosenthal 1966), not to dispute Coppieters' assertion that he reinforced all subjects in their judgments and gave no indication when judgments diverged from native norms (552-53). Experimenter effects can be unintentional and unnoticed; measures should be taken to reduce their likelihood.

More to the point, one must consider the nature of the data resulting from the two different elicitation procedures. Recall that the results displayed in Fig. 1 and the two left columns of Table 1 represent cumulative departures from NS norms, the norms being in the form of nominal statements of acceptability or grammaticality. The second procedure generates just such nominal statements for each item, and departures from these norms are straightforwardly tallied. The first procedure, by contrast, involves direct comparisons of two sentences. In this case, the subjects declare that the sentences are equally acceptable, or that they prefer one sentence over the other. Thus in one procedure judgments of acceptability reflect assessments against some external criterion of acceptability, while in the other procedure the judgments reflect comparisons of one sentence with another. A further difference between the procedures should also be noted. The first procedure, in eliciting three-part answers, generates complex commentaries on subtle interactions of syntax and semantics. Anyone who has ever elicited (or offered) such comments recog- nizes that they may contain references to imagined contexts, statements re- flecting judgments based on degrees of well-formedness, expressions of

2 See Carroll et al. 1983 for discussion of distortion of judgments when subjects consider themselves 'on the spot'. Arguably this was the case of Coppieters' NNS in the interview context. Forced introspections may result in rationalizations or retreats to a perceived prescriptive norm; see Berthoud 1982:166ff.

713




ambivalence, etc. This type of data may be contrasted with the simple nominal statements ('correct or good'/'incorrect or bad') generated by the second procedure. (Moreover, what is meant by 'good' and 'bad' is not spelled out for participants, creating the possibility of intersubject variability as to the basis of judgment.) In light of multiple differences between the two procedures, one must recognize that their results are inherently dissimilar and therefore not comparable. The two types of data should not be lumped together as if they were identical.

However, Coppieters seems to do just that. The results of the first procedure, in order to be considered relevant input for the NNS-NS comparisons displayed in Fig. 1 and Table 1, have apparently been converted into nominal data. The details of this conversion operation are not spelled out. How does one establish a unique NS norm, and nominal departures from it, when the task generates a complex response? By what algorithm is a comparison of two related items, with an accompanying three-part, fine-grained discussion, transformed into an independent statement of acceptability for each of the items?3 These questions speak to the reproducibility of the results, as well as to their validity.

With respect to the more general question of whether there are NNS-NS competence differences, Coppieters' methodology is again problematic. Recall that the differences he attests are derived from comparisons of NS and NNS deviations from a prototypical native norm. The norms represent the majority of NS nominal judgments. For 90 of the 107 items, there was 80% or more agreement in judgment among NS. For the other 17 items, 'the majority opinion of NS's' (553) must be understood as any level of agreement above 50%. Clearly, it would have been desirable to establish norms at a high level of NS agreement for all items. Even if this had been the case, one must question the use of nominal data to establish norms. Nominal judgments are insensitive to degrees of grammaticality or acceptability. Chaudron 1983 has argued that scaled judgments of grammaticality (e.g. 1 = totally unacceptable/ungrammatical; 5 = perfectly acceptable/grammatical)4 are more informative than nominal data, inasmuch as they permit intergroup comparisons (e.g. t-tests and ANOVAs) on the basis of means and variance from means. The psychological validity of scalar grammaticality statements is supported by Barsalou 1987 and others working in category/concept theory. The category of well-formedness is susceptible to gradedness effects in experimental performance, just like other categories such as BIRD (robins and sparrows are judged more birdlike than emus, ostriches, penguins, and apteryxes). Moreover, Chomsky 1986 has argued for relative grammaticality in syntax as a function of the number of barriers

3 Coppieters employed yet another elicitation procedure for the alde contrast. These prepositions were not embedded in stimuli sentences; rather, speakers were asked to make up their own sentences, interpret them, and judge them (561). It is difficult to see how this procedure can work consistently and predictably. Nor is it clear how the results (interpretation of a vs. de coupled with judgments of acceptability) can be reliably translated into nominal statements of native norms.

4 It is widely recognized that grammaticality and acceptability are not synonymous, and that there are serious problems with eliciting judgments and using them as evidence for grammatical competence. Results of such elicitations must be interpreted cautiously; see Birdsong 1989.

714




crossed in extractions. In sum, there are numerous reasons for preferring scalar responses to nominal statements in determining differences between NS and NNS.

The composition of the subject groups in the Coppieters study presents additional problems. To qualify for participation, nonnative subjects had to perform at a near-native level of speech, as determined by native-speaker friends of the subjects and by the experimenter himself. In addition, by Coppieters' reckoning, all performed orally at a level of 'superior' by ACTFL criteria. (For a description of the ACTFL oral proficiency interview, see ACTFL 1986.) However, both of these screening procedures are imprecise and nondetermin- istic. Informal near-native ratings cannot assure native-like performance across a variety of linguistic structures. It is quite possible to sound 'native-like' by avoiding production of complex structures which are not fully mastered. It is also possible to sound 'native-like' in restricted contexts, such as bank trans- actions or philosophy lectures. Indeed, for restricted and practiced contexts, there may be apparent pragmatic mastery, while for other contexts there may be manifest inadequacies. The second screening measure leaves a great deal of latitude for composing the near-native group, inasmuch as the ACTFL category of 'superior' ranges between the 3rd and 5th levels of a 5-point scale. By virtue of these screening procedures, it is quite possible that the near-native group was not particularly proficient. It is not clear that Coppieters satisfies the desideratum of Long 1990 for investigating the general question of competence differences, viz., that at least some subjects sampled be among 'the very best' L2 learners.

The biographical profiles of NNS and NS subjects pose further problems. As has been documented by Ross 1979 and others, one may expect differences in grammaticality judgments-even among native speakers-if sociological and educational variables are not controlled. Coppieters' 21 NNS were 'highly educated' (551) and included 19 professors or researchers, eleven of whom were professors of linguistics or literature. In contrast, his 20 NS had merely 'had some education' (551). Alone, such background differences could account for a good deal of the NNS-NS deviance. That is, failure to control for background differences between the NNS and NS increases the chances that the two groups will not coincide on judgments for 'subtle areas' (Long 1990:281) of the grammar, just as NS or NNS with heterogeneous backgrounds would be likely to disagree among themselves.5

5 It is worth pointing out that within Coppieters' NS group 'the level of education ... was more varied than that of the NNS group. Five of the NS's were themselves linguists, along with two engineers, two housewives, two professors of English, one professor of Spanish literature, two special educators, two professors of French, one public employee, one secretary, one caretaker, and one retired public employee' (551). This heterogeneity, plus the fact that the speakers represented a variety of dialects from France and Belgium, may have contributed to a robust intragroup variance: recall that the range of deviation from the native norm was between approximately 5% and 16%. That is, some NS subjects deviated from the native norm three times more often than others.

An anonymous Language reviewer commented on the background differences issue. If I understand her/him correctly, the high level of education among the NNS 'would put NNS at an

715




The risks of heterogeneity are also evident in the native language backgrounds of the NNS. Recall that Coppieters' subjects are from a variety of disparate linguistic backgrounds. In principle, this design feature would allow one to determine whether results generalize beyond speakers of a single native language. However, with only 2 subjects representing Farsi, only 5 subjects representing 'Oriental' languages, and 6 different native languages among the 14 native speakers of Germanic and Romance, one cannot assume validity of results for the individual native language groups, much less generalize across groups.6

Because of these shortcomings, and others that will come to light in later sections of this paper, the Coppieters study must be viewed with some skep- ticism. At the very least, it should be evident that the two main questions of maturational effects in L2A-whether there are competence differences between NNS and NS, and if there are, which linguistic domains are affected- remain open. Further examination of the issues is therefore warranted. As a first step, it is important to determine whether the principal findings of Cop- pieters can be replicated with methodological modifications and under conditions of tighter procedural control.

THE PRESENT STUDY

3. As mentioned above, the present study takes as its point of departure Coppieters' investigation, and reexamines the two main questions from several perspectives.

Three tasks were administered in the course of a single experimental session. The first task involved judgments for the most probable interpretation of decontextualized ambiguous sentences. The second task required subjects to examine decontextualized sentences containing the adverb bien, and to decide which of several possible meanings for bien was the most appropriate. The third task required subjects to judge the acceptability of French sentences chosen to exemplify + UG and - UG constructions.

Details of subject selection, procedure, and rationales underlying the choice of experimental task will be presented in the sections that follow.

advantage' relative to NS and make even more impressive Coppieters' finding of competence differences. Education may confer an advantage when numbers of 'right' and 'wrong' answers, as determined by prescriptivists, are the relevant data (consider judgments of English sentences with stranded prepositions or split infinitives, and case markings in sentences such as 'Who/Whom did you see?'). However, in the Coppieters study and in the present study, external criteria of grammaticality-as designated by theoretical linguists, descriptive linguists, or prescriptivists-are not in question. What is at issue is whether near-natives and natives agree with each other. The fact that one or both groups may disagree with determinations made by linguists or prescriptive gram- marians is a separate matter. (I touch on this issue briefly in ?3.5.)

6 To see if there is any evidence of generalizability across native languages in the Coppieters data (see his Figure 4, p. 563), I carried out correlations of performance for each native language group with every other, for all linguistic structures. There were no significant correlations (p < .05); that is, the groups did not perform similarly across linguistic structures. On the possibility that there might be more generalizability across native languages forjust those structures Coppieters deems + UG, I carried out intergroup correlations using these variables only. Again there were no significant correlations. The same results obtained for the - UG structures.

716




3.1. SUBJECTS. In the present study there were two groups of subjects: 20 native speakers of English who were near-native speakers of French, and 20 native speakers of French. To avoid confusion with Coppieters' subjects or with natives and near-natives in general, I will hereafter refer to the natives in the present sample as FNS (French Native Speakers) and to the near-natives as ENS (English Native Speakers).

Recall that Coppieters tested few subjects from many native language backgrounds. This design feature yielded results of dubious validity for the individual groups and made for risky generalizations across groups. Here, a single native language background is represented in the ENS group. This choice en- tails a tradeoff: the validity of the results is enhanced, but any interpretation of findings must be restricted to the sample. No claims are made for generalizability to near-natives from other mother-tongue backgrounds.

The subjects were the same for all three tasks. Measures were taken to ensure that FNS and ENS were comparable along relevant sociological, educational, and linguistic dimensions. The principal criteria for the nonnatives were three years of continuous residence in the host country prior to testing (see Johnson & Newport 1989) and exposure to the target language after puberty. In addition, all ENS spoke French fluently. Some had detectable accents, however. More formal screening for proficiency was not carried out for two main reasons. First, formal instruments for determining native-speaker-like proficiency, such as the ACTFL interview, are problematic (see discussion in ?2.2 above). Sec- ondly, I was interested in correlations of ENS subjects' experimental results with an assortment of biographical variables. These included not only years of residence, years of study, etc., but also self-assessed mastery of French. Thus it was not desirable to restrict the ENS sample to a narrow range in terms of proficiency; it sufficed that they be fluent and roughly comparable in their spoken French. Details of the composition of the groups are as follows:

FNS. All were college-educated. Their ages were not elicited, but the average age was estimated at 35-40 years. All were linguistically naive; none were teachers of French. All were speakers of standard French; all had lived and grown up in major metropolitan areas of France (Paris, Nice, Nancy). The sample included 13 women and 7 men.

ENS. All were college-educated. Their average age was 40 years. All were linguistically naive and none were teachers of French. All but two had had exposure to formal instruction in French; the average length of study was 6 years, with a range of 1-14 years. All had been exposed to French or had begun French study at or post puberty (average = 14.9 years; range = 11-28 years). All had been living continuously in France (almost exclusively in Paris) for at least three years immediately prior to testing (range 3-36 years). The average cumulative (not necessarily continuous) length of residence was 11.8 years. The average age of arrival in France was 28.5 years, with a range of 19-48 years.7 The sample included 10 women and 10 men.

7 Most ENS subjects did not fit the classical immigrant profile: they had not moved to France definitively with their families, but had travelled to and from France on several occasions, usually

717




Subjects were recruited by conventional methods, principally through posted advertisements in libraries, bookstores, and university buildings. A few subjects were personal friends. Many were acquaintances of other subjects. Sub- jects were given a modest compensation of 50 F for their participation.

3.2. THE ACCEPTABILITY JUDGMENT TASK. The acceptability judgment task was the third in the sequence. Results and procedures for the two other tasks will be reported in ?3.7.

PROCEDURE. All experimentation was carried out in France. This measure was instituted to avoid possible 'contamination effects' that an ambient language other than French might have on subjects' intuitions. The task was administered to subjects individually. Most sessions were conducted in an office at the Laboratoire de Sciences Cognitives et Psycholinguistique in Paris. The experimenter left the room after giving instructions on the pencil-and-paper and think-aloud portions of the task (see below).

Subjects were presented with 76 grammatical and ungrammatical French sentences in written form and were asked to render scalar (1-5) acceptability judgments. They were instructed to make their judgments along the following scale: A = pas du tout acceptable; je ne la dirais pas 'not at all acceptable; I would not say it'; B = acceptable dans de rares contextes 'acceptable in rare contexts'; C = acceptable dans a peu pres la moitie des contextes 'acceptable in about half the contexts'; D = acceptable dans la plupart des contextes 'acceptable in most contexts'; E = tout a fait acceptable; je la dirais 'completely acceptable; I would say it'. Subjects indicated their judgments on computerized answer sheets and were instructed not to look back at previously-judged items. For purposes of statistical analysis, A's were given a numerical value of 1, E's were converted to 5's, and so on.

Note that the criteria for the extremes of the scale (A and E) are defined in terms of subjects' own grammar (je ne la dirais pas/je la dirais). This was done to decrease the likelihood that subjects would appeal to prescriptive statements of 'standard' French, to make the criteria seem reasonable and comprehensible to all subjects, and to make the instructions as unambiguous as possible, leaving little room for interpreting or second-guessing. Note also that the B, C, and D points on the scale require subjects to imagine possible contexts where the items might occur. This feature was instituted to guide all subjects toward a single search set (Tversky & Kahneman 1974) prior to judgment. (Imagining contexts is a search set often employed by subjects even without instruction to do so. Appropriateness of an item in numerous and easily-imagined contexts appears to be the basis of judgments of acceptance; see McCawley 1985.) The

for only brief stays. Clearly, it would be unreasonable to use the date of their first visit to France as their age of arrival in France. As a matter of necessity, therefore, this figure was derived by subtracting subjects' cumulative years of residence in France from their age at the time of the survey. Obviously, for those subjects who do fit the immigrant profile, the derived age of arrival and the 'true' age of arrival would be the same.

Correlations of this and other biographical variables with deviance from NS norms will be reported in ?3.6.

718




points of the scale are so worded (pas du tout acceptable, de rares contextes, la moitie des contextes, la plupart des contextes, tout a fait acceptable) as to permit assignment of numerical values (1, 2, 3, 4, 5) to these points respectively, and thus to license the assumption of roughly interval data.

As subjects carried out the pencil-and-paper part of the task, they provided parallel think-aloud data, i.e. verbalizations of their thought processes as they decided on the acceptability of the stimuli (e.g. Ericsson & Simon 1984, Faerch & Kasper 1987). These data were unforced; that is, subjects were not required to say something about each item. This procedural feature was instituted to avoid rationalizations which might result if subjects were obliged to comment on every sentence (see comments on forced introspections, n. 2). Think-alouds were tape recorded on a cassette recorder manipulated by the subjects themselves. Unlike the Coppieters study, the experimenter was not present during the procedure. This feature was intended to eliminate the possibility that the experimenter could influence judgments by subtle linguistic or paralinguistic cues. Because verbal reports were being elicited, the acceptability judgment task was perforce unpaced.

Consideration of think-alouds was taken into account in deciding on order of item presentation. Presentation in random order for each subject or in two order-reversed versions has the advantages of minimizing fatigue effects and rating distortions resulting from subjects' having previously judged similar items. However, it is often the case that subjects' comments for a given item will include references to prior items. Thus the decision was made to have a single order of presentation in order to ensure comparability of comments for a given item across all subjects. Summaries of the think-aloud protocols will be discussed in ?3.4 below.

In addition to the written instructions for the acceptability task and the verbal reports, an explanation of the general purpose of the experimentation was given orally to each subject. They were informed that they were participating in a study of intuitions about the French language. They were asked to try to answer spontaneously, not to attempt to analyze the items. It was reiterated that there were no 'right' or 'wrong' answers. Subjects were told, however, that the answers of certain groups of subjects would be compared to those of other groups. To reinforce the nonevaluative nature of the experimentation, subjects were assigned numbers and were told that their anonymity would be maintained.

Written instructions for the think-aloud procedure made it clear that subjects should not try to rationalize or justify their judgments. The purpose and significance of the think-aloud data were explained orally. The following example was given to illustrate the interest in knowing what people are thinking when they make a decision: 'Two people may agree that the Ferrari is the greatest make of car in the world. But one may think it's best because of its engineering and speed, and another may believe it's best because of its styling and snob appeal.'

To minimize fatigue effects, a rest period was permitted after approximately half the items were judged.

719




STIMULI. The stimuli were 76 sentences exemplifying the following structural features of French: Adjacency, en-Avant, Prenominal Past Participle, that- Trace, de + Modifier, celil (elle), and Middle Voice. A brief description of each of these linguistic variables appears in Appendix 1, which also includes comments relating to choice of structures and comparisons with Coppieters' items.

In choosing the linguistic variables, efforts were made to: (1) include as many of Coppieters' variables as possible; (2) exclude those variables in Coppieters' study which posed problems for elicitation or interpretation; and (3) include new structures to permit testing of the + /- UG distinction. In addition, following Long 1990's call for more demanding tests of ultimate attainment, it was important to avoid structures that are 'too easy' (274) and to exemplify instead 'subtle areas' (281) of the target-language grammar, i.e. syntactic phenomena more complex and less frequently-occurring than those tested in John- son & Newport 1989. The resultant seven variables included four that were tested by Coppieters: de + Modifier, celil (elle), en-Avant, and Prenominal Past Participle. (As spelled out in Appendix 1, the last two of these are actually restricted subsets of the variables tested by Coppieters; see the Appendix for specific category and terminology differences between my variables and his.)

Appendix 2 lists the 76 items exemplifying these structures. Approximately half were grammatical and half ungrammatical (*), as determined in the relevant theoretical literature; in a few cases, the grammatical status was questionable (?). Considerations bearing on the choice of exemplars of each variable included using items tested by Coppieters and increasing the numbers of exemplars for each variable (see Appendix 1). In cases where new items were added, just those exemplars found in the literature were used. This was done to deflect the frequent suspicion that results may be artifacts of an experimenter's bias in choosing or concocting tokens. The only exemplars not taken verbatim from Coppieters or from the relevant literature in theoretical linguistics were certain that-Trace items which were straightforward translations into French of well- known minimal contrasts given in the literature on that-Trace effects. For some items, a few words were added to aid comprehension. Thus, for example, the item in Coppieters 1976, C'est un intellectuel qui n'a peur de rien, was changed to Bernard Pivot, c'est un intellectuel qui n'a peur de rien 'Bernard Pivot [he] is an intellectual who is afraid of nothing' (Item 10). The source of the items is given in parentheses.

RESULTS. A comparison of ENS and FNS judgments for each of the 76 items was performed using a two-tailed t-test procedure. A summary of the results, by linguistic variable, is given in Table 2. A complete record of these results, by linguistic variable and by item, is given in Appendix 2. The results in Table 2 reveal that ENS and FNS differed significantly in their judgments on 17 of the 76 items. This translates into an ENS-FNS difference on approximately 22% of items overall. The linguistic variable with most divergences was Middle Voice, where ENS judgments differed from FNS judgments on 4 out of 9 exemplars. The variable with fewest divergences was celil (elle), for which there were no differences for any of the 10 items.

720




NUMBER OF INTERGROUP DIFFERENCES/ DIFFERENCES

LINGUISTIC VARIABLE NUMBER OF EXEMPLARS AS % OF EXEMPLARS

ce I il (elle) 0/10 0% en-Avant 1/9 11% Prenominal Past Participle 1/6 17%

Adjacency 3/16 19% de + Modifier 4/16 25% that-Trace 4/10 40% Middle Voice 4/9 44% OVERALL 17/76 22%

TABLE 2. Summary of statistically significant intergroup differences, by linguistic variable and across all exemplars.

Divergence on 22% of items overall offers only modest evidence of differences in ultimate attainment. That is, these data constitute little support for the notion of generalized competence differences between natives and near- natives.8

The distribution of ENS-FNS differences by linguistic structure speaks to the linguistic domain of differences between natives and nonnatives. Among the uncontroversially +UG variables (en-Avant, Adjacency, that-Trace), the results are mixed, ranging from one departure among the nine en-Avant items, to 3 of 16 for Adjacency, to 4 of 10 for that-Trace. For the lone uncontroversially - UG variable, Prenominal Past Participle, the FNS-ENS divergences are in the middle of the range (1 of 6 exemplars). These results are displayed, with reference to their theoretical status as assigned by Coppieters and other linguists, in Table 3.

It could be argued that a pattern of deviances is discernible whereby performance on + UG items is actually worse than for -UG. Assume that the

8 An anonymous reviewer questioned this method of determining ENS-FNS differences. One

concern was that 76 separate t-tests (comparisons of ENS responses with FNS responses for each of the items) were performed. There is a risk of Type I statistical error (i.e. false rejection of the null hypothesis) when cross-comparisons are made (e.g., results for Item 1 are compared with those for Item 2, then 1 with 3, 2 with 3, etc.). However, no such multiple comparisons were made in the present study. The t-test is therefore appropriate, and an analysis of variance procedure is not required.

The reviewer suggested carrying out 'an overall statistical testing of the hypothesis' that the groups are similar. A simple example will suffice to show that this would not yield desirable data. Imagine that all ENS subjects rated Item 1 as 5 and that all FNS gave the same item a rating of 1. Then imagine that for Item 2 all ENS gave a rating of 1 and all FNS gave 5s. Obviously, the groups are giving different responses. But if one collapsed the data across the two items, the means and standard deviations for the two groups would be identical. The same masking of true differences at the item level would obtain if, say, the results for the 8 items exemplifying a given linguistic variable were taken together, or if the results for all 76 items were collapsed. For the purposes of this study, the meaningful comparisons are at the item level.

It is certainly true that a comparison of the two groups' judgments over all items or for each linguistic variable might reveal differences. However, the differences would be in terms of the direction of judgments. For example, a group mean below 3 across all items would suggest that the subjects tend to reject both grammatical and ungrammatical sentences. Such a comparison was carried out, but there were no significant ENS-FNS differences (alpha = .05) either across all items or by linguistic variable.

721




LINGUISTIC + /-UG + I-UG PROPORTION OF CONSISTENT COPPIETERS' VARIABLE (ACCORDING TO (ACCORDING ITEMS WITH WITH PREDICTIONS

COPPIETERS) TO OTHER SIGNIFICANT COPPIETERS' OF NS/NNS

LINGUISTS) DIFFERENCE RESULTS? DIFFERENCES

BETWEEN

FNS & ENS (SEE TABLE 2)

en-Avant +UG +UG small (11%) YES Adjacency n/a + UG mid range (19%) n/a small that-Trace n/a + UG large (40%) n/a small Middle Voice n/a +UG(?) largest (44%) nla small, if +UG de + Modifier - UG -UG, +UG mid range (25%) NO ce I il (elle) - UG - UG, + UG smallest (0%) NO Prenominal Past - UG - UG mid range (17%) NO

Participle TABLE 3. Linguistic variables, their status as + UG or - UG, and comparisons with results and

predictions of Coppieters 1987.

top 3 variables (en-Avant, Adjacency, and that-Trace) are + UG and that the bottom 3 (de + Modifier, celil (elle), Prenominal Past Participle) are -UG: for the former the average deviance is 23.3%, and for the latter the average is 14%. Inclusion of Middle Voice in the + UG category creates for the + UG items an even greater ENS-FNS deviance figure (28.5%). However, assuming this classification, one would have to account for the low deviances for en- Avant items (+UG).

Thus there is no neat pattern of results that conforms to the + UG/- UG distinction. In fact, the most conspicuous feature of Table 3 is the apparent absence of patterning along any + / - UG dimension, whether the determination of UG status is made by Coppieters or by others. In other words, there is no evidence for the idea that performance on + UG items differs monolithically from performance on - UG items.

Although the present data offer no clear picture of where differences in ultimate attainment will be manifest, it is clear that these results are not consistent with Coppieters' results, or with his predictions. This can be seen by examining the last three columns. In only one case, en-Avant, do the results resemble Coppieters'. On that-Trace and Middle Voice items, where the +/- UG distinction would predict small differences, the differences were large. On Ad- jacency items (+ UG), the deviance numbers are fairly elevated as well.

The general question of ENS-FNS divergence can be addressed with additional evidence in the form of ENS-FNS cumulative deviance. Table 4, below, is an attempt to represent cumulative deviances from native norms in a manner that permits comparison with Coppieters' graph of cumulative deviance (Fig. l).9 Numerical data in the next-to-rightmost column represent the difference

9 Since Coppieters' starting point was nominal judgments construed as norms, the two graphs are not strictly comparable. Norms in the present study were based on FNS mean judgments for individual items. Deviance from a norm is taken to be the absolute value of the difference between an individual judgment and the FNS mean. Thus if the FNS mean for a given item is 4 (after conversion from letters to numbers) and a subject judges the item completely acceptable (E or 5), the deviance is understood as 1. Cumulative deviance is calculated in this way over all 76 items for each of the FNS and ENS subjects.

722




CUMULATIVE CUMULATIVE

SUBJECT SUBJECT DEVIANCE DEVIANCE

GROUP NUMBER (MAX. = 304) (% OF MAX.)

ENS 3 115.05 37.8% 5 112.65 37.1%

16 107.55 35.4%? 8 105.65 34.8%

17 100.75 33.1% 20 96.05 31.6% 18 94.05 30.9% 2 93.25 30.7%

10 91.25 30.0% 19 85.95 28.3% 14 - 83.85 27.6%

1 77.25 25.4% 7 72.15 23.7% 4 71.85 23.6% 9 69.85 23.0%

13 67.75 22.3% 15 67.55 22.2% 11 67.05 22.1%S 12 _64.35 21.2%E 6 _55.65 18.6%E

FNS 20 196.15 31.6%E 9 -94.55 31.1%E

13 85.55 28.2%> 5 i81.05 26.7%E

10 175.15 24.7%E 2 73.45 24.2%S

12 72.75 23.9%E 4 69.65 22.9%o 1 _68.35 22.5%S

14 67.85 22.3%S 3 _67.85 22.3%E 8 65.65 21.6%S

16 65.35 21.5%S 15 _64.05 21.1%S 7 _60.65 20.0%o

19 _58.45 19.2%o 11 _57.35 18.9% 17 _52.75 17.4%S 18 _51.55 17.0%E 6 lllllllllll 50.85 16.7%o

TABLE 4. Acceptability task. Cumulative deviances from native norms, by subject. Cf. Fig. 1.

Clearly, this is not an ideal method of calculating cumulative deviances, because it is based on means that are insensitive to variance figures. Means for many of the items were around 3, with standard deviations of 1.5 and higher not uncommon. To use a mean as a norm in such cases of great variability is to pervert the notion of norm: deviation from the mean is the 'norm'. However crude this method of calculating cumulative deviance, for NNS-NS comparisons it has to be pre- ferred to the way it is calculated by Coppieters (the theoretical status of a sentence is also inap- propriate for NNS-NS comparisons). Some encouragement may also be taken from the fact that cumulative deviances on two related tasks (acceptability judgment and interpretation of bien) are found to correlate significantly.

723




between an individual subject's response on a given item and the FNS mean for that item, summed over 76 items. Note that the potential difference between the FNS norm and any judgment is 4. For example, if the FNS norm for a given item is 3, deviances could range 2 points above or 2 points below this norm. Over all 76 items, the maximum cumulative deviance is 76 items x 4 points, or 304. In the rightmost column, the raw figures from the preceding column have been converted into percentages of 304.

In the leftmost columns are the subject groups (ENS or FNS) with the identification numbers that were assigned to subjects to preserve anonymity. To the right of this are the deviance data, expressed in bar graphs along the scales of maximum cumulative deviance (304 and % of 304). These bar graphs permit ready visual comparison of Table 4 with Fig. 1.

Casual inspection reveals that ENS subjects in the present study performed much more like FNS subjects than was the case with Coppieters' NNS and NS. Deviance figures for ENS range from approximately 56 to 115, while FNS deviances range from about 51 to 96. The mean deviance for ENS is approximately 85, and FNS mean deviance is 69. Statistical comparison reveals that the groups differ significantly in their divergence from FNS norms (F(1, 38) = 10.66, p = .002). However, the magnitude of this deviance is considerably smaller than for Coppieters' group, where the most native-like NNS diverged more from the NS mean than did the least native-like NS.

Examination of the data for individual subjects reveals that 15 of the 20 ENS subjects have deviance scores falling within the range of the deviance figures attested for FNS. In fact, several ENS (subjects 6, 12, 11, 15, 13) have scores in the lower-to-middle range of FNS deviance; that is, these ENS subjects are comparable to some of the better FNS subjects in terms of deviance from native norms. (For discussion of the relationship of individual subjects' biographical variables to native-like performance, see ?3.6.)

These results shed light on the question of whether nonnatives diverge from native competence. Considered as groups, ENS and FNS subjects are significantly different in their deviance from native norms. Yet, unlike the results of Coppieters, which showed all NNS subjects diverging significantly from NS norms, the present results suggest that some ENS subjects perform well within the range attested for FNS subjects.

3.3. THINK-ALOUD DATA FOR THE ACCEPTABILITY JUDGMENT TASK. In the Cop-

pieters study, the most striking evidence of NNS-NS differences was found in an examination of subjects' comments for 41 items representing the following linguistic variables: Imparfait/Passe Compose, il/elle vs. ce, a and de + In- finitive, prenominal vs. postnominal modification, and article use. Reviewing these commentaries, Coppieters found that NNS in some cases seemed to appeal to grammars different from those of NS. For example, with respect to the ce/il (elle) contrast, many NNS were characterized by 'a clear tendency to understand il/elle as more of a people pronoun, and ce as more of a thing pronoun' (555). Other NNS viewed the distinction as a matter of stylistics, with il/elle appropriate for more formal contexts than ce. Coppieters argues

724




that NNS and NS differ profoundly in their interpretations of the meanings of grammatical contrasts, as in the case of the contrasting readings of the verb savoir 'to know' in the passe compose and imparfait (Est-ce que tu {as sulsavais} conduire dans la neige? 'Did you {manage/know how} to drive in the snow?'). Subjects' comments were interpreted as evidence for greater NNS-NS divergence on - UG items.

Recall, however, that the verbal reports Coppieters elicited did not extend to all structures, making comparisons of - UG and + UG items problematic. Moreover, Coppieters only selectively documents the numbers of NNS who diverge from NS norms for a given item. Thus there are gaps in both the elicitation and presentation of the commentaries.

In the present study, verbal reports were elicited for all linguistic variables and all exemplars. To make the task comparable for both subject groups, the think-alouds were performed in French by ENS as well as FNS groups.

Below (?3.4) I present a summary of think-aloud data. In the section that follows (?3.5), I break down the comments by linguistic variable and review each set of comments. For reasons of space, these reviews will be brief; an exhaustive and in-depth examination is better left to a subsequent study.

3.4. SUMMARY OF THINK-ALOUD DATA (GRAND TOTALS). Because the data were unforced, not every subject commented on every item. Among the 20 ENS subjects, 18 chose to offer comments. There was a total of 903 ENS comments, out of a possible 1368 (18 subjects x 76 items), yielding a participation index of .66. (If the participation is calculated on the basis of 20 subjects x 76 items [1520], the participation index is 903/1520, or .59.) In the FNS group, 14 subjects offered a total of 720 comments out of a possible 1064 (14 x 76), yielding a participation index of .68. (The participation index on the basis of 1520 is .47 for FNS.)

Close examination of the think-aloud protocols yielded 7 principal categories of comments for which quantitative results could be tallied. Table 5 summarizes the results of these categories. Below the table is a key to the abbreviations for category labels, along with explanations of criteria for listing comments under a given category. In some instances, subjects made several comments about a single item. Although each of these comments is tallied, no single comment can belong to more than one category. That is, the categories are to be considered discrete.

ENS subjects made a total of 622 substantive comments. Not included in the substantive comment count are simple expressions of acceptance or rejection such as jamais 'never', fa va 'OK', pas de probleme 'no problem', or statements to the effect that the response 'A' was chosen, etc. There were 559 FNS substantive comments.

RIGHT SITE. Comments classified as Right Site spoke directly to the grammatical feature under consideration, e.g. the de in de + Modifier constructions, the position of the modifier in Prenominal Past Participle items, etc. Some 49% of ENS substantive comments and 32% of FNS substantive comments were so classified.

725




ENS RS (WANS) IRREL COMP- COMP + SEM STYL CONTEXT SUBST # SUBJS

COMM COMM

305 (58) 88 64 8 10 74 73 622 903/1368 49% (9%) 14% 10% 1% 2% 12% 12% 100% participation

index = .66

FNS RS (WANS) IRREL COMP- COMP + SEM STYL CONTEXT SUBST # SUBJS

COMM COMM 178 (28) 71 67 49 11 97 86 559 720/1064 32% (5%) 13% 12% 9%o 2% 17% 15% 100% participation

index = .68

TABLE 5. Quantitative summary of think-aloud data. Grand totals, by type of comment, over all linguistic variables (76 items). Raw numbers are given above expressions as % of substantive

comments. Key to abbreviations:

RS = Right Site. (WANS) = Wrong Answer. Since a wrong answer implies that an RS comment was made,

(WANS) numbers are included among the RS totals. Thus parentheses are to indicate that wrong answers are not to be counted a second time in the row totals; i.e., they are NOT added in with numbers for other categories of comments that make up the SUBST COMM category (described below).

IRREL= Irrelevant. COMP- = Incomprehensible. COMP+ = Comprehensible. SEM = Semantics. STYL = Stylistic features not subsumed under any other category above. SUBST COMM = Substantive Comments. The total number of comments qualifying for inclusion

in the above categories. Remarks such as e, jamais 'never', ;a va 'OK', pas de probleme 'no problem', etc. are not considered substantive. However, these are tallied under # SUBJS COMM (described below). As mentioned above, (WANS) numbers are likewise not included in SUBST COMM tally.

# SUBJS COMM = Number of Subjects Commenting. This represents all SUBST COMM and any other remarks not considered substantive. The figure on the left of the slash is the total number of subjects who made comments, the figure on the right is the total number of possible comments, calculated by multiplying 18 subjects X number of items for ENS and 14 subjects X number of items for FNS. The participation index is the quotient of the figures on either side of the slash.

The ENS-FNS asymmetry in such comments would seem to indicate that the two groups are not attending to the same features of the stimulus sentences. Taken at face value, these comments also suggest that, for more FNS than ENS, acceptability is determined not so much by considerations of grammar, but by stylistics and comprehensibility (see review of these categories below). Does this mean that FNS are relatively insensitive to 'grammar'? Do they relegate grammatical contrasts to the domain of stylistics?

Unfortunately, the think-aloud data do not permit a definitive response to these questions. Consider, for example, Item 3, an exemplar of that-Trace violations: *Qui crois-tu qui rendra visite a Marc? 'Who do you think that will visit Mark?' For this item, there were 6 ENS comments classified as RS, e.g., le deuxieme QUI ne marche pas 'the second QUI doesn't work'; il y a un QUI en trop 'there's one too many QUIs'; le deuxieme QUI me gene beaucoup 'the

726




second QUI bothers me a lot'. Only two FNS participants offered such Right Site comments. Instead, FNS subjects tended to refer to stylistic factors. Five of the 12 FNS substantive comments referred to concerns for aesthetics or discourse register: two said the sentence was possible in spoken contexts, one suggested it was the language of the theater, one called the sentence lourde 'heavy', and another said it was pasjolie 'not pretty'. Such comments, without making explicit reference to the qui in question, may in fact have been an indirect way of expressing rejection of just that grammatical feature. That is, one cannot reasonably exclude the possibility that the qui figured in these decisions.

This example points up a well-known limitation of think-alouds: subjects do not always report (and perhaps are not always aware of) the feature of a stimulus that triggers a response (see, e.g., Nisbett & Wilson 1977). Furthermore, assigning comments to nominal categories is an imperfect science, since the data are not always discrete. For these reasons, one should be cautious when interpreting intergroup differences.

WRONG ANSWER. This category notes evidence that subjects' judgments run counter to the grammatical status of an item as determined in the theoretical literature. Numbers in this column do not represent all the 'wrong answers' that subjects supplied on the pencil-and-paper part of the acceptability judgment task-only those that surfaced in the think-alouds.

This category of response is of only peripheral interest to the present study. As noted previously, the point of reference throughout is an internal norm as established by FNS subjects, and not an external criterion established by linguists: the essential comparisons are ENS vs. FNS, not ENS/FNS vs. linguists. In the interest of thoroughness, however, I have taken the analysis of the think- alouds a step further to determine numbers of incorrect acceptances of nominally starred items and incorrect rejections of OK items. These figures appear in parentheses beneath the wrong answer numbers. Of the 58 Wrong Answers given by ENS subjects (9% of substantive comments), 43 were rejections of OK sentences, 14 were acceptances of starred sentences, and one was a change of a starred item to another starred sentence. Among FNS subjects, Wrong Answers represent 5% of all comments. Improper rejections and improper acceptances are almost equally distributed, with 14 of the former and 12 of the latter. In addition, FNS subjects changed one nominally '?' item to a nominally starred sentence, and one starred to another starred item.10

IRRELEVANT. This category covers comments unrelated to the grammatical feature under consideration. Typically, these comments are associated with the rejection of an item.

In terms of numbers of Irrelevant comments, FNS and ENS subjects are

10 The preponderance of rejections of OK items by ENS subjects suggests the possibility of a response bias towards low judgments. If there were a response bias in one or the other group, the two groups' average numerical ratings on the pencil-and-paper part of the task should be different. However, as pointed out in n. 8, comparison of the mean responses for the two groups of subjects reveals no significant differences.

727




indistinguishable (13% and 14% of total substantive comments, respectively). As to the substance of the comments, ENS subjects and FNS subjects are again quite similar. Most refer to choice of lexical items or verb tense/aspect. Thus for item 17 Marie voulait dire a Jean qu'il etait un genie que tout le monde respecte 'Mary wanted to tell John that he was a genius that everyone respects', 3 FNS and 2 ENS faulted the tense of the last verb, preferring the past im- perfective respectait. For several Middle Voice items, ENS and FNS subjects alike were displeased with the choice of de + tonic pronoun -meme to express 'by itself/themselves'. The identification of sentence features unrelated to the linguistic variables no doubt contributed to response variability, and was an artifact of my having taken items verbatim from the linguistics literature, where features such as lexical choice are often overlooked (if not considered in fact 'irrelevant' to the grammatical feature under scrutiny).

COMPREHENSIBLE / INCOMPREHENSIBLE. The most prevalent comments under the Incomprehensible category are ne veut rien dire 'meaningless' and difficile a comprendre 'hard to understand'. The frequency of COMP- comments is approximately equal for ENS and FNS (10% vs. 12% of total). However, FNS subjects appear more sensitive to lucidity of expression, i.e. COMP+-, than ENS (9% vs. 1% of total substantive comments, respectively). Taking the numbers for COMP- and COMP+ together, FNS made almost twice as many such comments as ENS (21% vs. 11%).

As suggested earlier, these two categories, like the Stylistics category, may have absorbed many of the comments destined for Right Site, had the comments made more explicit reference to the grammatical features in question. It is hard to imagine, for example, that FNS 17 did not have the prenominal past participle in mind for Item 65 (*C'etait un etonne candidat 'He was a surprised candidate') when he observed, (7a ne veut rien dire 'It doesn't mean anything'. Again, it is impossible to analyze with any sense of certainty the FNS-ENS asymmetry for this category of think-aloud comments.

SEMANTICS. In a sense Semantics could be considered a subcategory of either Irrelevant or Comprehensible/Incomprehensible. A special classification was made for these comments, however, because they spoke specifically to the likelihood of misinterpretation of an item. Thus, for item 17, ENS 6 correctly pointed out that the referent of il was unclear, and recommended that the sentence be reconstructed to avoid this ambiguity. FNS 15 pointed out that d'ecrites 'written', which appeared in several de + Modifier items, was ho- monymous with decrites 'described' and could possibly cause misinterpretation in oral contexts.

Data under Semantics are infrequent for both groups of subjects (2% of substantive comments). Casual inspection of what comments there are reveals nothing that could be said to distinguish ENS from FNS.

STYLISTICS. Included in this category are mentions of oral vs. written contexts, formal vs. informal register, considerations for ease of articulation, as- sertions that an item was cornplique 'complicated', lourd 'heavy', gauche/

728




maladroit 'awkward', emberlificote 'jumbled', etc. I was tempted to assign Quelle horreur! 'What a horror!' to Stylistics, but there was no way of knowing what this visceral comment was in reference to. It was therefore not included in any category of substantive comments.

As mentioned in the discussion of Right Site, there is a problem of indeter- minacy with respect to Stylistic comments. Since a remark such as lourd could be an allusion to the presence of an extra qui in a that-Trace item, one cannot assume that the numbers under STYL (17% of total for FNS, 12% of total for ENS) necessarily mean that FNS are less sensitive to grammar and more sensitive to stylistics than ENS. As for features of style noted by ENS and FNS (discourse level or register, awkwardness of expression, etc.), there is no apparent difference between the two groups.

CONTEXT. This category is reserved for comments to the effect that acceptance was dependent on a context the subject was able to concoct; this is not a question of register, oral context, etc., which are under Stylistics. Explicit comments about 'rare contexts' or 'most contexts' are also included in this category. While FNS have more comments under this category than ENS, the comments are qualitatively comparable.

3.5. REVIEW OF THINK-ALOUD DATA (BY LINGUISTIC VARIABLE). Table 6 is a breakdown by linguistic variable of the summary data that appears in Table 5.

EN-AVANT (9 items)

RS (WANS) IRREL

48 (2 rej. of 4 OK items)

RS (WANS) IRREL

28 (3 ace. of 7 *items)

COMP- COMP +

9 1

COMP- COMP +

ENS SEM STYL CONTEXT SUBST

COMM 3 4 7 76

FNS SEM STYL CONTEXT SUBST

COMM

6 4 0 9 8 62

# SUBJS COMM

107/162

# SUBJS COMM

91/126

CE/IL (ELLE) (10 items)

RS (WANS) IRREL

45 (5 rej. 20 OK; 4 ace.*)

RS (WANS) IRREL

22 (1 rej. 18 OK; 5 ace.*)

COMP- COMP + ENS SEM

3 0 3

STYL CONTEXT SUBST COMM

13 5 89

FNS COMP- COMP + SEM STYL CONTEXT SUBST

COMM 3 6 1 14 8 72

# SUBJS COMM

120/180

# SUBJS COMM

981140

TABLE 6. Quantitative summary of think-aloud data, by linguistic variable. Refer to key to abbreviations in Table 5.

729




ADJACENCY (16 items) ENS

RS (WANS) IRREL COMP- COMP + SEM STYL CONTEXT SUBST # SUBJS COMM COMM

73 (18 rej. 13 1 3 1 16 11 118 192/288 OK)


COMM COMM

47 (3rej. 3 1 15 0 7 11 84 116/224 OK)

MIDDLE VOICE (9 items) ENS


23 (1 rej. 26 16 2 1 1 11 80 112/162 OK; 3 ace.)


COMM COMM

8 0 22 19 6 2 4 9 70 89/126

THAT-TRACE (10 items) ENS


24 (5 rej. OK; 4 16 1 0 20 7 72 116/180 4 acc.*; 1 change * to another *)


COMM COMM 18 (7 rej. OK; 7 16 7 0 27 11 86 106/140

4 acc.*; 1 change * to another *)

DE + MODIFIER (16 items) ENS


49 (12 rej. 18 19 1 2 15 25 129 182/288 OK)


COMM COMM

21 0 6 18 9 8 10 33 125 157/224

TABLE 6. (Continued)

730




PRENOMINAL PAST PARTICIPLE (6 ITEMS)

ENS RS (WANS) IRREL COMP- COMP + SEM STYL CONTEXT SUBST # SUBJS

COMM COMM

43 (3 acc.*) 3 0 0 0 5 7 58 74/108 FNS


34 (3 rej. 5 4 2 0 8 6 59 63/84 OK; 1 change ? to *)

TABLE 6. (Continued)

To review in complete detail the data for each variable would exceed the scope of this paper; selected observations must suffice. I have attempted to distill out comments which speak to ENS-FNS differences and similarities that might have been obscured in the pencil-and-paper task. In particular, for all items with significant ENS-FNS quantitative differences, I report key comments. More generally, I supply comments that illuminate the nature and bases of subjects' judgments. To save space, I do not write out items but merely refer to their numbers; the stimuli may be found in Appendix 2.

EN-AVANT. For many of these items, e.g. Item 57, FNS and ENS alike view the en as a superfluous element. With others, e.g. Item 73, subjects in both groups aver that pronominalization with en is impossible.

On the item where there was a significant difference in ENS and FNS ratings (Item 28), 4 of the 8 substantive remarks by ENS were clear statements of the superfluousness of en. In contrast, only 1 of 4 FNS substantive comments raised this objection. One subject praised the clarity of the sentence. Two dealt with style: curiously, one of them condemned the lack of elegance, while the other referred to the style as trop soutenue 'too elevated'.

As for the ENS-FNS asymmetry in distribution of Wrong Answers, the small numbers involved (2 rejections of OK among ENS, 3 acceptances of * for FNS) do not permit substantive discussion.

CE/IL (ELLE). Though ENS-FNS divergences were not found in analyses of the numerical data for any of the celil (elle) items, certain isolated differences emerged in the think-alouds of some ENS subjects. Most revealing are the comments for Item 8, which was included in this study and in Coppieters 1987. Three ENS subjects correctly accepted the sentence, but voiced their concerns about the use of ce instead o

Date post:	01-Feb-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Linguistic Society of America - UT Liberal Arts › frenchitalian › ...been amply documented (e.g....

Documents