The Impact of Pragmatic Aspect in the Writing Assessment across Raters with

Different L1 Backgrounds

Boyoung Kim

1. Introduction

The rise of the concept of communicative competence marked a shift in the view of successful second language learning (L2) from mastery of grammar items to acquisition of usage of socially appropriate forms. Accepting this view, one cannot deny that pragmatic competence holds a significant position in communicative competence. Indeed, an L2 speaker cannot be expected to carry out a successful communicative act if (s)he has not achieved some mastery of grammar, phonology, and vocabulary in the target language. Yet, as many of them have already experienced first-hand, even a perfectly gram-matical, phonologically correct speech sometimes fail, either because their pragmatic competence is undeveloped or is not properly met in the target culture.

What, then, is pragmatics? Kasper (1997) defines pragmatics as the study of how a speaker uses language in social interaction and its effect on other participants in the communicative event. Crystal (1985) defines it as “the study of language from the point of view of users, especially of the choices they make, the constraints they encounter in using language in social in-teraction and the effects their use of language has on other participants in the act of communication” (p. 240). Interlanguage pragmatics, then, is the study of non-native speakers’ use and acquisition of L2 pragmatic knowledge (Kasper, 1996).

The following scenario illustrates the importance of interlanguage prag-

70 Boyoung Kim

matic competence: Two English learners approach a native speaker of Eng-lish, with whom they are unacquainted. While one student says, Tell me what time it is, the other one asks, Excuse me, do you mind telling me what time it is? Both requests are quite straightforward and easy to understand, and will likely bring about the desired response from their interlocutor. However, the native speakers would respond more favorably to the second learner over the first one, simply because it was more appropriate in that specific context. As we can see, pragmatic incompetence of L2 learners leads them to utter socially inappropriate or unacceptable expressions, which often result in mis-understanding and misinterpretation. This, in turn, may leave the impres-sion of being rude or ignorant to the native speaker interlocutor. If pragmatic competence is vital, not only to successful communication, but also to suc-cessful integration to the target culture and community, then it is also vital that English teachers help their learners develop this important feature of language competence.

The recent years saw the blooming interest in interlanguage pragmat-ics and its teachability. Majority of the previous work have focused on the positive effect of instruction on the acquisition of pragmatic competence (Bardovi-Harlig, & Vellenga, 2012; Bouton, 1994; Félix-Brasdefer, & Cohen, 2012; Rose, 2005; Taguchi, 2011; Wildner-Bassett, 1994). Some of the works have focused on the assessment of interlanguage pragmatic competence (Brown & Ahn, 2011; Ishihara, 2009; Roever, 2006; Roever, 2011). Yet, very few interlanguage pragmatics studies have been conducted with an attempt to link interlanguage pragmatics to its effect on the interlocutor (Al-Ghatani & Roever, 2010; van Lier & Matsuo, 2000). Thus, it is the motivation of this paper to examine the impact of pragmatic competence reflected in writings to raters with varying L1 backgrounds, particularly, in the communicative act of complaining and requesting. This paper is organized as follows: First, the previous work on this subject is covered. Second, the method of the experi-ment is explained. Third, the results of the experiment are reported and dis-cussed. Finally, concluding remarks, along with limitations of the study, are considered.

The Impact of Pragmatic Aspect in the Writing Assessment ... 71

2. Previous Studies

2.1 Interlanguage PragmaticsThe question, Can pragmatics be taught?, has intrigued numerous research-

ers into exploring the effect of instruction on the acquisition of pragmatic knowledge. Bouton (1994) and Billmyer (1990) found that the ESL learners showed improvement as a result of instruction in pragmatics. In fact, some previous studies have demonstrated the superiority of a target language en-vironment over a foreign language environment for pragmatic development (Bardovi-Harlig & Dörnyei, 1998; Barron, 2002; Hoffman-Hicks, 1992; Li, 2000; Matsumura, 2001, 2003; Olshtain & Blum-Kulka, 1985; Schauer, 2006; Taguchi, 2008; Takahashi & Beebe, 1987). For example, following Bardovi-Harlig and Dörnyei’s (1998) study, Schauer (2006) conducted a research on L2 learners’ detection of pragmatic and grammatical errors and perceived severity of these errors. German learners in the ESL and EFL environments watched a set of video interactions where the interlocutors’ speech acts — apologies, refusals, requests, and suggestions — contained either gram-matical or pragmatic errors. Results corresponded to Bardovi-Harlig and Dörnyei’s findings: Compared with the EFL learners, the ESL learners rec-ognized a considerably higher number of pragmatic errors than grammatical ones. Their pragmatic awareness continued to improve during their 9-month stay in the target language country, almost to the native speaker level. Post-hoc interview data revealed that the opportunities to observe everyday life interactions in the ESL environment helped the ESL learners notice differ-ences between their own output and that of native speakers and to modify their language according to native speaker norms.

Although such findings appear to argue that pragmatic competence can only be achieved through the exposure to the target language community and that the imbalance of grammatical and pragmatic competences in the EFL setting is inevitable, other researchers came up with rather encouraging findings which suggest that it is possible to improve interlanguage pragmat-ics through instruction, even in EFL classrooms (Bardovi-Harlig & Hartford,

72 Boyoung Kim

1993; Kinginger & Belz, 2005; Kinginger & Blatter, 2008; Kinginger & Far-rell, 2004; Marriot, 1995; Niezgoda & Röver, 2001; Röver, 2005; Soler, 2005). Among such works, Soler (2005) found that the EFL learners have shown improvement in their awareness of requests, in response to both direct and indirect instruction. With the observations made from the study, the author urges the EFL teachers to foster pragmatic competence in the students, es-pecially by making use of explicit instruction, with measures such as direct awareness-raising tasks and written meta-pragmatic feedback on the use of appropriate requests. Such research findings lend support to the claim that it is not only possible, but also necessary, to teach pragmatics in the EFL classrooms.

2.2 L2 Writing Assessment and PragmaticsA central goal of academic purposes in ESL and EFL writing programs

is to help students develop writing and thinking strategies needed to write academic texts of different types. Aside from academic texts, one type of writing which seems to pose a great challenge to L2 students is the writing task which also requires pragmatic knowledge, namely, letter-writing task. Personal letter-writing, whether it is formal or informal, can be seen as a special type of writing, demanding a different type of processing than writing academic texts. Writing personal letters in a foreign language is associated with fulfilling a series of required procedures to meet the criteria of social appropriateness and politeness. Brown and Levinson (1987) identified three crucial factors involved in the realization of such social interactions: relative Power of the interlocutors, degree of Social Distance (i.e. acquaintanceship and membership in a similar social group), and degree of Imposition (i.e. the cost to the hearer of carrying out the interlocutor’s request or the damage caused by an action requiring an apology). Thus, L2 learners should deter-mine what type of communicative strategies to use, considering who their interlocutor or addressee is, based on the three factors above.

The biggest difficulty of writing letters in a foreign language appears to arise out of the absence of “shared knowledge”. Kachru (1997), stressing the importance of “shared knowledge”, reported that “people who share a

The Impact of Pragmatic Aspect in the Writing Assessment ... 73

common language and culture have an easier time ‘making sense’ of each other’s utterances and actions” (p. 6). Kaplan (1966) was the first to consider the cross-cultural rhetoric patterns to be the key to analyzing errors in L2 learning. In his study, Kaplan argued that patterns of rhetoric are culturally relative and that paradigms of logic are not universal. The thought patterns of speakers of English, according to Kaplan, are predominantly linear in se-quence, which contrasts with ways of thinking in other cultures. He claimed that non-native writers “employ a rhetoric and a sequence of thought that violate the expectations of the native speakers” (p. 4). His seminal work es-tablished a foundation upon which the subsequent research regarding cross-cultural rhetoric patterns reflected in writing were built (Atkinson, 2003; Kachru, 1994, 1997; Leki, 1991; Matsuda & Silva, 2014; Mauranen, 1993; Ortega, 2012; Ramanathan & Kaplan, 2000; Uysal, 2008). For example, Ka-chru (1997) assumed that different speech communities have different ways of organizing ideas in writing, which reflect their cultural thought patterns, and claimed that non-native speakers of English make use of rhetoric pat-terns which are different from those of native speakers. Mauranen (1993) also described writing as “a cultural object”, existing only in the social world of humans as a product of social activities, which is also very much shaped by the educational system in a writer’s native culture.

Despite the large number of studies carried out regarding the rhetoric pat-terns across various cultures, few have focused on the pragmatics displayed in L2 learners’ writings (Crane, 2008; Hyland & Milton; 1997), and none has yet linked the acquisitional pragmatics reflected in their writings to its ef-fect on their interlocutor. The present study, therefore, sets out to contribute to the existing body of interlanguage pragmatics studies by addressing the underrepresented area in the field, that is, the influence of pragmatic com-petence on the interlocutors of different L1 backgrounds. Thus, responses of the raters on the pragmatic aspect of writings of the participants were of particular interest and will be discussed in detail.

74 Boyoung Kim

3. Experiment

3.1 Research QuestionsThis study seeks to give answers to the following questions:

(1) Is the assessment of the native speaker rater affected by pragmatic as-pect of the learners’ writings?

(2) Is the assessment of the non-native speaker rater affected by pragmatic aspect of the learners’ writings?

(3) What causes the divergence in the ratings between the two groups of raters?

3.2 ParticipantsThe participants in this study consist of three groups: Korean EFL learn-

ers, English native speaker raters, and non-native speaker raters.1) Korean EFL learners: 19 Korean college students in Daegu, Korea, who

have had no prior experience of living in English-speaking countries were collected as participants. They were recruited from three classes which are of the same level – intermediate. Their standardized English test scores ob-tained from TOEIC ranged from 640 to 750. For the sake of their anonymity, their names were replaced with numerical codes from 1 to 17.

2) English native speaker raters: Three native speakers of English who are working as English teachers in Seoul and Daegu, Korea, participated as na-tive speaker raters. All of them are from the US, and their teaching experi-ence ranges from two to six years. Their names were replaced with randomly assigned alphabetical letters from A to C.

3) Non-native speaker raters: Three Korean teachers of English who are working in middle and high school in Daegu, Korea, participated in rating. Their teaching experience ranges from two to five years. Their names were replaced with randomly assigned alphabetical letters from D to F.

The Impact of Pragmatic Aspect in the Writing Assessment ... 75

3.3 Setting and ProcedureThe student participants were given the assignment of writing an e-mail

to a professor (See Appendix Ⅰ). The context was set based on Brown and Levinson (1987); the context of complaining to and asking of a professor who exhibits a higher social power to carry out a request with a high degree of imposition calls for a skillful use of pragmatic competence. The prompt de-scribed the situation on which the students had to come up with a specific scenario to write the e-mail with, and they were not required to use their own name and story.

Upon submission, the researcher gave the writings of the students to the two groups of raters, the native speaker rater group and non-native speaker rater group. The raters, regardless of their group, were given the same in-struction by the researcher: Give holistic assessment which ranges from 0 to 10 and add comments as to what led them to give that specific score, tak-ing into account six factors – organization, content, grammar, vocabulary, mechanics and politeness, the last factor approximating to the pragmatic feature which is to be examined in the present research. However, the raters were not trained in advance nor given specific rubrics to base their decision on, because the purpose of the study was to find out the very aspect each rater group focuses on in writing assessment without giving them explicit instructions about it. Following the retrieval of the graded writing samples, the researcher compared the scores and comments against one another; the quantitative and qualitative reports on the results will follow in the next ses-sion.

4. Results and Discussion

Prior to providing answers to the research questions addressed above, quantitative analyses regarding the raters were obtained through SPSS. The mean score of each rater and the standard deviation (SD) of the score distri-bution were computed and reported in Table 1. The correlation coefficients and the resulting inter-reliability of the two groups were also calculated with SPSS. Pearson correlation coefficients between the members of the native

76 Boyoung Kim

speaker rater group are described in Table 2. Using these correlations, the inter-rater reliability using Spearman-Brown formula was 0.84. The Pearson correlation coefficients for the non-native speaker rater group are described in Table 3. Using the correlations, the inter-rater reliability among the non-native speaker group was calculated as 0.86, according to Spearman-Brown formula. The computed figures of both groups reveal that the agreement be-tween the raters in each rater group is sufficiently high, thus reliable.

Concerning our first research question of whether the assessment of the native speaker rater group is affected by pragmatic aspect of the learners’ writings, a total of 45 out of 51 ratings from the native speaker rater group (Rater A: 16, Rater B: 15, Rater C: 14) included comments on appropriate-ness, or politeness. Some of the writings were rated as polite and respectful: ‘Kind and polite approach’, ‘Apologetic tone’ (Rater A); ‘’Addressed in proper appropriateness’ (Rater B); ‘The overall attitude the student was pleasant and professional’, ‘Very polite and well thought-out e-mail’ (Rater C). On the

Table 1. Descriptive Statistics for the Raters

Raters Mean SD

Native speaker rater group

Rater A 7.29 0.98

Rater B 7 1.69

Rater C 7.5 1.07

Non-native speaker rater group

Rater D 6.64 1.49

Rater E 6.47 1.28

Rater F 6.52 1.84

Table 2. Pearson Correlations for Native Speaker Rater Group

Ra Rb Rc

Ra 0.524 0.678

Rb 0.720

Table 3. Pearson Correlations for Non-Native Speaker Rater Group

Rd Re Rf

Rd 0.645 0.820

Re 0.570

The Impact of Pragmatic Aspect in the Writing Assessment ... 77

other hand, some of the writings were considered rather impolite: ‘Addresses the professor as sir which is nice, but still seems aggressive and challeng-ing’, ‘A bit accusatory, calling the professor unfair’ (Rater A); ‘Too direct and lack the respect to the professor’, ‘Complete lack of polite expressions for the recipient’, ‘Too demanding and too direct’ (Rater B). Comments on a variety of aspects in pragmatic skills show that it is a general tendency for native speaker raters to consider pragmatic features in the writings assessment.

The second research question considered whether non-native English teacher raters consider pragmatic aspects of writing important. The raters gave 11 out of 51 ratings comments about politeness (Rater D: 5, Rater E: 2, Rater F: 4). While some of the comments were given on the skillful use of pragmatic competence: ‘Very politely written’ (Rater E); ‘Briefly but politely written with strong argument’ (Rater F)’; some of the comments were given on the lack of such pragmatic skills: ‘Even though it has to be a little bit for-mal when writing an e-mail to a professor, it seems it is an e-mail to a close friends’, ‘The writer sounds hostile, which is inappropriate for an e-mail to a professor’ (Rater D); ‘The third argument is too subjective and too demand-ing, considering that the recipient is his professor’ (Rater E). The relatively low number of comments given on pragmatics, compared to the native speak-er rater group, implies that the non-native speaker raters are less sensitive to appropriateness in English writings than the native speaker raters are.

9

direct’ (Rater B). Comments on a variety of aspects in pragmatic skills show that

it is a general tendency for native speaker raters to consider pragmatic features in

the writings assessment.

The second research question considered whether non-native English teacher

raters consider pragmatic aspects of writing important. The raters gave 11 out of

51 ratings comments about politeness (Rater D: 5, Rater E: 2, Rater F: 4). While

some of the comments were given on the skillful use of pragmatic competence:

‘Very politely written’ (Rater E); ‘Briefly but politely written with strong

argument’ (Rater F)’; some of the comments were given on the lack of such

pragmatic skills: ‘Even though it has to be a little bit formal when writing an e-

mail to a professor, it seems it is an e-mail to a close friends’, ‘The writer sounds

hostile, which is inappropriate for an e-mail to a professor’ (Rater D); ‘The third

argument is too subjective and too demanding, considering that the recipient is his

professor’ (Rater E). The relatively low number of comments given on pragmatics,

compared to the native speaker rater group, implies that the non-native speaker

raters are less sensitive to appropriateness in English writings than the native

speaker raters are.

The third research question asked: What causes the divergence in the ratings

between the two groups of raters? To find out, the means of the rater groups were

computed and compared visually, as shown in Figure 1.

FIGURE 1

Comparison of Mean Scores between the Two Rater Groups

6.3

7.3

6.3

7.7

5.3

7.3 6.7

9.2

7.2

8.7

6.7 7.0

6.3

8.7

6.2

7.7

9.0

5.3

7.7 7.0

7.3

6.3

8.0 7.3

9.0

7.7 8.3

7.0 6.7

4.7

8.0

6.7 6.3

9.3

4.0

5.0

6.0

7.0

8.0

9.0

10.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Score

Student Number

NS Rater Group

NNS Rater Group

Figure 1. Comparison of Mean Scores between the Two Rater Groups

78 Boyoung Kim

The third research question asked: What causes the divergence in the rat-ings between the two groups of raters? To find out, the means of the rater groups were computed and compared visually, as shown in Figure 1.

The two lines similarly pattern, with a few exceptions which diverge by more than one point: Students 1, 5, 13 and 16. For Students 1, 13 and 16, the scores given by the native speaker rater group were higher, and for Stu-dent 5, the score given by the non-native speaker rater group was higher. Regarding Student 1’s writing, native speaker raters commented: ‘Decent appropriateness, grammatical errors are numerous’ (Rater A); ‘Good use of vocabulary but lacks ability in sentence structure. Many misplaced punc-tuation and lack of capitalization. Still, the attempt to approach politely is noteworthy’ (Rater B); ‘I could tell the intention of the writer was polite, but the improper choices of grammar, syntax, and lack of punctuation really took away from this response’ (Rater C). Non-native speaker raters commented on the same piece of writing: ‘Some grammar mistakes, a few awkward ex-pressions, punctuation mistakes, no indentation, didn’t start a sentence with upper cases’ (Rater D), ‘Numerous errors occur regarding tense usage, verb agreement. Too weak content’ (Rater E); ‘The writing is organized but there are not much sufficient supporting details for the arguments. Also this stu-dent needs to care for the capital letters and punctuation marks’ (Rater F)’. Regarding Student 13’s writing, the native speaker rater group commented: ‘Appropriate language and tone. Grammar errors but well organized’ (Rater A); ‘Sentence fragments. Incorrect use of vocabulary and prepositions’ (Rater B); ‘Most of the marking off for the score is for the grammar. While I don’t feel the logic of the request is solid, the earnest politeness of the request makes it land a little bit softer’ (Rater C). The same writing was commented by the non-native speaker group as well: ‘Same grammar mistakes over and over again. The writer needs to learn basic grammar from the start. I can’t figure out what the writer is trying to point out’ (Rater D); ‘The student does not seem to be aware of English sentence patterns. Too many grammar mis-takes’ (Rater E); ‘First, too many serious grammar errors, the student does not know how to use modal verbs. It’s difficult to read and understand the writing. The reasons are not reasonable or logical either’ (Rater F). Regard-

The Impact of Pragmatic Aspect in the Writing Assessment ... 79

ing Student 16’s writing, the native speaker rater group commented: ‘Apolo-getic tone. Minor grammar errors’ (Rater A); ‘Good use of vocabulary and proper verb conjugation. Missing articles in some sentences, and mistakes in prepositions. Proper amount of respect shown for the addressee’ (Rater B); ‘I mostly marked off for the grammar. I feel this request is very polite and fairly well presented’ (Rater C). The non-native speaker raters commented on his writing: ‘He uses inappropriate expressions like gonna in an e-mail to professors. He needs to be more careful in choosing expressions, when writ-ing a letter to elderly person. Though he used the technique, appeal to emo-tion, he conveys his points efficiently’ (Rater D); ‘The content is too simple, repeated use of the expression give me one more chance, and lack of logical reason’ (Rater E); ‘Not many grammar errors but need to care for punctua-tion. And the point is understandable but not logical that much’ (Rater F). The comments from both groups reveal that the scores were deducted mainly due to the errors on mechanics and grammar. With a closer examination of the comments, the difference between the two rater groups is observed when most of the comments from the native speaker raters include positive com-ments about their attitude toward the recipient, while most of the comments from the non-native speaker counterpart do not. Combined with the com-ments given, the score gap between the two rater groups can be attributed to the native speaker group’s partial compensation of the deducted scores with the pragmatic features shown in the writings.

On the other hand, Student 5’s writing was commented by the native speaker raters: ‘Inappropriate rambling about the weather at the beginning. Few grammar errors’ (Rater A); ‘Good use of vocabulary. However, there were sentence fragments, mistakes in verb conjugations and use of preposi-tions. Lacks expressions showing respect for recipient’ (Rater B); ‘The student sounds very offensive. I think there is no logical reason for the student to demand a reassessment of the assignment. We all have bad days, but our ef-forts are still our own’ (Rater C). Non-native speaker raters commented: ‘The first paragraph seems inappropriate for an e-mail to a professor. Why doesn’t he write it in his own diary? But it shows both coherence and cohesiveness in the writing’ (Rater D); ‘The organization is well done and the arguments are

80 Boyoung Kim

sound. The content is a bit short, but very few grammatical errors and easy to understand’ (Rater E); ‘Overall writing is good. Not many grammar errors. Reasons are understandable and the details and supporting ideas are rea-sonable and logical’ (Rater F). Comments regarding the grammar and use of words were positive in both groups, but all of the native speaker raters and one non-native speaker rater commented on the lack of respect shown for the recipient. Therefore, the lower average score given by the native speaker group can be addressed to the poor pragmatic skills, despite the advanced use of grammar and vocabulary. Meanwhile, it is worth noting that a non-native speaker rater, Rater D pointed out on the same inappropriate attitude of the writer on which Rater A also gave comment. Thus, we can assume that some pragmatic aspects do not go unnoticed by non-native speaker teachers.

5. Conclusion

The present study has examined the effects of pragmatic aspect shown in the writings of Korean college students of English on the writing assessment, conducted by raters with different L1 backgrounds. The experiment con-ducted demonstrated how the native speakers behaved differently from the non-native speakers in rating the writings which involve the communicative act of complaining and requesting. The ratings received from different raters reflected the particular area on which each rater places more importance. Although different opinions were gathered from raters on the same piece of writing, the tendency of native speaker raters to emphasize pragmatic as-pects in assessment was observed, the tendency of giving higher score than the non-native counterpart when the writer showed proper respect and lower score when the writer failed to do so. The results are expected to provide gen-eral guidelines for students with different academic goals and the pedagogi-cal implications for those involved in English education: While students who want to receive higher grades in the EFL setting should work on the particu-lar areas the EFL English teacher focused on, students who aim to study in English-speaking countries in the future should put forth effort to improve on the area the native speaker rater focused on, namely, pragmatic aspect,

The Impact of Pragmatic Aspect in the Writing Assessment ... 81

not only to receive higher English score, but also to successfully integrate to the target community without trial and error. The implications suggest the EFL classrooms strive to provide opportunities to practice pragmatic skills to promote pragmatic competence. To that end, implementing instruction along with tasks, such as role-play and simulation, into classrooms can prove to be much more effective than simple exposure in developing pragmatic compe-tence (Martínez-Flor & Soler, 2007; Takahashi, 2001; Tateyama, 2001).

At this point, some limitations of the study merit discussion. First and foremost, the findings from this study cannot be generalized, due to the small number of participants, both students and raters. More insightful findings and pedagogical implications might have been reaped with a larger sample size. Second, what constitutes the abstract concept, ‘pragmatic competence’, was not operationalized to represent concrete figure. Chomsky (1980) elabo-rated on pragmatic competence in the following:

For purposes of enquiry and exposition, we may proceed to distinguish ‘grammatical competence’ from ‘pragmatic competence,’ restricting the first to the knowledge of form and meaning and the second to knowl-edge of conditions and manner of appropriate use, in conformity with various purposes. Thus we may think of language as an instrument that can be put to use. The grammar of the language characterizes the instrument, determining intrinsic physical and semantic properties of every sentence. The grammar thus expresses grammatical compe-tence. A system of rules and principles constituting pragmatic compe-tence determines how the tool can effectively be put to use. (p. 224)

On the other hand, Hymes (1972), although he recognizes ‘pragmatic compe-tence’, he also maintains that “‘the creative aspect of language use’ remains as much a mystery to us as it was to the Cartesians” (p. 138), where by mys-tery he means something ultimately beyond human capacity to understand. As such, pragmatic competence has been such evasive concept to capture for an operationalization due to its abstract nature. A review of L2 pragmatics research in the last 20 years supports this, as it shows that researchers have not sought a common metric by which development can be measured, but rather have utilized multiple measures. Instead of seeking global proficiency

82 Boyoung Kim

measures for pragmatics, acquisition studies have utilized different measures of development that are appropriate to the research questions posed and the research designs used to investigate pragmatic development. Therefore, fur-ther research on the assessment of pragmatic competence of L2 learners can contribute in bringing about the agreement on the common metric for prag-matic competence, to answer Cohen and Olshtain’s (1981) question, “Can a rating scale be developed for assessing sociocultural competence?” (p. 115).

References

Al-Ghatani, S., & Roever, C. (2010). Role-playing L2 requests: Proficiency

and discursive development. Unpublished manuscript, University of Melbourne.

Atkinson, D. (2003). Writing and culture in the post-process era. Journal of Second Language Writing, 12(1), 49-63.

Bardovi-Harlig, K., & Dörnyei, Z. (1998). Do language learners recognize pragmatic violations? Pragmatic versus grammatical awareness in in-structed L2 learning. TESOL Quarterly, 32(2), 233-262.

Bardovi-Harlig, K., & Hartford, B. S. (1993). Learning the rules of aca-demic talk: A longitudinal study of pragmatic development. Studies in Second Language Acquisition, 15, 279-304.

Bardovi-Harlig, K., & Vellenga, H. E. (2012). The effect of instruction on

conventional expressions in L2 pragmatics. System, 40(1), 77-89.Barron, A. (2002). Acquisition in interlanguage pragmatics: Learning how to do

things with words in a study abroad context. Amsterdam: John Benjamins Publishing.

Billmyer, K. (1990). “I really like your lifestyle”: ESL learners learning how

to compliment. Penn Working Papers in Educational Linguistics, 6(2), 31-48.

Bouton, L. F. (1994). Can NNS skill in implicatures in American English

be improved through explicit instruction? – A pliot study. Pragmatics and Language Learning Monograph Series, 5, 89-109.

Brown, J. D., & Ahn, R. C. (2011). Variables that affect the dependability

The Impact of Pragmatic Aspect in the Writing Assessment ... 83

of L2 pragmatics tests. Journal of Pragmatics, 43(1), 198-217.Brown, P., & Levinson, S. D. (1987). Politeness: Some universals in lan-

guage usage. Cambridge: Cambridge University Press.Cohen, A. D., & Olshtain, E. (1981). Developing a measure of sociocultural

competence: The case of apology. Language Learning, 31, 113–134.Chomsky, N. (1980). Rules and Representations. Oxford: Blackwell.Crane, C. P. (2008). Evaluative choice in advanced L2 writing of German: A

genre perspective (Unpublished doctoral dissertation). Georgetown Uni-versity, Washington, DC.

Crystal, D. (1985). A dictionary of linguistics and phonetics (2nd ed.). Oxford: Blackwell.

Félix-Brasdefer, J. C., & Cohen, A. D. (2012). Teaching pragmatics in the foreign language classroom: Grammar as a communicative resource. Hispania, 95(4), 650-669.

Hoffman-Hicks, S. (1992). Linguistic and pragmatic competence: Their rela-tionship in the overall competence of the language learner. In L. Bou-ton & Y. Kachru (Eds.), Pragmatics and language learning (pp. 66–80). Urbana: University of Illinois.

Hyland, K., & Milton, J. (1997). Qualification and certainty in L1 and L2

students’ writing. Journal of Second Language Writing, 6(2), 183-205.Hymes, D. (1972). On communicative competence. J. B. Pride & J. Holmes

(Eds.), Sociolinguistics (pp. 269-293). Harmondsworth: Penguin.Ishihara, N. (2009). Teacher-based Assessment for foreign language prag-

matics. TESOL Quarterly, 43(3), 445-470.Kachru, Y. (1994). Monolingual bias in second language acquisition re-

search. TESOL Quarterly, 28(4), 795-800.Kachru, Y. (1997). Cultural meaning and rhetorical styles: Toward a frame-

work for contrastive rhetoric. In G. Cook (Ed.), Principle and practice in applied linguistics (pp. 171-185). Oxford: Oxford University Press.

Kaplan, R. B. (1966). Cultural thought patterns in intercultural education. Language Learning, 16, 1-20.

Kasper, G. (1996). Introduction: Interlanguage pragmatics in SLA. Studies in Second Language Acquisition, 18, 125-148.

84 Boyoung Kim

Kasper, G. (1997). Can Pragmatic Competence be Taught? [HTML document]. Retrieved from http://www.lll.hawaii.edu/nflrc/NetWorks/NW6/

Kinginger, C., & Belz, J. A. (2005). Sociocultural perspectives on pragmatic development in foreign language learning: Case studies from telecol-laboration and study abroad. Intercultural Pragmatics, 2, 369–421.

Kinginger, C., & Blatter, G. (2008). Development of sociolinguistic aware-ness in study abroad. In L. Ortega & H. Byrnes (Eds.), Longitudinal studies and advanced L2 capacities (pp. 223–246). London: Routledge.

Kinginger, C., & Farrell, K. (2004). Assessing development of metaprag-matic awareness in study abroad. Frontiers: The Interdisciplinary Jour-nal of Study Abroad, 10, 19–42.

Leki, I. (1991). Twenty-five years of contrastive rhetoric: Text analysis and

writing pedagogies. TESOL Quarterly, 25(1), 123-143.Li, D. (2000). The pragmatics of making requests in the L2 workplace: A

case study of language socialization. Canadian Modern Language Re-view, 57, 58–87.

Marriot, H. (1995). The acquisition of politeness patterns by exchange stu-dents in Japan. In B. F. Freed (Ed.), Second language acquisition in a study abroad context (pp. 197–224). Amsterdam: John Benjamins Pub-lishing.

Martínez-Flor, A., & Soler, E. A. (2007). Developing pragmatic awareness of suggestions in the EFL classroom: A focus on instructional effects. Canadian Journal of Applied Linguistics, 10(1), 47-76.

Matsuda, P. K., & Silva, T. (2014). Second language writing research: Per-spectives on the process of knowledge construction. NY: Routledge.

Matsumura, S. (2001). Learning the rules for offering advice: A quantita-tive approach to second language socialization. Language Learning, 51, 635–679.

Matsumura, S. (2003). Modelling the relationship among interlanguage pragmatic development, L2 proficiency, and exposure to L2. Applied Linguistics, 24, 465–491.

Mauranen, A. (1993). Contrastive ESP rhetoric: Metatext in Finnish-Eng-lish economics texts. English for Specific Purposes, 12, 3-22.

The Impact of Pragmatic Aspect in the Writing Assessment ... 85

Niezgoda, K., & Röver, C. (2001). Pragmatic and grammatical awareness: A function of learning environment. In K. Rose & G. Kasper (Eds.), Pragmatics in language teaching (pp. 63–79). New York: Cambridge Uni-versity Press+

Olshtain, E., & Blum-Kulka, S. (1985). Degree of approximation: Nonnative reactions to native speech act behavior. In S. M. Gass & C. G. Madden

(Eds.), Input in second language acquisition (pp. 303–325). Rowley, MA: Newbury House.

Ortega, L. (2012). Epilogue: Exploring L2 writing-SLA interfaces. Journal of Second Language Writing, 21(4), 404-415.

Ramanathan, V., & Kaplan, R. B. (2000). Genres, authors, discourse com-munities: Theory and application for (L1 and) L2 writing instructors. Journal of Second Language Writing, 9(2), 171-191.

Roever, C. (2006). Validation of a web-based test of ESL pragmalinguistics. Language Testing, 23(2), 229-256.

Roever, C. (2011). Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463-481.

Rose, K. R. (2005). On the effects of instruction in second language prag-matics. System, 33(3), 385-399.

Röver, C. (2005). Testing EFL pragmatics. Tübingen: Gunter Narr Verlag.Schauer, G. A. (2006). Pragmatic awareness in ESL and EFL contexts:

Contrast and development. Language Learning, 56, 269-318.Soler, E. A. (2005). Does instruction work for learning pragmatics in the

EFL context? System, 33(3), 417-435.Taguchi, N. (2008). The role of learning environment in the development of

pragmatic comprehension. Studies in Second Language Acquisition, 30(4), 423-452.

Taguchi, N. (2011). Teaching pragmatics: Trends and issues. Annual Review of Applied Linguistics, 31, 289-310.

Takahashi, S. (2001). The role of input enhancement in developing prag-matic competence. In K. R. Rose & G. Kasper (Eds.), Pragmatics in lan-guage teaching (pp. 171-199). Cambridge: Cambridge University Press.

Takahashi, T., & Beebe, L. (1987). The development of pragmatic compe-

86 Boyoung Kim

tence by Japanese learners of English. JALT Journal, 8, 131–158.Tateyama, Y. (2001). Explicit and implicit teaching of pragmatics routines:

Japanese sumimasen. In K. R. Rose & G. Kasper (Eds.), Pragmatics in language teaching (pp. 200-222). Cambridge: Cambridge University

Press.Uysal, H. H. (2008). Tracing the culture behind writing: Rhetorical pat-

terns and bidirectional transfer in L1 and L2 essays of Turkish writers in relation to educational context. Journal of Second Language Writing, 17(3), 183-207.

Wildner-Bassett, M. A. (1994). Intercultural pragmatics and proficiency: ‘Polite’ noises for cultural appropriateness. International Review of Ap-plied Linguistics, 32(1), 5-17.

van Lier, L., & Matsuo, N. (2000). Varieties of conversational experience: Looking for learning opportunities. Applied Learning, 11, 265-287.

The Impact of Pragmatic Aspect in the Writing Assessment ... 87

APPENDIX I Construct a specific scenario based on the following prompt and write

three-paragraph-long, at least 20-sentence-email. (You do not have to use your own name and your own story. You can come up with a probable yet imaginary situation when writing the email.)

☞ You are writing an email to your professor to complain about the grade you received on your assignment and request for a re-mark.

88 Boyoung Kim

ABSTRACT

The Impact of Pragmatic Aspect in the Writing Assessment across Raters with

Different L1 Backgrounds

Boyoung Kim

Drawing on the finding that the EFL teachers and learners tend to be less sensitive to pragmatic errors than their ESL counterpart (Bardovi-Harlig & Dörnyei, 1998; Schauer, 2006), and that rhetoric patterns in writing differ across cultures (Kachru, 1997; Kaplan, 1966; Mauranen, 1993), this study investigates the underdeveloped area of the effect of pragmatic aspect of L2 learners’ writings on the raters with different L1 backgrounds, namely, Kore-an and English. Student participants in the experiment were provided with a prompt which required them to write a complaining and requesting e-mail to a person with a higher social power (Brown & Levinson, 1987). Compared to the non-native speaker rater group, the native speaker rater group were observed to be more attentive to pragmatic features of the writing samples. The results suggest that improvement in pragmatic skills is crucial for suc-cess in both academic achievement and adjustment to the target language community.

Key Words pragmatic competence, interlanguage pragmatics, L2 writing, writing assessment