Date post: | 03-Apr-2018 |
Category: |
Documents |
Upload: | ipaflashcards |
View: | 218 times |
Download: | 0 times |
of 17
7/29/2019 Diagnostic Test for Social English Programs
1/17
Diagnostic Test for Social English Programs
Ramon Mislang
December 12, 2012
7/29/2019 Diagnostic Test for Social English Programs
2/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 1
Audience and Intended Purpose
Trends inusing social media platforms such as Facebook, You Tube, and Gmail, have
changed the way people communicate. As a result, language learners must now deal with a genre
of English that demands the integration of listening and writing skills as well as knowledge of
idiomatic language to communicate effectively in Social environments. I have designed a
Diagnostic Test (Appendix A) for Japanese learners of English who are preparing for study in
America. The purpose of the test is to determine the degree to which the specific instructional
objectives of Social English programs have already been met. The test results will inform
teachers which areas of English students still need support.
Definition of the Constructs
The first construct the test seeks to measure is vocabulary, which is the total number of
lexical items a learner knows. Davies (1999) states that the function of a vocabulary test is to test
a learners breadth (number of words) of vocabulary as well as the depth (how well words are
known). My test is concerned with idioms, which according to Oxford Dictionary (2011), are
groups of words established by usage as having a meaning not deducible from those of the
individual words. My test will help teachers observe learners depth of vocabulary. A description
of how idioms will be tested will be explained in the prompt and response attributes section.
The test also seeks to measure listening comprehension. Listening is a process that is
difficult to observe directly (Davies, 1999) because meaning is not simply extracted from input
but involves test-takers knowledge of the language system, their prior knowledge, and the
context of the interaction (Vandergrift & Goh, 2009). However, there are tasks that test-
developers can create to elicit behaviors that are desired from language learners engaged in the
process of listening.
7/29/2019 Diagnostic Test for Social English Programs
3/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 2
Buck (2001) describes three types of observable behaviors, which include; (1) the ability
to process extended samples of realistic spoken language automatically and in real time, (2) to
understand the linguistic information that is unequivocally included in the text, and (3) to make
whatever inferences are unambiguously implicated by the content of the passage (Buck, 2001).
Listening ability will be evaluated on the basis of recalling content heard in the audio clip and
applying pragmatic knowledge to display understanding of the content recalled. Pragmatic
knowledge refers to the understanding of illocutionary force of an utterance and interpreting the
intended meaning in terms of that (p.104).
The final construct my diagnostic test seeks to measure is summary writing. According to
Brown and Day (1983), summary writing is the process of deleting trivia, avoiding redundancies,
substituting a super-ordinate action for a list of actions, selection of a topic sentence, and
inventing a topic sentence.
Rationale and Framework for Design
Following principles of communicative testing (Buck, 2001), this test is less concerned
with how much learners know about the target language and more concerned with how
effectively they can communicate in the target language. Buck (2001, p.253) states that in many
communicative situations, verbal information is accompanied by visual information and playing
a disembodied recording from an audio-player does not create a very realistic listening
situation. Therefore, audio and video stimuli are supported by contextual information such as
the name of the audio and video clip and character profile pictures.
One ofSwains (1984) principles of communicatve language development, bias for best,
refers to encouraging and eliciting the test takers best performance, which is directly related to
the way a test is designed. Bias for best features include the clarity of instructions, explicit
7/29/2019 Diagnostic Test for Social English Programs
4/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 3
attention to time for completing a task, suggestions for how to proceed with a particular task,
visual aids that support mental processes for completing a task, and space for note taking.
Listening comprehension and composition sections include all of the above elements for the
purpose of encouraging the best possible performance out of test-takers.
Prompt and Response Attributes
The Vocabulary Knowledge section (Appendix B) consists of ten multiple choice discrete
point items. Each discrete point item has a sentence which contains an idiomatic expression that
is underlined and embedded in a sentence to give it context because as Nation and Chung (2009)
state, words often do not make much sense unless they are in phrases or larger units. Below each
sentence are four choices from which test-takers must choose which best represents the meaning
of the underlined idiomatic expression. The second section, listening comprehension, contains a
dictocomp or integrative skills section where test-takers listen to auditory stimuli in the form of
two audio clips and then respond to a prompt by writing a short answer
The final section is composition for which test-takers watch a short video clip and carry
out mental processes similar to those of the listening section, but must also connect paralinguistic
(body language) features with sociolinguistic variables. Sociolinguistic knowledge refers to the
understanding of the language of a particular socio-cultural setting and interpreting utterances in
terms of the context of the situation (Buck, 2001). After watching the video clip, test-takers must
compose a summary of fifty to one hundred words. Answer keys and scoring rubrics for response
attributes are located in Appendix B.
7/29/2019 Diagnostic Test for Social English Programs
5/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 4
References
Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language
assessment. New York: McGraw-Hill.
Brown, A. L., & Day, J.D. (1983). Macrorules for summarizing texts: The development of
expertise.Journal of Verbal Learning and Verbal Behavior, 22, 1-14.
Buck, G. (2001). Assessing listening. Cambridge: Cambridge University Press.
Davies, A. (1999).Dictionary of language testing. Cambridge: Press Syndicate of the University
of Cambridge.
Idiom. (2010). In Oxford Dictionaries. Retrieved April, 15, 2010 from Oxford Dictionaries.
http://oxforddictionaries.com/view/entry/m_en_us1256944.
Nation, P., Chung, T. (2009). In M.H. Long & CJ. Doughty (Eds.) The handbook of language
teaching (p.543-559). West Sussex, UK: Wiley-Blackwell.
Swain, M. (1984). Large-scale communicative language testing: A case study. In S. Savignon &
M. Burns (Eds.), Initiatives in communicative language teaching (pp. 185-201). Reading,
MA: Addison-Wesley
Vandergrift, L. & C. Goh, Teaching and testing listening comprehension, in Long, M. &
Doughty, C. (eds.)Handbook of language teaching(Blackwell, 2009), pp. 395-411.
http://oxforddictionaries.com/view/entry/m_en_us1256944http://oxforddictionaries.com/view/entry/m_en_us1256944http://oxforddictionaries.com/view/entry/m_en_us12569447/29/2019 Diagnostic Test for Social English Programs
6/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 5
Appendix A: Diagnostic Test
Section A: Vocabulary Knowledge (1 x 10 points)
Choose the best definition that you think represents the meaning of the words in bold type. You
have 10 minutes to complete this section.
Example:
English is too hard. I give up!
a. Quit a habit b. Please help me
c. Offer to you d. Stop trying
1. You got the job! Good for you!a. You are smart! b. Healthy!
c. Congratulations! d. Rich!
2. Tom said he will be right back.a. turn around b. return soon
c. never be wrong d. stand behind you
3.
Hes rich, but moneyisn't everything.a. everywhere b. a lot
c. important d. useful
4. An increase of 25 cents per gallon of gas is nothing to sneeze at!a. quite significant! b. hard to understand!
c. not really important! d. something that makes you sneeze!
5. Paul is down on himself just because he got one bad grade.a. lying on the floor b. criticizing
c. disappointed with d. agreeing with
7/29/2019 Diagnostic Test for Social English Programs
7/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 6
6. The test will be next month, so make a mental noteofthat!a. remember b. write down
c. believe d. forget
7. John will break upwith Mary today.a. divide into pieces b. stop working
c. end the relationship d. start laughing
8. You got to be kidding me!a. teasing me! b. joking with me!
c. serious with me! d. making fun of me!
9. I never know where a pimple will pop upnext.a. burst upwards b. fly outwards
c. suddenly appear d. go away
10.Things rarely work out in such a simple way.a. get exercised b. get resolved
c. get created d. get calculated
This is the end of Section A. If you have finished early, please remain in your seats
and wait for further instructions.
7/29/2019 Diagnostic Test for Social English Programs
8/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 7
Section B: Listening Comprehension (2 x 5 Points)
This section has two audio clips. Before listening to each audio clip, take 30 seconds and read
information about each audio clip:
Name of audio clip 1:Elaines Birthday
Information about audio clip 1: The two characters in audio clip 1 are Jerry and Elaine. The
audio clip starts with Jerry and Elaine talking at Jerrys house.
Elaine Jerry Setting
Audio Clip 1
- Before audio clip 1 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to the
question below.
*You may use this space to take notes while you are listening.
Question 1: Please describe what is happening in the audio clip.
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
7/29/2019 Diagnostic Test for Social English Programs
9/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 8
Name of audio clip 2:Elaines Birthday
Information about the audio clip 2: The three characters in the audio clips are Jerry, Elaine,
and Kramer. The audio clip starts with Kramer visiting Jerrys house.
Elaine Jerry Kramer Setting
Audio Clip 2
- Before audio clip 2 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to thequestion below.
*You may use this space to take notes while you are listening.
Question 1: Please describe what is happening in the audio clip.
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
__________________________________________________________________
This is the end of Section B. If you have finished early, please remain in your seats and wait
for further instructions.
7/29/2019 Diagnostic Test for Social English Programs
10/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 9
Section C: Composition (10 points)
A short video clip (5min) will be shown. After watching the video clip, you will have 20 minutesto write a summary about the video clip you just watched. Your summary should be between a
50 and 100 words. But before watching the video clip, take 30 seconds and look at the name of
the video clip as well as the characters below:
Name of Video clip:Elaine goes to the beauty salon
Characters:
Elaine George Mr. Costanza(Georges Father)
Kramer Jerry
While you are watching the video clip, you may use the space below to take notes.
*Notes:
7/29/2019 Diagnostic Test for Social English Programs
11/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 10
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
This is the end of the test. If you have finished early, please remain in your seats and wait
for further instructions.
7/29/2019 Diagnostic Test for Social English Programs
12/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 11
Appendix B: Test Answer Key
Section A: Vocabulary Knowledge (1 x 10 points)
Choose the best definition that you think represents the meaning of the words in bold type. You
have 10 minutes to complete this section.
Example:
English is too hard. I give up!
a. Quit a habit b. Please help me
c. Offer to you d. Stop trying
1. You got the job! Good for you!a. You are smart! b. Healthy!
c. Congratulations! d. Rich!
2. Tom said he will be right back.a. turn around b. return soon
c. never be wrong d. stand behind you
3.
Hes rich, but moneyisn't everything.a. everywhere b. a lot
c. important d. useful
4. An increase of 25 cents per gallon of gas is nothing to sneeze at!a. quite significant! b. hard to understand!
c. not really important! d. something that makes you sneeze!
5. Paul is down on himself just because he got one bad grade.a. lying on the floor b. really criticizing
c. disappointed with d. Agreeing with
7/29/2019 Diagnostic Test for Social English Programs
13/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 12
6. The test will be next month, so make a mental noteofthat!a. remember b. write down
c. believe d. forget
7. John will break upwith Mary today.a. divide into pieces b. stop working
c. end the relationship d. start laughing
8. You have got to be kidding me!a. teasing me. b. joking with me.
c. serious with me. d. making fun of me.
9. I never know where a pimple will pop upnext.a. burst upwards b. fly outwards
c. suddenly appear d. go away
10.Things rarely work out in such a simple way.a. get exercised b. get resolved
c. get created d. get calculated
This is the end of Section A. If you have finished early, please remain in your seats
and wait for further instructions.
7/29/2019 Diagnostic Test for Social English Programs
14/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 13
Section B: Listening Comprehension (2 x 5 Points)
This section has two audio clips. Before listening to each audio clip, take 30 seconds and look at
the name of the audio clip, information about each audio clip, and the characters that will be in
the audio clip:
Name of audio clip 1: Elaines Birthday
Information about audio clip 1: The two characters in audio clip 1 are Jerry and Elaine. The
audio clip starts with Jerry and Elaine talking at Jerrys house.
Elaine Jerry Setting
Audio Clip 1
- Before audio clip 1 starts, take a few seconds to preview the prompt below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to the
prompt below.
Scoring Rubric
Score General Description
0 Missing Content/completely Misunderstood1 Complete and correct information
Rater 1 and Rater 2 scores will be averaged to calculate final score.
Question 1: Please describe what is happening in the audio clip.
(1)Jerry surprises Elaine with a birthday present.
(1)But Elaine is unhappy with the birthday present because it is cash.
(1)Because they are friends, it appears that she expects more than cash.
(1)Elaine seems to have expected a more thoughtful present that reflects their friendship.
(1)So she is surprised that Jerrys gift is less than expected.
7/29/2019 Diagnostic Test for Social English Programs
15/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 14
Name of audio clip 2: Elaines Birthday
Information about the audio clip 2: The three characters in the audio clips are Jerry, Elaine,
and Kramer. The audio clip starts with Kramer visiting Jerrys house.
Elaine Jerry Kramer Setting
Audio Clip 2
- Before audio clip 2 starts, take a few seconds to preview the question below.- You will soon listen to an audio recording that is 1 minuteand 30 seconds long.- After the audio recording stops, you will have 5 minutes to write your answer to thequestion below.
Scoring Rubric
Score General Description
0 Missing Content/completely Misunderstood
1 Complete and correct information
Rater 1 and Rater 2 scores will be averaged to calculate final score.
Question 2: Please describe what is happening in the audio clip.
(1)Elaine is surprised that Kramer gave a birthday present to her.
(1) The present turned out to be a bench, which is actually something that she wanted.
(1)In addition, Kramer gives Elaine a birthday card with a poem written in it.
(1) Elaine seems touched by the poem.
(1) So she is happy with both Kramers present and his birthday card.
This is the end of Section B. If you have finished early, please remain in your seats and wait
for further instructions.
7/29/2019 Diagnostic Test for Social English Programs
16/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 15
Section C: Composition (10 points)
A short video clip (5min) will be shown. After watching the video clip, you will have 20 minutesto write a summary about the video clip you just watched. Your summary should be between 50
and 100 words. But before watching the video clip, take 30 seconds and look at the name of the
video clip as well as the characters below:
Name of Video clip:Elaine goes to the beauty salon
Characters:
Elaine George Mr. Costanza
(Georges Father)
Kramer Jerry
While you are watching the video clip, you may use the space below to take notes.
*Notes:
7/29/2019 Diagnostic Test for Social English Programs
17/17
Ramon Mislang MATESOL Component 8 Assessment Instrument 16
Scoring Rubric
Score General Description
0 Missing Content/completely Misunderstood
1 Partial Information/mostly Correct
2 Complete and correct information
Rater 1 and Rater 2 scores will be averaged to calculate final score.
(2)Elaine is late for her appointment at the nail salon, which is run by Korean
ladies.(2)Because she is late, she has to wait for the next available salon
employee. (2)While she is waiting for someone to work on her nails, she hears
the Korean women speaking Korean and thinks they are speaking about her. (2)
After she suspects that they are calling her a dog, Elaine gets Georges dad
who speaks Korean and brings him to the salon. (2)Georges father becomes
angry and confronts the Korean ladies because he knows they are saying bad
things about him. (97 Words)
This is the end of the test. If you have finished early, please remain in your seats and wait
for further instructions.