Date post: | 21-Dec-2015 |
Category: |
Documents |
Upload: | edmund-townsend |
View: | 216 times |
Download: | 1 times |
1
TSL 3123 – LANGUAGE
ASSESSMENTTOPIC 3
BASIC PRINCIPLES OF ASSSESSMENT
2
LEARNING OBJECTIVESDefine practicality, objectivity, washback effect,
authenticity and interpretability Describe positive and negative effects or washback
of tests Demonstrate an understanding of the features of
authentic test items and discuss its importance Discuss how interpretability can be ensured in
language testing
(Main reference - Brown, H. Douglas, 2004. Language Assessment: Principles and classroom practices. )
3
CHARACTERISTICS OF A TESTHow do you know if the test is effective?
To answer this question, you need to identify five criteria for “testing a test”
PracticalityObjectivity Authenticity, Washback effect Interpretability
To understand these principles let’s read the story on page 26.
Should be applied to assessments of all kinds in general.
Questions to ponder:Can it be given within appropriate administrative
constraints? Is it dependable?Does it accurately measure what you want it to
measure? Is the language in the test representative of real-world
language use?Does the test provide information that is useful for the
learner?
CHARACTERISTICS OF A TEST
5
PRACTICALITYWhen do you say that this test is practical? When
It is not expensive (within budget)The time is appropriateIt is easy to administer (clear directions)The scoring and evaluation procedures are clear
and efficient.Does not exceed available material resources
Applying the practicality principle – read page 40
OBJECTIVITYRefers to the degree to which equally competent
scorers obtain the same results.Most standardised tests of aptitude and
achievement are high in objectivity.The test items are objective type (e.g. MCQ), and
the resulting scores are not influenced by the scorers’ judgement / opinion.
In fact, such tests are usually constructed so that they can be accurately scored by trained clerks and scoring machines.
Highly objective procedure are used – the reliability of the test results is not affected by the scoring procedures.
OBJECTIVITYFor classroom assessments constructed by teachers
or performance-based assessments, objectivity plays an important role in obtaining reliable measures of achievement.
Teachers may not only use objective tests, but also other methods of assessment that require judgemental scoring.
Therefore, to ensure high objectivity:Select assessment procedures most appropriate for the
learning goals being assessed.Make the assessment procedure as objective as
possible – e.g. carefully phrasing the questions and providing a standard set of rules for scoring.
AUTHENTICITYThe degree of correspondence of the characteristics
of a given language test task to the features of a target language task (Bachman & Palmer, 1996).
Lewkowicz (2000) discussed the difficulties of operationalising authenticity in language assessment:Who can certify whether a task or language sample is
“real-world” or not?Chun (2006) asserts that many test types fail to
simulate real-world tasks.
9
4- AuthenticityIf you are claiming that your test is authentic then you are
saying that “ this task is likely to be enacted in the real world” *A reading passage is selected from real world sources
that the students are likely encountered or will encounter.”
“Are the items contextualized rather than isolated?“ the sequencing of items that show no relationship to
one another lacks authenticity” How can you determine somebody's language
performance in reality in the task does not correspond to reality?
Authenticity matters mainly in productive, communicative tasks.
AUTHENTICITYAN AUTHENTIC TEST…
contains language that is as natural as possible.has items that are contextualised rather than
isolated.includes meaningful, relevant, interesting topics.provides some thematic organisation to items, such
as through a story line or episode.offers tasks that replicate real-world tasks.
Applying the Authenticity principleRead page 44 – 45
AUTHENTICITYThe authenticity of test tasks in recent years has
increased noticeably.Many large-scale tests nowadays offer simulation of
real-world tasks in speaking and writing components, of which the performance of these productive skills were not included previously.
Reading passages are selected from real-world sources that test-takers are likely to have encountered or will encounter.
Listening comprehension sections feature natural language with hesitations, white noise, and interruptions.
More tests offer items that are episodic in that they are sequenced to form meaningful units, paragraphs, or stories.
WASHBACK EFFECTThe effect of testing on teaching and learning – e.g.
the extent to which assessment affects a student’s future language development.
Messick (1996) reminded us that the washback effect may refer to both the promotion and the inhibition of learning (beneficial versus harmful/negative) washback.
WASHBACK EFFECT A TEST THAT PROVIDES BENEFICIAL WASHBACK…
positively influences what and how teachers teach.positively influences what and how learners learn.offers learners a chance to adequately prepare.gives learners feedback that enhances their language
development.is more formative in nature than summative.provides conditions for peak performance by the
learner.
WASHBACK EFFECTIn large-scale assessment, washback refers to the
effects that tests have on instruction in terms of how students prepare for the test – e.g., cram courses and teaching to the test.
The current worldwide use of standardised tests for gate-keeping purposes can lead students to focus on gaining an acceptable score rather than on language development.
Positively, many enrollees in test-preparation courses report increased competence in certain language-related tasks (Chapelle, Enright, & Jamieson, 2008).
WASHBACK EFFECTIn classroom-based assessment, washback can
have a number of positive manifestations, ranging from the benefit of preparing and reviewing for a test to the learning that accrues from feedback on one’s performance.
Teachers can provide information to students on useful diagnoses of strengths and weaknesses.
WASHBACK EFFECTWashback also includes the effects of an assessment on
teaching and learning prior to the assessment itself, i.e., on preparation for the assessment.
The challenge to teachers is to create classroom tests that serve as learning devices through which washback is achieved.
Washback enhances a number of basic principles of language acquisition: intrinsic motivation, autonomy, self-confidence, language ego, interlanguage, and strategic investment.
To imply that students have ready access to you to discuss the feedback and evaluation you have given.
WASHBACK EFFECTWays to enhance washback:
To comment generously and specifically on test performance.
Through a specification of the numerical scores on the various subsections of the test.
Formative versus summative tests: Formative tests provide washback in the form of information
to the learner on progress towards goals. Summative tests provide washback for learners to initiate
further pursuits, more learning, more goals, and more challenges to face.
18
INTERPRETABILITY
The concept of interpretability appears in the literature with different names: interpretability, comprehensibility, intelligibility, transparency, etc.
Interpretability concerns the meaningfulness of scores produced by an instrument.
how interpretable are the scores of the instrument?The interpretability of a test's scores flows
directly from the quality of its items and exercises.
19
Interpretability consideration is especially important when persons other than the designers of the test are to interpret the results. In order to be interpretable, the measuring instrument must be supplemented by the following:
detailed instructions for administering the test,scoring keys,evidence about the reliability, andguides for using the test and interpreting results.
INTERPRETABILITY
20
Samples of English Tests to Evaluate using the five principles
of language testinghttp://www.nysedregents.org/Grade3/EnglishLan
guageArts/samplelisteningselection.pdfhttp://www.henry.k12.ga.us/wle/home/crct/first%2
0grade/English%20test2.pdfhttp://www.henry.k12.ga.us/wle/home/crct/first
%20grade/Grammar%20and%20Mechanics%20Test.pdf
21
TUTORIALPairwork:
Study some commercially produced tests and evaluate the authenticity of these tests/ test items
Discuss the importance of authenticity in testing.
ISL
Look up materials on commercially produced tests.
22
ReferencesBrown, H. Douglas, 2004. Language Assessment: Principles
and classroom practices. Pearson Education, Inc.Chitravelu, Nesamalar, 2005. ELT Methodology: Principles
and Practice. Penerbit Fajar Bakti, Sdn, Bhd.