Demystifying Second Language Assessment

Shawna WilliamsBC TEAL Annual Conference

May 24, 2014

Agenda

IntroductionAssessment terminology – definitions

Principles of Assessment

My assessment history…

Language Assessment: Principles and Classroom Practices

Assessment Terminology

Assessment Terminology

Assessment ≠ Testing

Assessment is…“appraising or estimating the level or

magnitude of some attribute of a person.”

(Mousavi, 2009)

“an ongoing process of collecting information about a given object of interest according to procedures that are systematic and substantively grounded.”

(Brown & Abeywickrama, 2010)

Assessment“A good teacher never ceases to

assess students, whether those assessments are incidental or intended.”

(Brown & Abeywickrama, 2010)

Assessment and Learning

Teaching

Assessment

Measurement

Tests

Evaluation

(Brown & Abeywickrama, 2010, p. 6)

Function of AssessmentInformal FormalIncidental,

unplanned comments

CoachingImpromptu

feedback on homeworkNonjudgmental

Systematic and planned

Give T and Ss appraisal of achievement

Tests and assignments

Function of AssessmentFormative Summative“evaluating students

in the process of ‘forming’ their competencies and skills with the goal of helping them to continue that growth process”

feedback on performance

future continuation of learning

“aims to measure, or summarize, what the student has grasped”

end of course or unitlooking back and

taking stock

(Brown & Abeywickrama, 2010, p.7)

Five Principles of AssessmentHow do you know if an assessment task is effective, appropriate, useful, or . . . “good”?

Practicality

Reliability

Validity

Authenticity

Washback

A PRACTICAL TEST . . .

A PRACTICAL TEST . . . budgetary limitsappropriate time constraintsclear directions for administrationappropriately utilizes human resourcesdoes not exceed available material resources

considers time and effort for design and scoring

(Brown & Abeywickrama, 2010, p. 26)

A RELIABLE TEST . . .

=

A RELIABLE TEST . . .consistent across two or more

administrationsclear directions for scoring/evaluationuniform rubrics for scoring/evaluationconsistent application of rubrics by

scorerunambiguous to the test-taker

(Brown & Abeywickrama, 2010, p. 27)

Reliability con’tRater Reliability

Inter-Rater Reliability

Intra-Rater Reliability

Reliability con’tRater Reliability

Inter-Rater ReliabilityIntra-Rater Reliability

Student-Related ReliabilityTest Administration ReliabilityTest Reliability

A VALID TEST . . .

A VALID TEST . . . measures exactly what it proposes to measuredoes not measure irrelevant or

“contaminating” variablesrelies on empirical evidence (performance)performance that samples the test’s criterion

(objective)useful, meaningful information about test-

taker’s abilitysupported by theoretical rationale or

argument(Brown& Abeywickrama, 2010, p. 30)

Validity con’tContent-Related EvidenceCriterion-Related EvidenceConstruct-Related EvidenceConsequential Validity (Impact)Face Validity

AN AUTHENTIC TEST . . .

AN AUTHENTIC TEST . . .language as natural as possibleitems are contextualized rather than

isolatedmeaningful, relevant, interesting

topicsthematic organizationreal-world tasks

(Brown & Abeywickrama, 2010, p. 37)

A TEST THAT PROVIDES BENEFICIAL WASHBACK . . .

A TEST THAT PROVIDES BENEFICIAL WASHBACK . . .positively influences teachers’

teachingpositively influences learners’

learninglearners can adequately preparefeedback for language developmentmore formative than summativeconditions for peak performance

(Brown & Abeywickrama, 2010, p. 38)

Applying Principles to Creation of Assessment Tools1. Practical test procedures?2. Test is reliable?3. Rater reliability?4. Content validity?5. Impact has been accounted for?6. Procedure is “biased for best”?7. Test tasks are authentic?8. Test offers beneficial washback?

9. See Brown & Abeywickrama, Chpt. 2

Questions or Comments?Shawna Williams

swilliams@listn.info