Common Formative Common Formative Assessments Assessments

for for ScienceScience

Monica Burgio Daigler, Erie 1 Monica Burgio Daigler, Erie 1 BOCESBOCES

ReviewReview

Look at Deconstruction Look at Deconstruction Template and Trend MapTemplate and Trend Map

Summarize Summarize work/progress thus farwork/progress thus far

Overview of CFAsOverview of CFAs

Read handout from Larry Ainsworth, the CFA guru

Tossed TermsDiscuss

Summative -- Assessment OF Summative -- Assessment OF LeaLearningrning

Summative assessment for unit, quarter, semester, grade level, or course of study

Provides “status report” on degree ofstudent proficiency or mastery relative to targeted standard(s)

Stephanie L. Bravmann, Assessment’s “Fab Four.”Education Week, March 17, 2004, p. 56

Summative -- Assessment OF Summative -- Assessment OF LeLearningarning

Answers question: “Have students achieved the goals defined by a given standard or group of standards?”

Helps teachers judge effectiveness of their teaching practices

Intended to support the assignment of grades

Formative -- Assessment Formative -- Assessment FORFOR LeLearningarning

Formative: given before and during the teaching process

Diagnostic: intended to be used as a guide to improve teaching and learning

Answers key questions: Do students possess critical pre-requisite skills and knowledge? Do students already know some of the material that is to be taught?

Formative -- Assessment FOR Formative -- Assessment FOR LeLearningarning

Provides teachers with information they need to create appropriate work for groups of learners or individual students

Not typically used to assign grades

Assessment InventoryAssessment Inventory

LIST all the important assessmentsyou administer to students during the year.

RANK each assessment as a 1, 2, or 3in terms of having a real impact on

instruction and student learning, with a“1” having the greatest impact and a“3” having the least impact.

DiscussionDiscussion

Which assessments did you rank a “1”?Which assessments did you rank a “1”? Are you administering any assessments Are you administering any assessments

that ranked a “3”?that ranked a “3”? Are there assessments that have no Are there assessments that have no

purpose?purpose? Which of the assessments are used Which of the assessments are used

formatively?formatively? What type of remediation is givenWhat type of remediation is given??

Step by Step….Step by Step….

How to create a How to create a quality CFA…quality CFA…

Steps to Creating a CFASteps to Creating a CFA

Identify power standards from longitudinal data

Determine topics to address with CFA Deconstruct test items to determine

skills and big ideas Write essential questions matched to

skills and big ideas Design CFA

General Item Writing Tips…General Item Writing Tips…

Quality items shouldQuality items should– Reflect higher order thinking skills Reflect higher order thinking skills

(HOTS)(HOTS)– Students should not be able to answer Students should not be able to answer

by by recallrecall, the should have to , the should have to apply apply their their knowledgeknowledge

– Be brief and clear—goal is to “test Be brief and clear—goal is to “test mastery of material, not students’ ability mastery of material, not students’ ability to figure out what you’re asking.” to figure out what you’re asking.” ––Richard StigginsRichard Stiggins

Things to Consider…Things to Consider…

Design fair and bias free items (no Design fair and bias free items (no bias toward gender, ethnicity, or bias toward gender, ethnicity, or language)language)

Format items to match district Format items to match district benchmark assessments, final benchmark assessments, final exams, and state testsexams, and state tests

Top 5 Roadblocks to Effective Top 5 Roadblocks to Effective Item Item WritingWriting

1.1. Unclear Directions or Ambiguous Unclear Directions or Ambiguous StatementsStatements

2.2. Unintentional CluesUnintentional Clues

3.3. Complex PhrasingComplex Phrasing

4.4. Difficult VocabularyDifficult Vocabulary

W. James Popham, Test Better, Teach Better, 2003, p. 64W. James Popham, Test Better, Teach Better, 2003, p. 64

Two Major Types of Two Major Types of AssessmAssessmentent Items Items

Selected ResponseSelected Response

Constructed ResponseConstructed Response

Selected ResponseSelected Response

Requires students to select one Requires students to select one response from a provided list or response from a provided list or provide very brief answerprovide very brief answer

Types include: multiple choice, true-Types include: multiple choice, true-false, matching, short answer/fill-infalse, matching, short answer/fill-in

Can be used to effectively assess Can be used to effectively assess students knowledge of factual students knowledge of factual information, basic concepts, and basic information, basic concepts, and basic skillsskills

Selected ResponseSelected Response

Benefit:Benefit: Student answers can be Student answers can be quickly scored and objectively scored quickly scored and objectively scored as correct or incorrectas correct or incorrect

Drawback:Drawback: Tends to promote Tends to promote memorization of factual information, memorization of factual information, rather than evidence of higher-level rather than evidence of higher-level understanding—understanding—unless the items are unless the items are deliberately designed to do sodeliberately designed to do so..

Reasons for Selected Reasons for Selected RespoResponsense

Better content domain samplingBetter content domain sampling Higher reliabilityHigher reliability Greater efficiencyGreater efficiency ObjectivityObjectivity Measurability for HOTSMeasurability for HOTS Mechanical ScoringMechanical Scoring

Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, pp 65-67pp 65-67

Reasons Against Selected Reasons Against Selected ResponseResponse

Emphasis on learning of isolated Emphasis on learning of isolated facts (teaching to the test)facts (teaching to the test)

Inappropriate for some Inappropriate for some writing/creative purposeswriting/creative purposes

Lack of student writingLack of student writing

Haladyna, Writing Test Items to Evaluate Higher Order Thinking, Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, pp 65-671997, pp 65-67

Multiple Choice to Assess Multiple Choice to Assess HiHigher gher Level Thinking?Level Thinking?

Common myth is that Common myth is that multiple choice items ONLY multiple choice items ONLY assess lower-level thinking assess lower-level thinking skills and therefore will not skills and therefore will not be appropriate for be appropriate for evaluating students higher-evaluating students higher-level thinking skills.level thinking skills.

Dispelling the MythDispelling the Myth

““Research has not yet been done to Research has not yet been done to prove this point. If you want to prove this point. If you want to measure understanding and some measure understanding and some types of mental skills and abilities, types of mental skills and abilities, multiple choice formats can be written multiple choice formats can be written to measure these behaviors to measure these behaviors effectively.”effectively.”

Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, Haladyna, Writing Test Items to Evaluate Higher Order Thinking, 1997, p. 98p. 98

Criteria for Writing SelectCriteria for Writing Selecteded Response ItemsResponse Items

1.1. Write clearly in a focused mannerWrite clearly in a focused manner

2.2. Ask a question with only Ask a question with only oneone best best answeranswer

3.3. Write items consistent with grade level Write items consistent with grade level reading expectation reading expectation

4.4. Eliminate clues leading to correct answerEliminate clues leading to correct answer

5.5. Make response options Make response options briefbrief

Adapted from Richard J. Stiggins, Student-Centered Assessment, 2001.Adapted from Richard J. Stiggins, Student-Centered Assessment, 2001.

Constructed ResponseConstructed Response

Includes short-and extended response Includes short-and extended response (i.e. short answer prompts, essays, and (i.e. short answer prompts, essays, and problem solving requiring writingproblem solving requiring writing

Requires students to organize and use Requires students to organize and use knowledge and skills to answer a knowledge and skills to answer a question or complete a taskquestion or complete a task

Requires scoring guide (rubric) to Requires scoring guide (rubric) to evaluate degree of student proficiencyevaluate degree of student proficiency

Constructed Response IteConstructed Response Itemsms

Benefit:Benefit: Provide teachers with more Provide teachers with more valid inferences about student valid inferences about student understanding than those derived understanding than those derived from selected response itemsfrom selected response items

Drawbacks:Drawbacks: Take longer to score; Take longer to score; can have errors in design; dependent can have errors in design; dependent on student writing proficiency; on student writing proficiency; challenge to score accuratelychallenge to score accurately

More Valid Inferences from More Valid Inferences from Constructed ResponseConstructed Response

““Because a student really needs to Because a student really needs to understand something in order to understand something in order to construct a response based on that construct a response based on that understanding students’ responses to understanding students’ responses to these sort of items will better these sort of items will better contribute to valid inferences than will contribute to valid inferences than will students’ answers to selected-response students’ answers to selected-response items”items”

W. James Popham, Test Better, Teach Better, 2003, p. 87W. James Popham, Test Better, Teach Better, 2003, p. 87

Resources for Common Resources for Common FormFormativeative Assessment Items Assessment Items

Textbook Questions (that meet Textbook Questions (that meet criteria for well-written items)criteria for well-written items)

Assessment or evaluation Assessment or evaluation components of text seriescomponents of text series

State ExamsState Exams Check for permission to duplicate Check for permission to duplicate

copyrighted material!copyrighted material!

Creating a Scoring GuideCreating a Scoring Guide

Scoring Guide (Rubric)Scoring Guide (Rubric)– A set of general and/or specific criteria used to A set of general and/or specific criteria used to

evaluate student performance on a given taskevaluate student performance on a given task– Descriptions of competence or proficiencyDescriptions of competence or proficiency– Identifies degree of proficiency student has Identifies degree of proficiency student has

reached in relation to particular reached in relation to particular standard/content areastandard/content area

ProficientProficient– The level of performance students must meet The level of performance students must meet

to demonstrate attainment of particular to demonstrate attainment of particular standardsstandards

Avoid Subjective LanguaAvoid Subjective Languagege

Such as…Such as…

– CompleteComplete– PartialPartial– AdequateAdequate– GeneralGeneral– SuccessfulSuccessful– GoodGood

– NiceNice– SomeSome– FewFew– ManyMany– MostMost– LittleLittle– CreativeCreative

Strive for Objective Strive for Objective LanguageLanguage

Language that is Language that is – SpecificSpecific– MeasurableMeasurable– ObservableObservable– UnderstandableUnderstandable– Matched to task directionsMatched to task directions

Next Steps…Next Steps…