Summative/Formative Assessments

February 12, 2015Seminar participants will understand the definition of summative and formative assessments by participating in a discussion and activity and then applying this understanding to their classroom practice.

Popham’s ESSENTIAL Question:

Should I set out to fundamentally transform my instructional approach with a significant infusion of formative assessment?

Popham’s Do I or Don’t I?

Get into groups of four. Read the handout. Complete the 4 As Protocol Discuss

Reflect on . . .

…the kinds of assessments you have already used in your classroom experience.

Try to list as many specific examples as you can.

Using that list think now about

The purpose of these assessments

Why do you assess students in your classroom?

Compare and Contrast

Work with a shoulder partner and compare and contrast formative assessment with summative assessment.

Compare Contrast

Summative Formative

Summative Assessments

Assessment of learning Purpose different from formative

Provide evidence of student achievement for the purpose of making a judgment about student competence or program effectiveness

Can be used to inform what comes next for individuals or groups of students

Information used primarily by teachers, administrators, and policy makers to audit performance

Succinct Definition

“…a planned process in which teachers or students use assessment-based evidence to adjust what they are currently doing.”

Popham

Planned process

Assessment-elicited evidence

Teachers instructional adjustments

Students learning tactics adjustment

Why Should I Use the Formative-Assessment Process? 1998 Paul Black and Dylan William

10 year’s worth of published research dealing with classroom assessment, almost 700 studies in all

Conclusions: “The research reported here shows conclusively that formative assessment does improve student learning.” Students gains were “amongst the largest ever reported for educational interventions.”

It is clear that formative-assessment works big time as long as those employing it incorporate the process’s central attributes.

Research

John Hattie (2009) Meta-analysis of more than 800 meta-analyses

that looked at the relative impact of many factors, including family structure, curriculum, teaching practices, and feedback on student achievement. Effect size, or the percentage of standard deviation in

student achievement. Effect size (d) of .6 means that the relationship between a certain factor and student achievement was 60 percent of one standard deviation. In other words the higher the effect size the greater the possible impact.

Evidence Based Argument: ProbabilityResearch (Hattie 2009)

Preterm birth weightTeacher clarityIllnessDietfeedbackDrug useExerciseVocabulary programsSocioeconomic statusTeacher-student relationshipsCreativity programsFamily StructureHome environmentParental involvementProfessional development

Diet (.12)Family structure (.17)Illness (.23)Exercise (.28)Drug use (.33)Parental involvement (.51)Preterm birthrate (.54)Socioeconomic status (.57)Home environment (.51)Professional Development (.62)Creativity Programs (.65)Vocabulary Programs (.67)Teacher-student relationships (.72)Feedback (.73)*Teacher clarity (.75)**

*.73 effect size equates to a 27% increase

From sorting students to having all

students meet standards Schools were designed to rank students on

achievement—the result: winners and losers Today’s schools need to prepare all students

for life-long learning skills—all students are expected to hit academic targets

Assessment for learning creates a culture of confidence: Students know what good work looks like Students can compare their work to the standard Students understand how to close gaps

The Formative Assessment Process

Immediate Instructional Adjustment

Near-Future Instructional Adjustments

Last-Chance Instructional Adjustments

Students’ Learning Tactics

Classroom Climate Shifts

How Might the Formative Assessment Process Be Best Applied?

Formative, Summative, or other?

Imagine that a teacher is explaining a complex concept to a class. Noticing one student’s puzzled expression, the teacher decides to re-explain the complex concept. Has this teacher made an instructional adjustment?

Yes Was it a good idea? Probably. Was this teacher using formative assessment? No (1) the instructional adjustment was based on a

visual clue—a student’s apparent puzzlement—and not on assessment-elicited evidence and (2) the adjustment was ad hoc and not part of a planned process.

Keys to Quality Classroom AssessmentStiggins

Design: The PURPOSE of the assessment is clear Appropriate METHODS and methodology are

employed The TARGETS of the assessment are clear

Effective Use: Effective COMMUNICATION is used Students are INVOLVED in the process

Effective Use of Formative Assessment

Formative assessments are effective when:

Results are clear, accurate, timely, and are understood when communicated.

They are used as a basis for student-centered assessment, student-involved record keeping, and student-involved communication

Student involvement is used to improve student motivation and learning. Students use information to make important instructional decisions.

Assessment Methods (Design)

Selected Response: good for knowledge and some reasoning targets

Constructed Response: good for knowledge and reasoning targets

Performance Assessment: good for knowledge, reasoning, skills, and product targets

Personal communication: good for knowledge, reasoning, and skills

Selected Response

Constructed Response

Performance Assessment

Personal Communication

Knowledge X X X X

Reasoning X X X X

Skills X X

Products X

Declarative and Procedural Knowledge

Declarative knowledge refers to mastery of subject matter content, whereas procedural knowledge is that understanding that is necessary to figure things out and solve problems (reasoning).

More than 80% of questions on PARCC are focused on reasoning skills.

Selected Response

Multiple Choice True/False Matching Fill in the blank Label a diagram

Multiple Choice (Formative)

A multiple choice consists of three parts:

The stem (the question) The key (the “correct” answer) The distractors (plausible choices)

Multiple Choice Design

Stem should be four sentences or fewer Research shows if it is longer than four

sentences, 86% get the question wrong There should be a space between the stem

and the series of answers The brain needs a visual pause

MC Design

Answer letters should be in caps. Students make mistakes when transferring lower

case from the exam to upper case on the scantron

ELL students can struggle with lower case before the answers:

a. At the beginning of the day.b. At the end of the day Note that some ELL

students might end up reading the word bat.

MC Design

Open ended questions fatigue students Poor readers especially They will read the entire stem for each possible

answerThe United States gained control of the land it needed to build the Panama Canal by

A. Negotiating with Columbia.B. Invading and attacking Columbia.C. Implementing the Open Door Policy.D. Encouraging and supporting Panamanian independence.

This does not mean that you should not use them, but you should transition from closed to open-ended

MC Design

Rule of balance: answers (distractors) should look the same or balance them two and two.

A. 33

B. Thirty-two and two thirds

C. 33.13

D. 31

MC Design

Do not use any picture or visual image if it is not used in the question. It fatigues students

Use bold to highlight a key word

No italics—the students skip these

Do not underline words—students might think it is a book title.

MC Design Formative Use

Factors to consider for the key and distractors:

Provide only one correct answer (this is “key”)

Include plausible options that demonstrate a student’s level of understanding

Do not use a distractor that is distracting!

Example

A teacher is planning a field trip and will need school buses to transport students. A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed?

A. 31

B. 31.33

C. 32

D. 36

Try it out

With a partner or partners, write a multiple choice question that follows the preceding rules. Be prepared to share.

Modified Exams

All essential outcomes should be addressed on exams for all students

Vocabulary that is required should not be altered. Vocabulary used in stems should be accessible for all students. (Universal Design)

Since all of the answers are plausible, and therefore used formatively, do not limit the number of distractors. Think of a doctor who gives a battery of tests to one patient

but limits the number for another patient. This limits the ability for the doctor and patient to find out what’s going on.

Found PoemYou have a list of, or highlighted, words from the Popham article. Use those words, plus any

from the powerpoint to produce a Found Poem. Here’s how: 1. Study the words you foundTry to cut your original find in half—50 words from the original

100, 100 from the 200. Change punctuation if you need to. A “found” poem is, after all, a found poem. Adding words not found in the excerpt is technically illegal. However, in order to smooth things out, to make sense, or to make a point, you may add up to four words of your own. Four. Total.

Read your cut-down list of words one more time. Arrange the words so they’re poem-like. (Sometimes you will put key words at the ends or beginnings of lines. Sometimes, for interest or surprise, you may want to break up words that often “go together” [like “white clouds”, by ending one line with “white” and starting the next line with “clouds.”] Break lines so you emphasize key words, get good sounds at the ends of lines, or heighten the reader’s interest.)

.READ ALOUD AS YOU ARRANGE! Test possible line ends by pausing ever-so-slightly at possible “breaks” or line ends. Reading aloud as you work will help you get lines that please you. If it sounds good, trust it. Which line breaks seem to add the most? Sometimes you’ll want lines to end with natural phrases or ends of sentences. Other times you won’t. Arrange the words so they make a rhythm you like.

Example

Wart Cure

Written by Sangita Baruah

Found in The Adventures of Tom Sawyer 

He’s the WARTIEST BOY

trying to cure WARTS.

Plays with frogs so much

the charm’s BUSTED!

Take ‘em off with a bean?

No.

Cure ‘em with dead cats

‘bout midnight

at the

crossroads!

Jam your hand in—

That’ll fetch any WART

and

pretty soon

off

she

comes!

See you in 25 minutes

What’s the point? To do something with the new information, something that makes connections in your brain.

Be prepared to present!

Stiggins

“We must begin to evaluate our assessments in terms of both the quality of the evidence they yield and they effect they have on future learning.”

“Second, we must abandon the limiting belief that adults represent the most important assessment consumers of data-based decision makers in schools. Students’ thoughts and actions regarding assessment results are at least as important as those of adults.”

How much time do you think schools spend on state- and district-mandated tests in ELA and math?

Teach Plus 1.7 Percent of a typical urban student’s school year is spent on testing in

grades three and seven 0.4 Percent of a typical urban student’s school year spent on testing in

kindergarten <1.3 Percent of a typical suburban student’s school year is spent on testing 3.1 Number of hours a typical kindergartner in an urban district spends on

testing annually 16+ Number of hours typical 3rd and 7th graders spend on testing annually in

urban districts 5x Amount of additional time spent on testing in a high-test urban district as

compared to a low-test urban district each year. 120 Number of additional hours a student in a high-test urban district would

spend on testing as compared to a student in a low-test urban school district by 8th grade

1.3 Average amount of additional testing, in hours, that this study’s urban districts require compared to surrounding suburbs, each year between kindergarten and 8th grade

2x Teacher estimates of test-administration time compared to district calendars for kindergarten and 8th grade.

Your own vignette

Working with others, come up with your own fictitious vignette, some of which are and some of which aren’t consistent with the particular definition of formative assessment.

Take 15 minutes Be prepared to share!