2/12/2016
1
Designing Rubrics for Classroom UseSue BrookhartApril 1, 2016
How do you and your colleagues currently use rubrics in your class or school? What are your current thoughts about the
effectiveness of these practices? What issues or questions have arisen?
2
© S. M. Brookhart, 2015
3
Both the student’s performance and how you observe or score it defines what kind of evidence you get.
2/12/2016
2
Some people call any scoring scheme a “rubric.”
ChecklistsRating scalesPoint schemesTrue rubrics
© S. M. Brookhart, 2015
4
My sentence has__ a capital letter.__ a period.__ a complete
thought.
Math problem solvingI described what the problem was asking.
EXCELLENT GOOD FAIR POORI wrote an equation, or used a diagram or model, to solve the problem.
EXCELLENT GOOD FAIR POORI calculated the correct answer.
EXCELLENT GOOD FAIR POORI labeled my answer.
EXCELLENT GOOD FAIR POOR
Did I reason from examples to arrive at a clear, accurate description of physical and chemical changes?
2 = Completely and clearly – response gives clear evidence of reasoning from the examples1 = Partially – response is accurate, but reasoning from examples isn’t clear or is only partial0 = No – response does not demonstrate reasonable conclusions from the examples © S. M. Brookhart, 2015
5
True rubrics have 2 elementsA coherent set of criteriaDescriptions of levels of
performance quality
© S. M. Brookhart, 2015
6
2/12/2016
3
© S. M. Brookhart, 2015
7
Parker & Breyfogle, 2011
Math Problem Solving
I can do the problem correctly and get the correct answer.
I can figure out a strategy to use to solve the problem.
I can explain my work – tell what I did and why I did it.
© S. M. Brookhart, 2015
8
Effective RubricsThe rubric itself must be well
designed.The rubric should be used
for learning as well as grading.
9
2/12/2016
4
Rubrics Help Teachers Teach A focus on the criteria by which learning
will be assessed means a focus on What students are going to learn rather
than what you are going to teach
10
Rubrics Help Coordinate Instruction
Show students what desired performance looks like Gives students a way to know how they’re doing and
decide what next steps to take Assessment
11
Rubrics Help Students Learn Research in many subjects
English, Mathematics, Science, Social Studies
At many grade levels Primary, Secondary, College
Brookhart & Chen, 2015
12
2/12/2016
5
13
Criteria Should be about the learning
to be demonstrated, not about the requirements for the assignment
Matched to the learning goal or achievement standard you want to measure and report on
© S. M. Brookhart, 2015
14
Effective Criteria Appropriate Definable Observable Distinct from one another Complete [for your purposes] Able to be described along a continuum
of quality
© S. M. Brookhart, 2015
15
2/12/2016
6
Effective Performance Descriptions Descriptive Clear Cover the whole range of performance Distinguish among levels Center the target performance (mastery,
passing) at appropriate levels Feature parallel descriptions from level to
level
© S. M. Brookhart, 2015
16
How should you choose the language you use for performance level descriptions in rubrics? [Use examples on the next slides, and explain your reasoning.]
17
© S. M. Brookhart, 2015
Example #1 Poor example
Good example
© S. M. Brookhart, 2015
18
4 3 2 1
Gram-mar & Usage
Grammar, punctuation, spelling and capitalization are correct or nearly so, and minor errors do not detract from reading the story.
Grammar, punctuation, spelling and capitalization are mostly correct. Readers can still figure out what is meant in the story.
Many errors in grammar, punctuation, spelling and capitalization detract from the story.
Numerous errors in grammar, punctuation, spelling and capitalization severely detract from the story.
4 3 2 1
Gram-mar & Usage
No errors in grammar,punctuation,spelling or capitalization.
1-3 errors in grammar, punctuation,spelling or capitalization.
4-5 errors in grammar, punctuation,spelling or capitalization.
6 or more errors in grammar, punctuation,spelling or capitalization.
2/12/2016
7
© S. M. Brookhart, 2015
19
Example #2 Poor example [one criterion from science lab report rubric]
4 3 2 1Intro-duct ion
1. Includesquestion to be answered.
2. States hypothesis based on research and/or reasoning
3. Title is relevant
4. Hypothesis (prediction) is testable
One of the “excellent” conditions is not met.
Two of the “excellent” conditions are not met.
Three of the “excellent” conditions are not met.
© S. M. Brookhart, 2015
20
Example #2Good example
4 3 2 1
Intro-duction—Stating Research Questions and Hypotheses
States ahypothesis that is based on research and/or sound reasoning and is testable. Report title reflects question or hypothesis.
States a hypothesis that is based on research and/or sound reasoning and is testable. Report title may not reflect the question or hypothesis.
States a hypothesis, although basis for the hypothesis is not clear or hypothesis is not testable. Report title may not reflect the question or hypothesis.
Does not state a hypothesis. Introduction may be a general statement of the topic or the assignment or may be missing or unclear.
© S. M. Brookhart, 2015
21Example #3 Poor example [one criterion from science oral report]
Good example
4 3 2 1Props and Visuals
Props and/or visuals (pictures, diagrams, maps) were used throughout the presentation.
A few props and/or visuals were used.
One props or visual was used.
No props and/or visuals were used.
4 3 2 1Props and Visuals
Props and/or visuals (pictures, diagrams, maps) give listeners an exceptionally clear understanding of the concepts.
Props and/or visuals mostly illustrate the concepts for listeners.
Some props and/or visuals illustrate the concepts and some do not.
Props and/or visuals were irrelevant to the concepts,inaccurate, and/or confusing.
2/12/2016
8
Example – Choral Music RubricA B C D F
Accuracy Always on pitchRhythms correct
Only a few incorrect notesMostly accurate rhythm, stable beat
Usually on pitchSome minor rhythm problems, generally a steady beat
Rarely on pitch, many incorrect notesSome rhythmic errors and changes in beat
Not on pitchMany rhythmic errors and changes in beat
Text Fully memorizedAlways clear with precise sounding consonants
Mostly memorizedMostly clear with attempted ending consonants
Some memorizationUsually clear but missing consonants
Little memorizationUnclear or mumbled
No evidence of memorizationNot clear
Part Work Knows all partsWell balanced sound (both parts heard equally)
Knows most partsMostly balanced sound
One part not well knownAdequate balance between parts
Both parts not well knownParts not balanced
Parts mostly unknownNot able to hear two parts
Tone Quality Clear and projectedEven tone between head & chest voiceConsistent and effective breath support
Mostly clear & projectedSome difference in tone between head & chest voiceBreath support effective most of the time
Beginning to be clear & projectedMuch difference in tone between head & chest voiceMore breath support needed
Unclear and unsupported (lack of projection), or a harsh tone in many sections
Weak (no projection)Tone harsh, unsupported, and/or uncontrolled
Interpretation Consistent and correct use of dynamics and expression
Effective use of dynamics and expression
Some dynamic variationAdequate expression
Very few dynamicsLittle expression
No dynamicsNo expression
22
© S. M. Brookhart, 2015
First draft, Life Cycle Project rubric© S. M. Brookhart, 2015
236 Points 4 Points 2 Points 0 Points
Title of poster
Title is evident on poster, correctly spelled and capitalized
Title is on poster, but with errors or it is hard to read
No title or heading
Order of life cycle stages
All the stages of the life cycle are in the correct order. Stages are correctly labeled.
One or more stages of the life cycle are in the wrong order.
Not included
Illustrations of life cycle stages
Illustrations of each stage are evident.
One or two illustrations of the life cycle stages are missing.
More than 2 illustrations of the life cycle stages are missing.
Not included
Description of life cycle stages
Stages are described with at least 2 details.
Stages are described with one detail. One or more stage is missing.
Stages are incomplete missing. Stages have one or zero supporting details.
Not included
Overall appearance of poster
Poster is very neat and organized. Title and all sentences have correct spelling, capitalization, and punctuation.
Poster is somewhat neat and organized. Some correct spelling, punctuation, and capitalization. Poster shows signs of little effort.
Poster is messy, many errors, not colored, or unfinished. Poster shows no signs of effort.
Revised version, Life Cycle Project rubric,Used with a checklist for assignment requirements
© S. M. Brookhart, 2015
24
Advanced Proficient Nearing Proficient Novice
Order of life cycle stages
All the stages of the life cycle are in the correct order and correctly labeled.
One or more stages of the life cycle are in the wrong order.
No order is specified, or order is incorrect.
Illustrations of life cycle stages
Each stage has an illustration that gives an especially clear or detailed view about what happens to the animal then.
Each stage has an illustration that helps show what happens to the animal then.
Some stage illustrations do not show what happens to the animal then.
Illustrations do not help show what happens to the animal during its life cycle.
Description of life cycle stages
Stages are described accurately. Descriptions are especially complete and detailed.
Stages are described accurately.
Stages are described with some inaccurate or incomplete information.
No stages are described, or stages are described inaccurately.
2/12/2016
9
Checklist for assignment requirements Not used for grading – used formatively By students (self and/or peers)
© S. M. Brookhart, 2015
25
My Poster Checklist___ My poster has a title.___ My poster is neat.___ My poster is well organized.___ My poster has correct spelling.___ My poster has correct capitalization.___ My poster has correct punctuation.
What happens when bad rubrics are used?26
© S. M. Brookhart, 2015
Student A – 7th grade inclusion
27
© S. M. Brookhart, 2015
2/12/2016
10
28
© S. M. Brookhart, 2015
Student B – 7th grade inclusion29
© S. M. Brookhart, 2015
30
© S. M. Brookhart, 2015
2/12/2016
11
Critique these Volcano Project Rubrics31
CATEGORY 5 4 3 2 1 0Student Info Included (Name, Date, Period)
All items included One item missing Two items missing No info
provided
General Info Included
All 8 items included
One item missing or inaccurate
Two items missing or inaccurate
Three items missing or inaccurate
More than 3 items missing or inaccurate
No info provided
Eruption Information
All 6 items included
At least 4 items are included and accurate
Half the information is included or accurate
One 2 items included and accurate
Minimal or no information included or accurate
No info provided
Volcano DiagramClear, accurate diagram; with all 15 parts shown.
Diagram is included; 11-14 of parts clear and accurately shown.
Diagram is included; 6-10 parts clear and accurately shown.
Diagram is included; 3-5 labeled parts.
Diagram has less than 3 parts labeled.
No info provided
Overall Presentation
Clear, neat, organized. Layout well planned.
Layout is planned and organized. Writing is not neat
Info could be better organized. Writing is sloppy
Not organized. Not all info fits properly. Some attempt to make it work. Writing and lines are hastily done.
Very disorganized and poorly prepared. Lines not straight. Spacing is sloppy. Writing is hastily done. May have been done in homeroom.
No suggestions followed for organization and neatness
Use of Creativity
Various materials are used for effect. Attention to detail obvious. Good use of color.
Some use of materials, attention to detail, and/or use of color.
Moderate use of varied materials, attention to detail, and/or color.
Minimal use of either varied materials, attention to detail, or color.
No use of either varied materials, attention to detail, or color.
No creativity
© S. M. Brookhart, 2015
Volcano Project Rubrics
© S. M. Brookhart, 2015
32
Only half of the score is about understanding volcanoes
All of the points that are about volcanoes have to do with counting facts (requiring copying, not even recall)!
33
CATEGORY 5 4 3 2 1 0Student Info Included (Name, Date, Period)
All items included One item missing Two items missing No info
provided
General Info Included
All 8 items included
One item missing or inaccurate
Two items missing or inaccurate
Three items missing or inaccurate
More than 3 items missing or inaccurate
No info provided
Eruption Information
All 6 items included
At least 4 items are included and accurate
Half the information is included or accurate
One 2 items included and accurate
Minimal or no information included or accurate
No info provided
Volcano DiagramClear, accurate diagram; with all 15 parts shown.
Diagram is included; 11-14 of parts clear and accurately shown.
Diagram is included; 6-10 parts clear and accurately shown.
Diagram is included; 3-5 labeled parts.
Diagram has less than 3 parts labeled.
No info provided
Overall Presentation
Clear, neat, organized. Layout well planned.
Layout is planned and organized. Writing is not neat
Info could be better organized. Writing is sloppy
Not organized. Not all info fits properly. Some attempt to make it work. Writing and lines are hastily done.
Very disorganized and poorly prepared. Lines not straight. Spacing is sloppy. Writing is hastily done. May have been done in homeroom.
No suggestions followed for organization and neatness
Use of Creativity
Various materials are used for effect. Attention to detail obvious. Good use of color.
Some use of materials, attention to detail, and/or use of color.
Moderate use of varied materials, attention to detail, and/or color.
Minimal use of either varied materials, attention to detail, or color.
No use of either varied materials, attention to detail, or color.
No creativity© S. M. Brookhart, 2015
2/12/2016
12
Sometimes rubrics can be very concrete Example – third grade Concepts of Multiplication poster Present a multiplication problem, a
repeated addition problem, a picture with equal groups, and an array for the same multiplication fact
Poster
34
© S. M. Brookhart, 2015
35
© S. M. Brookhart, 2015
Example36
© S. M. Brookhart, 2015
2/12/2016
13
37
© S. M. Brookhart, 2015
Example38
© S. M. Brookhart, 2015
More Complex Rubrics forMore Complex Tasks
Several criteria Analytic rubrics – criteria considered one at a time Holistic rubrics – criteria considered simultaneously
Criteria should be qualities that work should exhibit (e.g., position is supported, steps are explained with mathematical vocabulary, cites appropriate sources)
Criteria should NOT be about directions (e.g., has 4 sentences, cites 3 sources)
39
© S. M. Brookhart, 2015
2/12/2016
14
Example
40
2 1 0Understanding the Problem
Complete understanding of the problem
Part of the problem misunderstood or misinterpreted
Completemisunderstanding of the problem
Planning aSolution
Plan could have led to a correct solution if implemented properly
Partially correct plan based on part of the problem being interpretedcorrectly
No attempt, or totally inappropriate plan
Getting anAnswer
Correct answer and correct label for the answer
Copying error, computational error, partial answer for a problem with multiple answers
No answer, or wrong answer based on an inappropriate plan
(NCTM, 1987, 1991)© S. M. Brookhart, 2015
ExampleJan's Snack Shop has 3 flavors of ice cream—vanilla, chocolate, and strawberry. The ice cream can be served in a dish, a sugar cone, or a regular cone.There are 9 people who choose 1 dip of ice cream in a dish, or in a sugar cone, or in a regular cone, and all of their choices are different. List or show the 9 different choices.Could another person have a choice that is different from one of these 9 choices? Why or why not?
(NAEP 2003, grade 4,Block M10 #17)
41
© S. M. Brookhart, 2015
Example to Score42
© S. M. Brookhart, 2015
2/12/2016
15
Example to Score
43
© S. M. Brookhart, 2015
Consider a Template4 Task is complete and correct, plus it includes an extension,
exceptional insight, or some other feature that is “above and beyond” what was required or intended.
3 Task is complete and correct. Response shows understanding of intended learning outcome.
2 Task is partly complete and correct. Response shows some understanding of intended learning outcome.
1 Task is not complete or correct. Response shows little/no understanding of intended learning outcome.
44
NOTE: DON’T use this “rubric” as is – it’s a template. It lacks CRITERIA.Replace the “boilerplate” language with criteria specific to your learning outcome of interest.
© S. M. Brookhart, 2015
Example – 6th ELA – Writing a well-structured paragraph in response to a prompt
Direct instruction and practice, four weeks Learning outcome = can write a
paragraph that Answers the prompt question Has a focus/topic sentence Has detail sentences that relate to the
focus Has an effective ending sentence
45
© S. M. Brookhart, 2015
2/12/2016
16
Example – 6th grade writing4 Paragraph has a focus sentence, details, and conclusion, plus it
includes a particularly compelling response to the prompt, for example with especially vivid details.
3 Paragraph is has a focus sentence, details, and conclusion.
2 Paragraph is has some attempt at a focus sentence, details, and conclusion.
1 Paragraph does not have a focus sentence, details, and conclusion.
46
© S. M. Brookhart, 2015
Example – 6th grade writing47
© S. M. Brookhart, 2015
For Simple Tasks3 Task is complete and correct. Response shows understanding of
intended learning outcome.2 Task is partly complete and correct. Response shows some
understanding of intended learning outcome.1 Task is not complete or correct. Response shows little/no
understanding of intended learning outcome.
48
NOTE: DON’T use this “rubric” as is – it’s a template. It lacks CRITERIA.Replace the “boilerplate” language with criteria specific to your learning outcome of interest.
© S. M. Brookhart, 2015
2/12/2016
17
Kindergarten ExampleMaking Equal Sets
3 Elements of each set are organized in a single line. There is a one-to-one correspondence between elements of each set.
2 There is an attempt at organizing elements of each set and showing correspondence between elements of each set. These attempts are not entirely successful.
1 Elements of each set are not organized. No correspondence is shown between elements of each set.
49
© S. M. Brookhart, 2015
Kindergarten ExampleMaking Equal Sets
50
A B© S. M. Brookhart, 2015
Summary Task(s) or assignment must tap the student
learning outcomes Knowledge Skills Level of Thinking
Rubrics must accurately describe student performance along a continuum of quality ON THOSE SAME LEARNING OUTCOMES
51
© S. M. Brookhart, 2015
2/12/2016
18
4 3 2 1Topic Strongly and clearly
states a personal topic/opinion.
Some reference to the issue but does not state a topic/opinion.
Personal topic/opinion is hard to understand.
Personal topic/opinion is not clearly stated.
Reasons and Support
3 reasons stated with excellent support.
3 reasons are made but the support is weak or lacking evidence.
Only 2 points are made with weak arguments.
Arguments are weak and less than 2 points are made.
Introduction Paragraph introduces the paper and has 4 sentences.
Paragraph introduces the paper but contains only 3 sentences.
Introduction paragraph is evidence but does not introduce paper. Only 1-2 sentences.
Introduction paragraph is not evident.
Conclusion Paragraph concludes paper and contains 4 sentences.
Paragraph concludes the paper but contains only 3 sentences.
Concluding paragraph is evident but does not conclude paper. Only 1-2 sentences.
Concluding paragraph is not evident.
Mechanics and Grammar
Contains 1-2 errors in spelling, punctuation, and grammar usage.
Contains 3-5 errors in spelling, punctuation, and grammar usage.
Contains 5-7 errors in spelling, punctuation,and grammar usage.
Contains more than 7 errors in spelling, punctuation, and grammar usage.
Organization Paper is organized into 3 evident paragraphs of an intro, body, and conclusion. Sentences are well written.
Paper is organized into 3 evident paragraphs of an intro, body, and conclusion. Sentences are average.
Paragraph and sentence structure is inconsistent.
There is no logical organization to the paper.
52
Different Types of Rubrics Have different strengths
and weaknesses Serve different purposes
53
Rubrics Can be Classified According to how the criteria are
considered Analytic (consider one criterion at a
time) Holistic (consider all criteria
simultaneously) According to the specificity of the
performance descriptions General Task-specific
54
2/12/2016
19
Analytic Rubrics55
Definition Advantages DisadvantagesEach criterion is evaluated separately
• Gives diagnostic information to teacher
• Gives formative feedback to students
• Easier to link to instruction than holistic rubrics
• Good for formative assessment; adaptable for summative assessment; if you need an overall score for grading, you can combine the scores
• Takes more time to score than holistic rubrics
• Takes more time to achieve inter-rater reliability than with holistic rubrics
Holistic Rubrics56
Definition Advantages DisadvantagesAll criteria are evaluated simultaneously
• Scoring is faster than with analytic rubrics
• Requires less time to achieve inter-rater reliability
• Good for summative assessment
• Single overall score does not communicate information about what to do to improve
• Not good for formative assessment
General Rubrics57
Definition Advantages DisadvantagesDescription of work gives characteristics that apply to a whole family of tasks
• Can share with students, explicitly linking assessment and instruction
• Reuse same rubrics with several tasks or assignments
• Supports learning by helping students see “good work” as bigger than one task
• Support student self-evaluation
• Students can help construct generic rubrics
• Lower reliability at first than with task-specific rubrics
• Requires practice to apply well
2/12/2016
20
Task-Specific Rubrics58
Definition Advantages DisadvantagesDescription of work refers to the specific content of a particular task (e.g., gives an answer, specifies a conclusion)
• Teachers sometimes say using these makes scoring “easier”
• Requires less time to achieve inter-rater reliability
• Cannot share with students
• Need to write new rubrics for each task
• For open-ended tasks, good answers not listed in rubrics may be evaluated poorly
For most classroom purposes…
59
60
Quiz
2/12/2016
21
1. Why is a general scoring rubric useful?
a. It gives points for each fact or concept the student mentions in an essay or problem.
b. It provides information on criteria that students get to know as important.
c. It is easier to understand than a specific rubric.
61
2. Descriptions of the different quality levels of performance are part of scoring rubrics.
a. Trueb. False
62
3. Mr. Ong’s mathematics performance task requires students to develop and explain two or more ways to solve a mathematics problem. Because students will give many different but correct solutions to the problem, Mr. Ong’s scoring rubric does not need to describe how the alternative solutions will be marked.
a. Trueb. False
63
2/12/2016
22
4. Examples of student work can be used to anchor each quality level in a rubric or rating scale.
a. Trueb. False
64
5. An analytic scoring rubric gives students more detailed feedback than a holistic scoring rubric.
a. Trueb. False
65
6. Suppose correctness of spelling is not part of the learning targets being assessed by an essay test. How should spelling be handled when scoring the essays?
a. Let spelling correctness influence students’ scores because other learning targets require the ability to spell correctly.
b. You might comment on spelling but it is not one of the criteria used in determining students’ scores.
c. Ignore spelling as best you can while scoring the relevant factors.
66
2/12/2016
23
7. What is the most appropriate way to score a performance assessment that is used to identify students’ problems for remediation?
a. Analytic scoringb. Holistic scoringc. Point-based scoring
67
8. Rubrics for scoring questions or problems cannot be written until one specifies the criteria to be assessed.
a. Trueb. False
68
9. Ms. King’s students did not understand the meaning of each level on a rubric she used for oral presentations of reports. Does this mean she has crafted a poor quality rubric?
a. Yesb. No
69
2/12/2016
24
10.The performance level descriptions for Ms. Lee’s rubric for students’ science reports read: “failing”, “poor”, “good”, “very good”, and “excellent,” for each criterion. Does this mean she has created a poor quality rubric?
a. Yesb. No
70
On Target Not on TargetContent I shared an
insight or thought that was genuine, relevant, and worthwhile.
I shared athought that was made-up, irrelevant, or trivial; OR I didn’t share a thought.
Reason-ing
I explained why I chose the thought I shared.
I didn’t explain why I chose the thought I shared; OR myexplanation was not sensible.
71
More Details 72
