Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | clemence-jewel-pearson |
View: | 216 times |
Download: | 2 times |
10-21-14
•Review: Responding to Student Writing
•Understanding Rubrics
•Finish TAW (Assessment)
Agenda:
Starter: Discussion of Responding to Student Work
Teacher Comments on Successful and Unsuccessful Assignments…
Traits of Successful Assignments Students have a degree of choiceStudents are interested in the workStudents have a personal connection Work is relevant to student goalsAssignment is concrete & specific:
*clear instructions*clear expectations
Teacher provides tools (scaffolding)and feedback along the way
Models of successful & unsuccessfulwork are provided
Includes low stakes elements beforehigh stakes performance
Is appropriate for students’ ability leveland confidence level
Traits of Unsuccessful Assignments High stakes without adequate practiceTask is artificial (i.e., meaningless) Format is unclearSome terms are undefinedWork has no clear value to students Work is beyond students’ capabilitiesWork is outside students’ comfort zoneWork is outside students’ trust zoneWork is overwhelming
ProofreaderFixes errors (and assumes responsibility for finding errors)
EditorPolishes the text; tells writers what to do to improve (and assumes some responsibility for the quality of the revised draft)
CriticEvaluates quality; points out positives and negatives (and assumes responsibility for “correctness” of evaluations)
CheerleaderPraises the “good stuff” (and, like the critic, assumes responsibility for “correctness” of evaluations)
Roles Readers Can Play When Responding
FacilitatorHelps writers make their own decisions; points out potential problem areas and suggests options (and makes the writer responsible for his or her decisions)
AllyTries to help authors get by the gatekeeper (and shares some responsibility with the writer, at least for offering good advice)
GatekeeperStops sub-standard material from passing (and is responsible to the institution for “quality control”)
Quality of ideas
Appropriateness of the material
Accuracy of content presented
Organization of ideas
Depth/development of ideas
Likely audience reactions
Stylistic issues
Grammar/mechanics issues
Aspects on Which to Comment:
From Straub & Lunsford, 12 Readers Reading
Ways to Respond:
From Straub & Lunsford, 12 Readers Reading
More Options for RespondingNo Responding: Sharing
Sometimes it’s enough simply to let the writer read aloud
Descriptive Responding
•Sayback – Tell the reader what the text says to you
•Pointing – Point to, or identify, key words or phrases
•What’s Almost Said – Identify what the writing implies
•Structure / Voice / Point of View / Level of Abstraction / Attitude toward Reader / Language / Diction / Syntax – Identify these aspects for the writer
•Metaphorical Description – Describe the shape or some other feature of the text
Analytical Responding
•Skeleton feedback – Identify reasons & support, assumptions, and the implied audience
•Believing – Accept everything & offer additional ideas to help build the case
•Doubting – Challenge everything & offer counter arguments that are not addressed
•Descriptive outline – Explain what the text says and does
Reader-Based Responding: Movies of the Reader’s Mind
Criteria-Based Responding: Compare Text to a Rubric
From Elbow & Belanoff, Sharing & Responding
Teacher Comments on Effective and Ineffective Responses…
Traits of Effective Responses Specific goals/questions/tasksFocused/targetedPositive (i.e., praise)Evaluations included suggestionsNot too numerous: just a few comments per paperMaybe combine with a short conference to be sure students understand commentsTrain students how to read our particular comments
Traits of Ineffective Responses Generic (“nice,””good,””awk”)Not fully explainedToo many comments per paperIncomplete thoughtsIllegible
Respondingcould be, but is not necessarily, the same as
Assessingwhich could be, but is not necessarily, the same as
Grading
What does a grade mean?
Does an A in an Advanced Placement class mean the same as an A in a “regular” class?
Does a B in Mrs. Smith’s classmean the same as a B in Mr. Jones’ class?
What’s the difference between an 83 and an 85?
Does a C mean “OK” or “almost failing”?
Grades are not inherently meaningful; they can serve different purposes and even have different meanings in different situations. We therefore need to explain grades to invest them with meaning.
What IS a rubric?
Rubric = measuring stick
Fail Pass
Fail Pass
Fail Pass
Fail Pass?
A rubric should…
…list the criteria being evaluated
…describe a performance at each rating level
Ideally, the rubric should use similar language for each level within a category, so raters compare “apples to apples” when making judgments about that particular feature.
Also ideally, students should have models of each performance level for each category.
Kinds of Rubrics
Analytical Holistic
Score = sum of scores for individual traits
Score = reader’s impression of overall quality
Usually Formative:
intended to help thewriter identify specificareas to work on
Usually Summative:
intended only tomeasure the qualityof the writing
Yields more information Quicker & easier
BOTH can be useful
SCORE CONTENT/DEVELOPMENT ORGANIZATION VOICE CONVENTIONS
4
•Presents a clear central idea about the topic•Fully develops the central idea with specific, relevant details •Sustains focus on central idea throughout the writing
•Has a clear introduction, body, and conclusion.•Provides a smooth progression of ideas throughout the writing.
•Minor errors in standard written English may be present.
3
Presents a central idea about the topicDevelops the central idea but details are general, or the elaboration may be unevenFocus may shift slightly, but is generally sustained
Has an introduction, body, and conclusion.Provides a logical progression of ideas throughout the writing.
Uses precise and/or vivid vocabulary appropriate for the topicPhrasing is effective, not predictable or obviousVaries sentence structure to promote rhythmic readingStrongly aware of audience and task; tone is consistent and appropriate
Errors in standard written English may be present; however, these errors do not interfere with the writer’s meaning.
2
•Central idea may be unclear•Details may be sparse; more information is needed to clarify the central idea•Focus may shift or be lost causing confusion for the reader
Attempts an introduction, body, and conclusion; however, one or more of these components could be weak or ineffective.Provides a simplistic, repetitious, or somewhat random progression of ideas throughout the writing.
Uses both general and precise vocabularyPhrasing may not be effective, and may be predictable or obviousSome sentence variety results in reading that is somewhat rhythmic; may be mechanicalAware of audience and task; tone is appropriate
A pattern of errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English is present; these errors interfere somewhat with the writer’s meaning.
1
•There is no clear central idea•Details are absent or confusing•There is no sense of focus
Attempts an introduction, body, and conclusion; however, one or more of these components could be absent or confusing.Presents information in a random or illogical order throughout the writing.
Uses simple vocabularyPhrasing repetitive or confusingThere is little sentence variety; reading is monotonousThere is little awareness of audience and task; tone may be inappropriate
Frequent and serious errors in more than one category (e.g., capitalization, spelling, punctuation, sentence formation) of standard written English are present; these errors severely interfere with the writer’s meaning.
B BlankOff TopicInsufficient amount of original writing to evaluateUnreadable or illegible
OT
IS
UR
Why USE rubrics?
Identify the target
Save time
Avoid bias
Be transparent about grades
Prepare students for statewide tests
Principles for Using Rubrics Responsibly
Use a rubric that matches your instructional goals.
Acknowledge the limitations of rubrics.
Remember that rubrics don’t simply measure quality;rather, they define quality.
Distribute rubrics to studentsat the BEGINNING of the assignment.
Use a variety of rubrics. (One size DOES NOT fit all!)
Use a rubric that matches your instructional goals.
(Measure what you teach.)
___/60: Ideas
___/40: Organization
___/100 Total Points
Acknowledge the limitations of rubrics.
Irony?Irony?
Wit?Wit?
Using last week’s Using last week’s vocabulary words?vocabulary words?
Creative
Creative
subheadings?
subheadings?Avoiding a problem
Avoiding a problem encountered last time?
encountered last time?
Following the
Following the teacher’s advice?
teacher’s advice?
Remember that rubrics don’t simply measure quality; rather, they define quality.
“How sweet of her to write! What lovely handwriting! ”
“Did Bobby really say that???
“Organization? Diction? Development? Conventions?”
Distribute rubrics to students at the BEGINNING
of the assignment.
Use a variety of rubrics. (One size DOES NOT fit all!)
BeginningMiddleEnd
_____ Word count
_____ Number of examples
_____Creativity
_____Showing connections
_____Fourteen lines
_____Iambic pentameter
_____Rhyme scheme
_____Coherent stanzas
_____Final couplet
____Audience awareness
____Format
____Conventions
DONEDONE / NOT DONE
NOT DONE
Last week’s sample paper got grades from D+ to A. Let’s try scoring it with a “standard” rubric.
(Yes, this is a “top-down” rubric imposed on us by the administration, so the rubric is not directly related to the assignment. Still, rubrics do get imposed on teachers this way, so you might face such a situation in your classroom.)
Assignment for sample paper
Select a “hot topic” in education – an issue about which there is currently discussion and disagreement – and write a paper of 800-1200 words in which you explain the topic and the different views, cite at least four published articles (or editorials or book chapters) relevant to the discussion, and state and support your position on the issue. Use MLA format for citations, and include a Works Cited page.
Questions or comments so far?
Let’s look more closely at a high-stakes rubric for high school students…
SCORE OF 6An essay in this category demonstrates clear and consistent mastery, although it may have a few minor errors. A typical essay
•effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking, using clearly appropriate examples, reasons, and other evidence to support its position
•is well organized and clearly focused, demonstrating clear coherence and smooth progression of ideas
•exhibits skillful use of language, using a varied, accurate, and apt vocabulary
•demonstrates meaningful variety in sentence structure
•is free of most errors in grammar, usage, and mechanics
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
SCORE OF 5An essay in this category demonstrates reasonably consistent mastery, although it will have occasional errors or lapses in quality. A typical essay
•effectively develops a point of view on the issue and demonstrates strong critical thinking, generally using appropriate examples, reasons, and other evidence to support its position
•is well organized and focused, demonstrating coherence and progression of ideas
•exhibits facility in the use of language, using appropriate vocabulary
•demonstrates variety in sentence structure
•is generally free of most errors in grammar, usage, and mechanics
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
SCORE OF 4An essay in this category demonstrates adequate mastery, although it will have lapses in quality. A typical essay
•develops a point of view on the issue and demonstrates competent critical thinking, using adequate examples, reasons, and other evidence to support its position
•is generally organized and focused, demonstrating some coherence and progression of ideas
•exhibits adequate but inconsistent facility in the use of language, using generally appropriate vocabulary
•demonstrates some variety in sentence structure
•has some errors in grammar, usage, and mechanics
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
SCORE OF 3An essay in this category demonstrates developing mastery, and is marked by ONE OR MORE of the following weaknesses:
•develops a point of view on the issue, demonstrating some critical thinking, but may do so inconsistently or use inadequate examples, reasons, or other evidence to support its position
•is limited in its organization or focus, or may demonstrate some lapses in coherence or progression of ideas
•displays developing facility in the use of language, but sometimes uses weak vocabulary or inappropriate word choice
•lacks variety or demonstrates problems in sentence structure
•contains an accumulation of errors in grammar, usage, and mechanics
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
SCORE OF 2An essay in this category demonstrates little mastery, and is flawed by ONE OR MORE of the following weaknesses:
•develops a point of view on the issue that is vague or seriously limited, and demonstrates weak critical thinking, providing inappropriate or insufficient examples, reasons, or other evidence to support its position •is poorly organized and/or focused, or demonstrates serious problems with coherence or progression of ideas
•displays very little facility in the use of language, using very limited vocabulary or incorrect word choice
•demonstrates frequent problems in sentence structure
•contains errors in grammar, usage, and mechanics so serious that meaning is somewhat obscured
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
SCORE OF 1An essay in this category demonstrates very little or no mastery, and is severely flawed by ONE OR MORE of the following weaknesses:
•develops no viable point of view on the issue, or provides little or no evidence to support its position
•is disorganized or unfocused, resulting in a disjointed or incoherent essay
•displays fundamental errors in vocabulary
•demonstrates severe flaws in sentence structure
•contains pervasive errors in grammar, usage, or mechanics that persistently interfere with meaning
Essays not written on the essay assignment will receive a score of zero.
Scoring Guide for the SAT Essayhttp://www.collegeboard.com/student/testing/sat/about/sat/essay_scoring.html
•point of view on the issue; critical thinking
•organization, focus, coherence, and progression of ideas
•use of language (vocabulary)
•variety in sentence structure
•grammar, usage, and mechanics
What SAT Readers are Attending To
•variety in sentence structure
6 - demonstrates meaningful variety in sentence structure
5 - demonstrates variety in sentence structure
4 - demonstrates some variety in sentence structure
3 - lacks variety or demonstrates problems in sentence structure
2 - demonstrates frequent problems in sentence structure
1 - demonstrates severe flaws in sentence structure
Sample Trait at Difference Performance Levels
•variety in sentence structure
6 - demonstrates meaningful varietymeaningful variety in sentence structure
5 - demonstrates varietyvariety in sentence structure
4 - demonstrates some varietysome variety in sentence structure
3 - lacks variety or demonstrates problems in sentence structure
2 - demonstrates frequent problems in sentence structure
1 - demonstrates severe flaws in sentence structure
Sample Trait at Difference Performance Levels
•organization, focus, coherence, and progression of ideas
6 - is well organized and clearly focused
5 - is well organized and focused
4 - is generally organized and focused
3 - is limited in its organization or focus
2 - is poorly organized and/or focused
1 - is disorganized or unfocused
Sample Trait at Difference Performance Levels
•organization, focus, coherence, and progression of ideas
6 - is well organizedwell organized and clearly focusedclearly focused
5 - is well organizedwell organized and focusedfocused
4 - is generally organizedgenerally organized and focusedfocused
3 - is limited in its organization or focus
2 - is poorly organized and/or focused
1 - is disorganized or unfocused
Sample Trait at Difference Performance Levels
POV & Critical Thinking Organization & Focus Use of Language Sentence Structure Grammar, Usage, Mechanics
6
5
4
3
2
1
Analytically, the score would be 5+5+6+3+3, or 22. (ave = 4.4)
Holistically, should it be 4.4? 4? 5?
In mass scoring sessions (such as SAT or AP), readers “calibrate” their scores using “benchmark” papers – papers that experienced readers have agreed demonstrate a particular score. Readers discuss these benchmarks papers and “recalibrate” periodically throughout the scoring session.
In such readings, papers are measured against other papers in that group: that is, some papers should get the top score and some should get the bottom score, as all the scores are relative. (Hence, a given paper could get one score one year and a different score another year.)
The moral:
A rubric designed for someone else’s purposes might not be of much use
in your classroom.
So make your own rubrics!
Trait 1________
Trait 2________
Trait 3________
Trait 4________
Trait 5________
10 pts
9 pts
8 pts
7 pts
6 pts
5 pts
4 pts
3 pts
2 pts
1 pt
YOU decide what traits
to score
YOU decide the value of each trait
YOU create “descriptors” that describe performances at each level
Trait 1title
Trait 2hook
Trait 3intro
Trait 4conclusion
7 pts main idea & preview are artful, clever, or
intriguing to me
6 pts intro includes
main idea & preview
5 ptstitle grabs my
interest & makes me want to read more
hook grabs my interest & makes me want to read more
4 ptstitle reveals some,but not too much,
about topic
intro includesmain idea or
preview, but not both
3 pts hook is mildly
interesting to me
conclusion artfully ties up the piece
2 ptstitle is dull
or summarizesthe topic
has a separate introduction
conclusion gives me a sense of closure
1 pt has a titleopening line seems designed to hook
the reader
opening paragraph is both intro & body
final paragraph is separate from body
paragraphs
0 pts no title no hook no intro no conclusion
Suppose, for example, that you’ve been working on writing introductions and conclusions…
… so you create your own 20-point rubric for scoring only the intro and conclusion:
Trait 1________
Trait 2________
Trait 3________
Trait 4________
Trait 5________
10 pts
9 pts
8 pts
7 pts
6 pts
5 pts
4 pts
3 pts
2 pts
1 pt
YOU AND YOUR STUDENTS decide
what traitsto score
YOU AND YOUR STUDENTS decide the
value of each trait
YOU AND YOUR STUDENTS create “descriptors” that
describe performances at each level
YOUR STUDENTS learn to score
their own drafts
•WHY am I including these particular traits (and not those other ones)?
•WHY am I valuing this trait at X points and that one at Y points?
•HOW WELL do my descriptors match what I’m actually teaching?
•HOW WELL can my students evaluate their own work using this rubric?
Creating your own rubric forces you to REFLECT on your practice:
Key questions:
Analytical or holistic?
How many traits?
How many points (total or per trait)?
Who will be using the rubric?
Reality Check
Assessing does not always equal grading
Rubrics are no panacea for assessment
Brief interview with Maja Wilson
Maja Wilson, Alfie Kohn, and others have written about some of the problems with – and even potential dangers of – using rubrics.
A few concerns about rubrics:
• They create the illusion of objectivity. (The decision to include to exclude any particular trait is a subjective one.)
• They force readers to set aside their professional experience and read “as if” they were somebody else. (That is, rubrics create the fiction of a single reader, although in the real world, no two readers are exactly alike.)
• They can’t capture any feature not specifically targeted.
• They can stifle students’ creativity, especially if students write for the rubric.
• A rubric can be a substitute for “meaningful response.”
• The standardization of rubrics could lead to the standardization of writing.
Wilson’s principles for assessment (from RR, 64-65):
• Honor rhetorical purpose and effect, acknowledging that different readers read differently.
• Encourage new insights; allow readers to identify values in response to specific pieces of writing.
• Understand (and value) disagreement by using it to explore the writer’s purposes and the reader’s responses.
• Encourage readers to articulate their positions.
• Teach students how to extract clarity from disagreements.
(Short version: Acknowledge the messiness of writing.)
Gallagher’s List of Purposes
•Express & reflect•Inquire & explore•Inform & explain•Analyze & interpret•Take a stand•Evaluate & judge•Propose a solution•Seek common ground
You read the text differently, depending on its purpose.
You should therefore also assess the text differently, depending on its purpose.
Subject
Text
Writer
CONTEXT
Reader
What clues in the TEXTtell us what the WRITERthinks about the READER?
What clues
in the T
EXT
tell us th
e write
r’s PURPOSE?
What clues in the TEXTtell us about the CONTEXT?
Gallagher on Assessment
Assessment should improve writing
Adopt the stance of a reader, not a grader
Read through a craft lens
Gallagher on Assessment
Assessment should improve writing
Adopt the stance of a reader, not a grader
Read through a craft lens
Assess during process, not just at the end(Surgery is too late when the patient is already dead)
Gallagher’s Tenets for Teaching Editing
Address needs as they arise
Teach to individuals more than to the class
Don’t use grammar books; make them
Keep the focus narrow
Teach the “big” conventions (p 150)
Don’t correct too much per page
Student editing is generally ineffective
Track the “spelling demons”
Use conferences
“I am not Superman…”
This draft looks good. I like the way you bring in an extra source to comment on the topic of grade inflation today – years after the Staples article was published. The thesis and topic sentences are generally clear (see green highlights), and you offer a fair amount of text-based evidence. Most of the sentences make sense and are grammatically sound. I marked a few errors (blue) and a few questionable lines (yellow), so you can see there’s still room to make your work clearer and cleaner – but for this point in the semester, this draft is solid. B
If you can break out of the five-paragraph theme model, you might have some good papers. You make some good points, and your writing is delightfully error-free. Further, you make sense at the paragraph level. Citation note: I marked two pairs of sentences in gray. In each case, the quote illustrates some point in the sentence that precedes it. To show this connection, you could end the first sentence with a colon; then you wouldn’t have to say “he says” or “he exclaims” at the end. (Remember, a colon says, “Here’s some additional material to illustrate what I just now said.) It makes for smoother, tighter writing. Aside from the novel/short story confusion and the accompanying format errors, my only complaint is with the format: three unrelated features strung together to “support” a fairly obvious claim (that the story “illustrates the life as a parent”). That’s not really a purpose. (WHY is he illustrating parenthood? Is he maybe trying to make a point, or perhaps show that it’s not always as simple as some folks think? I think you could probably come up with a much better thesis, but that you bailed to the easy format so you wouldn’t have to think too hard. (In short, I’m saying that I think you could turn out a really good paper if you’d get away from this middle school format.) C+
The good news is that most of your sentences are solid, and a couple of the paragraphs actually engage in analysis. The bad news, however, is that the whole first third of this draft is merely a summary of the article – useful, but WAYYY too long – and you completely fail to cite any of the quotes, most of which are too long anyway. Starting out with a factual error doesn’t help. (The scene, not the whole book, takes place at a pool.) And using long quotes suggests that you don’t have much of your own to say. When you do have something to say, it’s pretty good. The paragraph that begins with “Another important lesion” (which, of course, should be “lesson,” not “lesion”) is actually pretty solid. Your writing is smooth and easy to follow, and you have something reasonable to say. If the whole paper were that good, it would get a much higher grade. I recommend that you do enough invention to give you enough to say to allow you to cut the extended summary and the overly long quotes; once you do that, you ought to have some good work. But this draft isn’t there yet. C-
End Comments from papers I recently graded
Classroom Realities
Students need to see lots of models
Good models > weak models
You need to write with your students
Gallagher’s Commandments
Remember: writers are fragile
Set “improvement” as your major goal
Focus on the writer, not just the paper
Intervene early
Conference as often as possible
Give students some voice with the rubric
Remember, it’s ENGLISH class, not just LITERATURE class.
(Yes, they need knowledge, but they also need skills.)
For next week, read and respond to ETC, ch. 5.
Philosophy statement due.