7/29/2019 measuring complex achiemvents
1/25
Measuring Complex
Achievement: Essay
Questions
7/29/2019 measuring complex achiemvents
2/25
Objectives for Chapter 10
Discuss the use of essay questions
Compare two types of essay questions
Hypothesize the advantages anddisadvantages of using essay questions
Compare two types of scoring rubrics
Evaluate sample essays using a scoring
rubricCreate examples of two types of essay
questions
7/29/2019 measuring complex achiemvents
3/25
Whats So Great About Essay
Questions?They allow assessment of students
abilities to:
Formulate problemsOrganize, integrate, and evaluate ideas
and information
Apply knowledge and skills
7/29/2019 measuring complex achiemvents
4/25
Forms and Uses of Essay
Questions
May be used to measure knowledge of
factual information Full potential realized with more higher-
order thinking problems
7/29/2019 measuring complex achiemvents
5/25
Useful in meeting learning outcomes
concerned with the abilities to
conceptualize, construct, organize,
integrate, relate, and evaluate ideas in
content areas such as: history, civics,
literature, science, math, etc.Most widely used when the main focus is
on student writing in any subject
Examples: distinguishing between
narrative essays, expository essays, andpersuasive essays or focusing on writing
process
7/29/2019 measuring complex achiemvents
6/25
Confucius Say
The freedom of response provided by essayquestions is not an all-or-nothing affair but,
rather, a matter of degrees.
RestrictedResponse
Extended-Response
7/29/2019 measuring complex achiemvents
7/25
Restricted Response Essay
QuestionsUsually limits both the content and the
response by restricting the scope of thetopic to be discussed, generallyindicated in the question
Useful for measuring learning outcomesrequiring interpretation and application ofdata in a specificarea
Any outcomes measured by an objectiveinterpretive exercise can be measuredby a restricted response essay question
7/29/2019 measuring complex achiemvents
8/25
Examples of Restricted
Response Essay QuestionsDescribe two situations that demonstrate
the application of the law of supply anddemand. Do not use those examplesdiscussed in class.
State the main differences between theVietnam War and previous wars in whichthe United States has participated.
Why is the barometer one of the mostuseful instruments for forecastingweather? Answer in a brief paragraph.
7/29/2019 measuring complex achiemvents
9/25
The Restricted Response essay questionprovides for more ease of assessment,but it restricts the scope of the topic to
be discussed and indicates the nature ofthe desired response to the studentwhich limits his or her opportunity to
demonstrate these behaviors on theirown.
Is there a better method for evaluatingcomplex achievement
7/29/2019 measuring complex achiemvents
10/25
Extended-Response Essay
Questions Freedom of response allows student to
select information that they think is
pertinent, to organize the answer inaccordance with their best judgment,
and to integrate and evaluate ideas as
they deem appropriate
Places value on higher-order thinkingskills
7/29/2019 measuring complex achiemvents
11/25
Examples of Extended-
Response Essay Questions Imagine that you and a friend found a
magic wand. Write a story about an
adventure that you and your friend hadwith the magic wand.
Compare developments in international
relations in the administrations of
President William Clinton and PresidentGeorge W. Bush. Cite examples when
possible.
7/29/2019 measuring complex achiemvents
12/25
To Keep Things Simple
Objective Interpretive- select
Restricted Response Essay- supply
Extended-Response Essay- write
7/29/2019 measuring complex achiemvents
13/25
Pros & Cons of Essay
Questions Apply your knowledge
They give student the
options to chose what
they want to write about reasoning
The teacher can tell if the
student studies or not
Can often prepare in
advance what to write
Time consuming
Grading: grammar usage
vs. content
Legibility
Hard to formulate ideas
in time
Takes longer to grade
7/29/2019 measuring complex achiemvents
14/25
How Can We Change Those
Cons to Pros?Unreliability- clearly defining learning
outcomes to be measured, properly
framing questions, carefully followingscoring rules, and obtaining practice in
scoring
Amount of time- reserve use of
extended-response questions forlearning outcomes that cannot be
measured well objectively
7/29/2019 measuring complex achiemvents
15/25
Limited Sampling- try to obtain as
representative a sample of learning
outcomes as possible (accumulate a
series of essays throughout the schoolyear to be included in a writing portfolio)
7/29/2019 measuring complex achiemvents
16/25
Suggestions for Constructing
Essay QuestionsRestrict use of essays to learning
outcomes that cannot be measured well
objectivelyConstruct questions that call forth skills
specified in learning standards
Example box on page 235
7/29/2019 measuring complex achiemvents
17/25
Phrase the question so that the students
task is clearly indicated
Example:
Poor: Compare the Democratic andRepublican parties.
7/29/2019 measuring complex achiemvents
18/25
Better: Compare the current policies of
the Democratic and Republican partieswith regard to the role of government in
private business. Support your
statements with examples when
possible. (Your answer should be
confined to two pages. It will be
evaluated in terms of the
appropriateness of the facts andexamples presented and the skill with
which it is organized.)
7/29/2019 measuring complex achiemvents
19/25
Indicate an approximate time limit for
each question
Avoid the use of optional questions Example Box page 239
Now that we have essay questions how dowe score them?
7/29/2019 measuring complex achiemvents
20/25
Scoring Essay Questions
Tips to remember
Use clear specifications of scoring
criteria Inform students of scoring criteria
Use an initial review to find anchorresponses for comparison
Use descriptive rather than judgmentalscores or levels (writing is clear andthoughts are complete vs. excellent)
7/29/2019 measuring complex achiemvents
21/25
Scoring for Restricted
Response Essay Questions In most instances, the teacher should
write an example of an expected
response For example, if the student is asked to
describe three factors that contributed to
the start of the Civil War, the teacher would
construct a list of acceptable reasons andgive the student 1 point for each of up to
three reasons given from the list
7/29/2019 measuring complex achiemvents
22/25
Scoring for Extended-
Response Essay QuestionsAnalytic Scoring Rubrics
Consist of a rubric broken down into key
dimensions that will be evaluated Enables teacher to focus on one
characteristic of a response at a time
Provides maximum feedback for
students
7/29/2019 measuring complex achiemvents
23/25
Holistic Scoring Rubrics
Yield a single overall score taking into
account the entire responseCan be used to grade essays more
quickly
Does not provide as much specificfeedback as analytic rubric
Should not consist of scores alone, butrather contain scores accompanied bystatements of the characteristics of theresponse
Example Table 10.3 and 10.4
7/29/2019 measuring complex achiemvents
24/25
Suggestions for Scoring Essay
Questions Prepare an outline of the expected
answer in advance and use a clear
scoring rubricUse the scoring rubric that is most
appropriate
Decide how to handle factors that are
irrelevant to the learning outcomes beingmeasured
7/29/2019 measuring complex achiemvents
25/25
Evaluate all responses to one question
before going on to the next one
When possible, evaluate answers
without looking at the students name
If especially important decisions are to
be based on the results, obtain two ormore independent ratings
Look out for bluffing! Page 247