Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | hamish-potter |
View: | 34 times |
Download: | 0 times |
Improving your classroom assessments
Dr. Patty LeBlanc
Southeastern University
Agenda
IntroductionsObjectives of the SeminarPurpose of Educational AssessmentsEducator and Educational StandardsBloomTest Blueprints Item Writing Commandments
Item Critique Activity Item AnalysisExample from grad studentsAuthentic AssessmentRubricsSummaryAction PlanQuestions?
Objective of the Seminar
To help teachers design valid assessments of instruction that inform instruction
What is the purpose of testing?
Participant Ideas
To encourage students and improve teaching
To assess the extent to which students have met the intended learning outcomes, and to look for and evaluate unintended outcomes
To recognize achievements and diagnose learning difficulties so that students learn to build on their strengths and overcome or cope with their weaknesses
To refine instruction and learning experiences in order to improve both individual and class learning
To help students set meaningful and realistic learning goals and assume responsibility for their own learning.
To communicate meaningful information to students, parents, and school authorities about student learning
To give realistic and helpful feedback about achievement, capabilities, behavior, attitudes, and dispositions
To put teachers, students, and parents/guardians in touch with one another about progress over time
To provide guidance for educational and vocational choices
To report learning achievement to school authorities such as school boards and governments
Florida Educator Accomplished Practices in Assessment
The effective educator consistently:Analyzes and applies data from multiple assessments and measures to diagnose students’ learning needs, informs instruction based on those needs, and drives the learning process;Designs and aligns formative and summative assessments that match learning objectives and lead to mastery;Uses a variety of assessment tools to monitor student progress, achievement and learning gains;Modifies assessments and testing conditions to accommodate learning styles and varying levels of knowledge;Shares the importance and outcomes of student assessment data with the student and the student’s parent/caregiver(s); andApplies technology to organize and integrate assessment information.
Educational Standards
Florida Sunshine State StandardsCommon Core StandardsOther
“Backward” Design
The standards dictate the lesson objectives…
The objectives dictate the instructionThe assessment must align with the
objectives, standards, and instruction in order to be valid.
Blessed Assurance…
A fantastic way to ensure that your test items align with your objectives: A Test Blueprint!
Content Validity Think matrix of instruction and assessment:
Objective Item 1
Item 2
Item 3
Item 4
Item 5
1 X X X
2 X
3
4 X X
Bloom’s Taxonomy of the Cognitive Domain
Another blueprint…
Item # Standard/Objective Knowledge/Comp Above Knowledge/Comp
Comment
Sung to the tune of the Farmer in the Dell
The teacher follows the standards…The teacher creates the objectives…The teacher writes the test…The teacher teaches the lesson…The students learn a lot…The parents beam with pride…
The Commandments of Test Item WritingItem Writing Item Writing
CommandmeCommandmentsnts
Comunicación y Gerencia
By W.James PophamBy W.James Popham
Click to add TextClick to add Text
General Item Writing Items
Thou shalt not
provide opaque directions to students regarding how to respond to your assessment instruments.
Thou shalt not
employ ambiguous or wordy statements in your assessment items.
The term hypothesis, as used in research, as defined as:a. A conception or proposition formed by speculation or deduction or by abstraction and
generalization from facts, explaining or relating an observed set of facts, given
probability by experimental evidence or by factual or conceptual analysis but not
conclusively established or accepted.
b. A statement of an order or relation of phenomena that so far as is known is invariable
under the given conditions, formulated on the basis of conclusive evidence or tests
and universally accepted, that has been tested and proven to conform to facts.
*c. A proposition tentatively assumed in order to draw out its logical or empirical
consequences and so test its accord with facts that are known or may be determined,
of such a nature as to be either proved or disproved by comparison with observed or know facts.
Better…
The term hypothesis, as used in research, is defined as:
a. An assertion explaining an observed set of facts that has not been conclusively
established.
b. A universally accepted assertion explaining an observed set of facts.
*c. A tentative assertion that is either proved or disproved by comparison with an observed set of facts.
Thou shalt not
Provide unintentional clues as to the correct response.
What meter is used to measure resistance?a. Voltmeterb. Ohmmeterc. Ammeterd. Oscilloscope
Thou shalt not
employ complex or give-away syntax in your assessment items.
In item response theory, the one-parameter model assumes that each item …….a. discriminates equally well.b. students perform equally well.c. students score the same across items.d. guessing affects all items the same.
Thou shalt not
use vocabulary that is more advanced than required.
As the level of fertility approaches its nadir, what is the most likely ramification for the citizenry of a developing nation?
a decrease in the labor force participation rate of women a dispersing effect on population concentration a downward trend in the youth dependency ratio a broader base in the population pyramid an increased infant mortality rate
Use plausible distractors
Which of the following artists is known for painting the ceiling of the Sistine Chapel?
a. Warhol
b. Flintstone
*c. Michelangelo
d. Santa Claus
Thou shalt
Thou Shalt
Avoid using "always" and "never" in the stem or the distractors.
Thou shalt
Rarely us negative statements…
Which of the following is NOT a symptom of osteoporosis? decreased bone density frequent bone fractures raised body temperature lower back pain
Binary Choice Items [True/False]
One of several types of Selected Response Items
Thou shalt not
Phrase items so that a superficial analysis by the student suggests a wrong answer.
Which of the following artists is known for painting the ceiling of the Sistine Chapel?
a. Warhol
b. Flintstone
*c. Michelangelo
d. Santa Claus
Thou shalt
NEVER use double negatives.
True or False: There is no advantage to not using specific determiners in true-false items.
Thou shalt
Have an approximately equal number of items representing the two categories being tested.
Keep item length similar for both categories being tested.
Thou Shalt
Stick with absolute truths or falsehoods.
True or False: Nearsightedness is hereditary in origin.
Thou Shalt Not
Use extreme qualifiers.
All sessions of Congress are called by the President. (F)
The Supreme Court frequently rules on the constitutionality of law. (T)
An objective test is generally easier to score than an essay test. (T)
Multiple Choice Items
Thou shalt
Write the stem so that it consists of a self-contained question or problem.
California:a. Contains the tallest mountain in the United States
b. Has an eagle on its state flag.
c. Is the second largest state in terms of area.
*d. Was the location of the Gold Rush of 1849.
Better…
What is the main reason so many people moved to California in 1849?
a. California land was fertile, plentiful, and inexpensive.
*b. Gold was discovered in central California
c. The east was preparing for a civil war.
d. They wanted to establish religious settlements.
Thou shalt
Include only one concept in each statement and only one correct answer.
The United States should adopt a foreign policy based on:a. A strong army and control of the North American continent.
b. Achieving the best interest of all nations.
c. Isolation from international affairs.
*d. Naval supremacy and undisputed control of the world’s sea lanes.
Thou shalt
NEVER use ‘all of the above’ alternatives, only occasionally use ‘none of the above’ alternatives [to increase item difficulty].
Alternative Weakness
“All of the above” Distractor can be eliminated by noting that one of the other alternatives is incorrect
“None of the above”
Answer measures the ability to recognize incorrect answers rather than correct answers [remember educational psychology 101?].Distractor does not appear plausible to some students.
Well-designed MC items CAN measure HOTS
A common goal of the Salt March in India, the Boxer Rebellion in China, and the Zulu resistance in southern Africa was to:
A. overthrow totalitarian leaders
B. force upper classes to carry out land reform
programs
C. remove foreign powers
D. establish Communist parties to lead the countries
Numbers of Items is Important!
Number of 4-Alternative MC Items
2 5 10 15 20 25
Chance of Scoring 70% or Higher by Blind Guessing
• 1 out of 16• 1 out of 64• 1 out of 285• 1 out of 8,670• 1 out of 33,885• 1 out of 942,651
Pop Quiz
Can you identify the problem with this item?
Due to budget cutbacks, the university library now subscribes to fewer than _?_ periodicals. 25,000 20,000 15,000 10,000
Matching Items
Most effective when you need to measure the learner’s ability to identify the relationship or association between similar items. They work best when the course content has many parallel concepts, for example:Terms and DefinitionsObjects or Pictures and LabelsSymbols and Proper NamesCauses and EffectsScenarios and ResponsesPrinciples and Scenarios to which they apply
Thou shalt
Use homogeneous lists.
Hitler A. Year in which WWII began
Mulroney B. A Canadian Prime Minister
Tank C. A German dictator during WWII
1939 D. An armored vehicle used originally to break the trench
war stalement in WWI
Thou shalt
Randomly assign correct answers to alternative positions.
Logical order of responses
Example
Numerical a. 1939b. 1940c. 1941d. 1942
Alphabetical a. Changing a from .01 to .05.b. Decreasing the degrees of freedom.c. Increasing the spread of the exam scores.d. Reducing the size of the treatment effect.
Sequential a. Heating ice from -100°C to 0°C.b. Melting ice at 0°C.c. Heating water from 0°C to 100°C.d. Evaporating water at 100°C.e. Heating steam from 100°C to 200°C.
Thou shalt not
let the length of alternatives supply unintended clues.
Which of the following is the best indication of high morale in a supervisor’s unit?
a. The employees are rarely required to work overtime.
*b. The employees are willing to give first priority to attaining group objectives,
subordinating any personal desires they may have.
c. The supervisor enjoys staying late to plan the next day.
d. The unit gives expensive birthday presents to each other.
Thou shalt
Use relatively brief lists, placing the shorter words or phrases at the right.
Thou shalt
Employ more responses than premises.
Thou Shalt
Provide good directions!
Directions: Match the following.
____1. City dwellers A. Wild animals
____2. Hunter-gatherers B. Farm
____3. Pastoral nomads C. Apartments
D. Graze animals
Better…
Directions: On the line to the left of each definition, write the letter of the term in the right hand column that is defined. Use each term only once. There are more responses than items. Each correct response is worth 2 points.
_1. Live in areas of high population density. A. Pastoral nomads
__2. Move from place to place in search of wild animals. B. Ranchers
__3. Move from one place to another with grazing animals. C. Hunter-gatherers
__4. Till land for cash crops. D. City dwellers
E. Farmers
Thou shalt
Describe the basis for matching and the number of times responses may be used in the instructions, as well as the point values for each item.
Place all premises and responses for an item on a single page.
Tip: Be sure to decide ahead of time whether you will give partial credit.
Matching HOTS item
Directions: Number (1-8) the following events in the history of ancient Egypt in the order in which they occurred, using 1 for the earliest event. _____Egypt divided; ruled by Libyan kings, Nubian pharaohs,
Assyrians, and Persians _____Seizure of power by Hyksos kings _____Upper and Lower Egypt are united by Menes _____Alexander the Great conquers Egypt _____Reunification of Egypt under pharaoh Mentuhotop II _____Rise of feudal lords leads to anarchy _____Thutmose III expands empire to the Euphrates _____Many kings with short reigns; social and political chaos
Short Answer/Essay Items
Aka Constructed Response Items
Short Answer/Fill in the Blanks
Thou Shalt
Ask a direct question.
Undesirable:Alloys are ordinarily produced by…
Desirable:How are alloys ordinarily produced?
Thou Shalt
Put a blank at the end of a sentence rather than the beginning.
Undesirable:
____________ is the measure of central tendency that is most affected by extremely high or low scores.
Desirable: The measure of central tendency that is most affected by
extremely high or low scores is the ____________.
Thou Shalt
Omit only significant words.
Undesirable:
Every atom has a central ____________ called a nucleus.
Desirable:
Every atom has a central core called a(n) ____________.
Thou Shalt Not
Omit so many words that the meaning in unclear.
Undesirable:
The ____________ were to Egypt as the ____________ were to
Persia and as ____________ were to the early tribes of Israel.
Desirable:
The Pharaohs were to Egypt as the ____________ were to Persia and as ____________ were to the early tribes of Israel.
Thou Shalt Not
Use essay items to measure trivial ideas
Thou Shalt
Give very clear directions. Know the criteria for grading and use a rubric. Make expectations clear—have a good idea of what
the teacher expects in a good response Not provide options… Indicate point values and suggested time limit per item Take the test yourself and see how you do… Ask a colleague to take the test and critique it. Consider using several short essays vs 1 long one. Analyze student responses to improve the item in
future.
How do I decide???
Another Pop Quiz
Objective or Essay?
Rubrics for Essay Items
How can a classroom teacher improve his/her tests?
Before the test: Table of specifications/matrix Pilot testing, especially essay items and rubrics Peer review
During the test: Observation Timing Ask the students to rate the test/items
After the test: Item Analysis Go over the test with students Item and Test Revision Re-examine instruction and re-teach if necessary
That’s a lot of trouble!
Yes, but it is the right thing to do.It’s intellectually honest.It makes grades easier to defend.
Test Improvement Project
By Christina Salas and Amanda Williams
Authentic Assessment
Aka performance based assessment
Great example: Edutopia
Authentic Assessment vs. Traditional Assessment
Traditional --------------------------------------------- Authentic
Selecting a Response ------------------------------------ Performing a Task
Contrived --------------------------------------------------------------- Real-life
Recall/Recognition ------------------------------- Construction/Application
Teacher-structured ------------------------------------- Student-structured
Indirect Evidence -------------------------------------------- Direct Evidence
How to Measure Authentic Assessments
RubricsCooperativelyPanel of Experts
Steps in Rubric Development
1. Determine the learning outcomes.
2. Determine the criteria
3. Determine measurable criteria [4,3,2,1]
4. Develop a matrix
5. Evaluate overall point value relative to the task.
Grading cooperative work…
Participation in Class Discussions
Sit down with a seating chart and mark contributions to the discussion with a rating systemFor example, 5 being outstanding and 1
being poor, 0 being absent or silent
Grading cooperative work
Group gradeRecognize that there will always be
loafers, workers, and compulsive worker beesSee article on grading cooperative projects
Assessment should…
Encourage and improve student learning Be a blessing for students that embraces, grace, justice, and
encouragement Be an integral part of learning, closely related to overall learning
goals Foster excellence in all a students’ gifts on the basis of the
student’s potential as a bearer of God’s image Emphasize assessment FOR learning as well as assessment OF
learning Develop plans that enable teachers and students to interact
frequently about students’ progress as well as challenges, with opportunities to refine or revise
Use a variety of meaningful tasks, products, and performances, with descriptive feedback to students
Use accurate descriptors for all grades based on the achievement of learning outcomes
Communicate meaningful and clear information to students, parents, and authorities.
Other suggestions?
Questions?
Resources
Van Brummelen, H. 2009. Walking with God in the Classroom, 3rd edition. Purposeful Design Publicatons.
Brookhart, S. 2008. Feedback that Fits. Educational Leadership 65, no.4:54-59.
Stiggins, R. 1997. Student-centered classroom assessment. 2nd edition. Merrill.
More Resources
How to prepare effective essay questions http://www.uwgb.edu/oira/teachLearn/betterTests/betteressays.pdf
Steven J. Burton, et al, How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty, (http://testing.byu.edu/info/handbooks/betteritems.pdf).
Still More Resources
A Short Guide to Writing Effective Test Questions
Rubric Maker Quiz Star
Thank You!!!
To download this presentation:
http://leblancspace.wikispaces.com/home