Post on 13-Dec-2015
transcript
What Do We Ask Ourselves Before Building an
Assessment Instrument?
• What do we want to measure?• What information do we want to
gather?• How long do we want the assessment to
take?• How will we determine which standards
we want to assess?• How will we use the results?
Guiding Principles for Building Assessment
Instruments• A minimum of three items for every
standard you want to measure.• Spread items measuring an individual
standard across the assessment.• Range of overall assessment difficulty and
for each standard measured.• Begin assessment with easier items, build
to more difficult and conclude with easier items.
Guiding Principles for Building Assessment
Instruments
• Spread items with graphics across test.• Readability levels for reading passages; genres
needed.• A minimum of five items per passage.• Even spread of answer choices across
assessment when possible; spread them out across the form.
• Check items for appropriateness to use online, paper, or both.
Use a blueprint to build validitySource: Ernie Bauer and Ed Roeber, “Technical Standards for
Locally-Developed Assessments”, April 8, 2010
• A grid is used to summarizes the content and format of the test.– The rows are the learning objectives– The columns are the level of cognitive complexity
• The cells list the types and numbers of items.• You can place a margin that can be used to
sum the total points.
Test Blueprint Depth of Knowledge Recall Skills/Concepts Strategic Extended Target Totals
# Points # Points # Points # Points # Points
Learnin
g Targets
DOK Totals
Excellent Examples
Webb (1997) Depth of Knowledge • Recall • Skills and Concepts• Strategic Thinking• Extended Thinking
Anderson, & Krathwohl (2001)
Revised Bloom’s Taxonomy
Depth of Knowledge
• Recall includes facts, definitions, terms, or a simple procedure,– “identify”, “recall” ,“recognize”
• Skills and Concepts includes some processing beyond a habitual response – “classify”, “organize”, “compare”
• Strategic requires reasoning, planning, or using evidence– “justify” ,“draw conclusion”, “explain”
• Extended Thinking requires complex reasoning, planning, developing, and thinking. – “evaluate”, “design”, “generalize”
Reverse DesignSource: Ernie Bauer, and Jim Gullen, “Locally Developed Assessment:
What Do the Results Mean?”, April 2010
• Start with your items• Group the items by content (GLCE/HSCE)– This will give you the rows for the blueprint
• Split the items in the content piles by DOK– This will give you the columns for the blueprint
• Create a blueprint document to check if this provides acceptable evidence for you.
Time Elements of an Assessment CalendarSource: White, S. H. (2005). “Beyond the Numbers: Making Data Work for Teachers and School
Leaders”. Lead and Learn Press: Englewood, CO
• When will we administer the assessment?• When will we collect the data?• When will we disaggregate the data?• When will we analyze the data?• When will we reflect upon the data?• When will we make recommendations?• When will we make the decisions about the
recommendations?• When will we provide written documentation about the
decisions?• When will we share the data with other stakeholders?
“You Be George” A Formative Assessment Activity
• Imagine that you have just received some feedback on a recent science assessment.
• Using the information from your teacher,– Identify which learning targets you got
right– Identify which learning targets you
need to review– Identify which learning targets you
need to keep practicing
Questions?Stan MastersCoordinator, Instructional Data ServicesLenawee Intermediate School District4107 N. Adrian HighwayAdrian, Michigan 49921
517-265-1606 (phone)517-265-2953 (fax)stan.masters@lisd.us
Mike HusbandData Entry SpecialistLenawee Intermediate School District4107 N. Adrian HighwayAdrian, Michigan 49921
517-266-6981 (phone)517-265-7405 (fax)mike.husband@lisd.us
Data Warehouse webpage:www.lisd.us/links/data