Assessment Items

transcript

DEVELOPING HIGH QUALITY

ASSESSMENT ITEMS

Region 7 Digital Learning Academy

Dr. LaTricia Townsend

Research Scholar

Educational Evaluation

NCSU Friday Institute

@tricia_townsend

HTTP://REGION7DLA.WIKISPACES.COM/

LOOKING BACK

o Activity-Table Share: Use of Session

1 Information Reflection

How have you used information

gained from Session 1 in your

practice?

How have you shared the

information from Session 1 in your

school or district?

LOOKING FORWARD

Take the poll about how you obtain

assessment items

ITEM DEVELOPMENT PROCESS

Curriculum Standards

ALIGNMENT

Curriculum Standards

Depth and Type of Thinking

ACCESS CURRICULAR DOCUMENTS

Activity- Take a moment to locate curriculum standards materials

Common Core ELA and Math

Essential StandardsNext Generation Science StandardsUnpacking documents completed by

school/districtCurriculum Maps/Pacing GuidesBegin to think about which ones you want

to address today

REVISED BLOOM’S TAXONOMY-KNOWLEDGE

PROCESS DIMENSION

FACTUAL KNOWLEDGE-basic elements students must know to be acquainted with a discipline or solve problems in it

CONCEPTUAL KNOWLEDGE-The interrelationships among the basic elements within a larger structure that enable them to function together

PROCEDURAL KNOWLEDGE-How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods

METACOGNITIVE KNOWLEDGE-Knowledge of cognition in general as well as awareness and knowledge of one's own cognition

REVISED BLOOM’S TAXONOMY-COGNITIVE

PROCESS DIMENSION

Revised Bloom’s Taxonomy According to Homer

Simpson

WEBB’S DEPTH OF KNOWLEDGE

Level 1: Recall

Recall a fact, information, or procedure

Level 2: Skill/Concept

Use information or conceptual knowledge, two or

more steps

Level 3: Strategic Thinking

Requires reasoning, developing a plan or a

sequence of steps, some complexity

Level 4: Extended Thinking

Requires an investigation, time to think and

process multiple conditions of the problem.

PRACTICE ALIGNMENT

o Activity-Write items (open ended- that align

to your content area and Revised Bloom’s

Taxonomy/Webb’s Depth of Knowledge)

• Write four questions to address each of

the four levels.

• Record your responses using the tool of

your choice for your own records.

• Share your responses on the Google

Spreadsheet

PRACTICE ALIGNMENT

o Activity-Find a partner who teaches/supports in the same grade span in a different district. Share your questions and discuss how you might approach the content differently.

o Reflect #R7DLA

MULTIPLE CHOICE ITEM GUIDELINES

Content General

Format Structure

• Stem

• Foil

Accessibility

• SWD and LEP

Bias/Sensitivity

MULTIPLE CHOICE TERMINOLOGY

Which is the state bird of North

Carolina?

A. bluebird

B. cardinal

C. robin

D. sparrow

Distractor

Passage

http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g3elapp.pdf

Stimulus

http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g5mathpp.pdf

MULTIPLE CHOICE GUIDELINES-CONTENT

Base items on curriculum standards and

Keep the testing vocabulary consistent

with grade level.

Measure a single

concept, principle, procedure, or

competency.

Write items with a mixture of difficulty

levels-Easy, Medium, Hard

MULTIPLE CHOICE GUIDELINES-CONTENT

Choose to assess important information, not trivial material.

POOR ITEM

How many different elements make up the Noble Gases?

IMPROVED ITEM

What common property is shared by all the noble gases? A. They are highly reactive and must be stored in air-tight containers.

B. They are inert, and do not easily form chemical compounds.

C. They are metallic in nature, and have basic oxides.

D. When mixed with water, they form a toxic blue-green solution.

MULTIPLE CHOICE GUIDELINES-GENERAL

Test the content, not the test-taking or

reading ability of students-avoid tricky

or misleading items.

Avoid errors of

grammar, abbreviations, punctuation

and spelling.

Do not use contractions.

Do not use first or second person.

Do not use humor.

MULTIPLE CHOICE GUIDELINES-FORMAT

Write clear, concise instructions.

Format foils vertically.

Use space in a balanced manner to

produce an easy to follow format

MULTIPLE CHOICE GUIDELINES-STRUCTURE

Avoid writing complex format multiple-choice

items.

Which of the following are fruits?

1. Tomatoes

2. Tomatillos

3. Habanero peppers

A. 1 & 2

B. 2 & 3

C. 1 & 3

D. 1, 2, & 3

MULTIPLE CHOICE GUIDELINES-STRUCTURE

Do not use fill-in-the blank format. Pose a question.

Be brief . Wordiness in the stem can clue or confuse the student. (“window-dressing”)

POOR ITEM

High temperatures and heavy rainfall characterize a humid climate. People who live in this

kind of climate usually complain of problems dealing with heavy amounts of perspiration.

Even moderately warm days are uncomfortable for most people in this climate. Which

biome is being described?

A. savanna

B. taiga

C. tundra

D. tropical rainforest

IMPROVED ITEM

Which term describes a biome with high temperatures and heavy rainfall?

A. savanna

B. taiga

C. tundra

D. tropical rainforest

MULTIPLE CHOICE GUIDELINES-STEM

Word the stems positively. Do not use any negative phrasing, such as words NOT or EXCEPT.

Do not write stems that ask for or express opinions.

Write the items so that the central idea and any common elements are included in the stem instead of the foils.

Place the interrogative as close to the foils as possible.

Use qualifying words in a stem, when needed, to emphasize the “best answer.” Words such as best, most likely, and most appropriate are bold and italicized.

MULTIPLE CHOICE GUIDELINES- FOIL

DEVELOPMENT

Each item must contain four foils (A,B,C,D)

Order the answer choices in a logical

order, such as ascending or descending value

for numbers, the order in which the words

appear in a chart, chronological order, etc.

Each item written should contain foils that are

independent and not overlapping.

To the extent possible, foils for an item should

be homogeneous in content and length.

DEVELOPMENT

Do not use the following as foils: all of the above, none of the above, I don’t know.

Word the foils positively, avoid any negative phrasing.

Avoid providing clues to the correct response.

Avoid writing items in which phrases in the stem are repeated in the foils.

DEVELOPMENT

Avoid absolutes such as ALWAYS, NEVER, TOTALLY, and ABSOLUTELY should not be used when writing items.

Qualifiers such as best, most likely, approximately, etc. should be bold and italic.

Present one and only one best (correct) answer.

Use the common errors made by students as distractors.

DEVELOPMENT

Use plausible distractors. The best (correct) answer must clearly be the best (correct) answer and the incorrect responses must clearly be inferior to the best (correct) answer. No distractor should be obviously wrong.POOR ITEM

Which example best describes the adhesion

property of water?

A. drops of water on the outside of a glass

B. sugar dissolving in water

C. ice floating in water

D. melting of aluminum

DEVELOPMENT

Use plausible distractors.IMPROVED ITEM

Which example best describes the adhesion

property of water?

A. drops of water on the outside of a glass

B. sugar dissolving in water

C. ice floating in water

D. water changing from a liquid to a gas

MULTIPLE CHOICE GUIDELINES-UNIVERSAL

DESIGN

Make items accessible and usable by students

of all abilities, including students with

disabilities and students with limited English

proficiency.

Use simple language and sentence structure.

Do not use colloquial language.

MULTIPLE CHOICE GUIDELINES-UNIVERSAL

DESIGN

Limit the use of homonyms, homophones, and homographs

fair1. Adjective. Meaning lovely or beautiful, when describing someone

2. Adjective. Meaning of light complexion, or light colored (blonde) hair

3. Adjective. Meaning reasonable, equitable, free from self-interest

3. Noun. Meaning a gathering of people to sell and barter goods, or to enjoy entertainment, and also a travelling show of rides, stalls and sideshows

SENSITIVITY AND BIAS

Do not use stereotypes based on

gender, race, religion, socio-economic

status, or other factors.

Respect diversity by including people from

all walks of life in use of names and artwork.

Ensure items do not highlight socio-

economic disparities.

Avoid espousing religious and political

beliefs.

REVISING AND WRITING QUESTIONS

o Activity-- Would you change any of the

questions that you posed, initially? Make

any necessary changes.

o Create 3 Multiple Choice Questions-

Record the questions using the same

tool you used before for your personal

use and share your responses on the

Google Spreadsheet

o Reflect #R7DLA

ASSESSING ITEMS

o Activity- Review Questions o Find a trioo Use the item review document to gauge

the quality of each of the itemso What changes would you make to the

items?

Reflect #R7DLA

GOT QUESTIONS…

Source:http://en.clipart-fr.com

DROP THEM ANYWHERE!

SCHOOL NET

o Activity- Tasks in School Net

Access the User Guide

o Log into School Net

o Try these:

o Create an Item in School Net

o Create an Express Test

o Create a Test using items from Item

Central

Reflect #R7DLA

WORK TASKS

Explore online resources

Brainstorm/Develop Culminating Resource

Reflect #R7DLA

ONLINE RESOURCES/SCHOOL NET

Activity- Explore the resources presented. How

could these be used in your instructional delivery or

assessment program? Are there other resources

you would like to share? Share your ideas to

Padlet.

FINAL PRODUCT IDEA BRAINSTORM/WORK

SESSION

Activity- Table Talk-With your team, discuss what

your product for School Net will be at the

culmination of DLA. What is the proposed product?

How do you anticipate incorporating future

information? How will it be disseminated in your

school/district?

Develop Concrete Next Steps

Share your ideas to wiki Padlet.

HTTP://REGION7DLA.WIKISPACES.COM/