The Development and Use of Selected-Response

Items

Selected-response item An item that requires a student to select a

response from available alternatives. Examples of these are :Multiple choiceTrue – FalseMatching items

Constructed-response item

An item requires a student to create or construct a response

Multiple -choice ItemMultiple-choice items take the general form of a

question or an incomplete statement with a set of possible answers , one which is correct.Stem – the part of the item that is either a

question or an incomplete statementAlternatives – the possible answersAnswers – the correct alternativeDistracters – the incorrect alternatives

Direct-Question Format1 Which river is the largest in the United States of

America?A. Mississippi <

B. Missouri

C. Ohio

D. Rio Grande

Most writers prefer the direct- question format because they feel it presents the problem in a clearer manner.

Incomplete-Sentence Format

1 The largest river in the United States of America is the ________________.

A. Mississippi <

B. Missouri

C. Ohio

D. Rio Grande

The advantage of the incomplete –sentence format is that it may present the problem in a more concise manner.

Best Answer Format1 Which variable is generally thought to be the

most important when buying a house ?A. Cost

B. Builder

C. Design

D. Location

Guidelines for Developing Multiple-Choice Items

1 Use a Printed Format That Makes the Item as Clear as Possible. Few recommendations :Provide brief but clear directions. Directions should

include how the selected alternative should be marked.The item stem should be numbered for easy

identification, while the alternatives are indented and identified with letters.

Either capital or lower case letters followed by a period or parenthesis can be used for the alternatives. If a scoring sheet is used, make the alternative letters on the scoring sheet and the test as similar as possible.

There is no need to capitalize the beginning of alternatives unless they begin with a proper name.

When the item stem is a complete sentence, there should not be a period at the end of the alternatives ( see Example 4)

When the stem is in the form of an incomplete statement with the missing phrase at the end of the sentence, alternatives should end with a period . (see Example 5)

Keep the alternatives in a vertical list instead of placing them side by side because it is easier for students to scan a vertical list quickly.

Use correct grammar and formal language structure in writing items.

All items should be written so that the entire question appears on the page.

Example 4

Directions : read each question carefully and select the best answer . Circle the letter of the answer you have selected.

1 Which type of validity study involves a substantial time interval between the test is administered and when the criterion is measured?

A. delayed study

B. content study

C. factorial study

D. predictive study <

Example 5

2. The type of validity study that involves a substantial time interval between when the test is administered and when the criterion is measured is a ____________________.

A. delayed study

B. content study

C. factorial study

D. predictive study <

2 Have the Item Stem Contain all the Information Necessary to Understand the Problem or Question

Example 6 Poor Item – Inadequate Stem

3 Absolute zero point.A. interval scale

B. nominal scale

C. ordinal scale

D. ratio scale <

Example 7 Better Item – Adequate Stem

1 Which scale of measurement incorporates a true or absolute zero point?

A. interval scale

B. nominal scale

C. ordinal scale

D. ratio scale <

Example 8 Poor Item – Unnecessary content

1 There are several different scales of measurement used in educational settings. Which scale of measurement incorporates a true or absolute zero point?

A. interval scale

B. nominal scale

C. ordinal scale

D. ratio scale <

3 Provide between Three and Five Alternatives

Certain research even suggests that items with three alternatives can be as effective as items with four or five alternatives ( e.g. Costin, 1970; Grier, 1975; Sidick, Barrett, & Doverspike, 1994)

4 Keep the Alternative Brief and Arrange Them in an Order That Promotes Efficient Scanning.

Example 9 Poor Item – Inadequate Stem and Lengthy Alternatives

1. Andrew Jackson _____________________________A. was born in Virginia

B. did not fight in the American Revolution due to a childhood illness.

C. was the 7th president of the United States <

D. served three terms as president of the United States

Example 10 Better Item – Adequate Stem and Brief Alternatives.

2. Who was the 7th president of the United States of America?A. Andrew Jackson <

B. James Monroe

C. John Adams

D. Martin Van Buren

Example 11 Poor Item – Illogical Arrangement of Alternatives

1 What year did the Spanish-American War occur ?

A. 1912

B. 1890

C. 1908

D. 1898 <

E. 1902

Example 12 Better Item – Logical Arrangement of Alternatives

2 What year did the Spanish-American War occur ?

A. 1890

B. 1898 <

C. 1902

D. 1908

E. 1912

5 Avoid Negatively Stated stems in Most Situations

Example 13 Poor Item – Negatively Stated Stem

6 Which state does not have a coastline on the Gulf of Mexico ?

A. Alabama

B. Florida

C. Tennessee <

D. Texas

Example 14 Better Item – Negative Term Highlighted

2 Which state does NOT have a coastline on the Gulf of Mexico ?

A. Alabama

B. Florida

C. Tennessee <

D. Texas

6 Make sure Only One Alternative Is Correct or Represents the Best Answer.

Example 15 Poor Item – Multiple-Choice Item1 Which states have a coastline on the gulf of Mexico

A. Alabama <

B. Florida <

C. Tennessee

D. Texas <

Example 16 Better Item – Multiple True-False Item

2. Which of the following states have a coastline on the Gulf of Mexico? Underscore the T if the state has a Gulf coastline or the F if the state does not have a Gulf coastline

Alabama T F

Florida T F

Tennessee T F

Texas T F

6. Avoid Cues That Inadvertently Identify the Correct Answer

Example 17 Poor Item – Stem Contains1 Why type of validity study examines the ability of

test scores to predict a criterion?A. interval study

B. content study

C. factorial study

D. predictive study

Example 18 Better Item – Cues Avoided

2. Which type of validity study involves a substantial time interval between when the test is administered and when the criterion is measured?A. interval study

B. content study

C. factorial study

D. predictive study <

Example 19 Poor Item – Unequal Length and Complexity of Alternatives

1. Ecology is the study of ______________A. genesis

B. organisms and their relationships to the environment <

C. internal balances

D. evolution

Example 20 Better Item – Alternatives similar in Length and Complexity

2. Ecology is the study of ______________A. The genetic and molecular basis of organisms

B. Organisms and their relationship to the environment <

C. How organisms maintain their delicate internal balance

D. How organisms have slowly evolved over the last million years

Example 21 Poor Item – Alternative Contains a Visual Cue to the Correct Answer

Example 21 – Poor Item – Alternative Contains

1. The correlation between two measures is 0.90. What is the coefficient of determination?

A. 0.1

B. 0.3

C. 0.81 <

D. 0.9

Example 22 Better Item – Cues Avoided

1. The correlation between two measures is 0.90. What is the coefficient of determination?

A. 0.10

B. 0.30

C. 0.81 <

D. 0.99

7 Make Sure All Alternatives Are Grammatically Correct Relative to the Stem.

Example 23 Poor Item – Grammatical Cue Present

1. Which individuals are credited with making the first successful flights in a heavier-than-air aircraft that was both powered and controlled ?

A. Octave Chanute

B. Otto Lilienthal

C. Samuel Langley

D. Wilbur and Orville Wright <

Example 24 – Better Item – Grammatical Cue Avoided

2. Which individuals are credited with making the first successful flights in a heavier than-air aircraft that was both powered and controlled?

A. Octave Chanute and Sir George Cayley

B. Otto Lilienthal and Francis Herbert Wenham

C. Samuel Langley and Alphonse Penaud

D. Wilbur and Orville Wright

Another common error is inattention to the articles a and an . See the following.

Example 25 Poor Item – Grammatical Cue Present

1. A coherent and unifying explanation for a class of phenomenon is a _______________

A. analysis

B. experiment

C. observation

D. theory <

Example 26 Poor Item – Grammatical Cue avoided

2. A coherent and unifying explanation for a class of phenomenon is a _______________

A. conjecture.

B. hyphotesis.

C. prediction.

D. theory <

Example 27 Poor Item – Grammatical Cue avoided

2. A coherent and unifying explanation for a class of phenomenon is a(n) _______________

A. experiment.

B. hyphotesis.

C. observation.

D. theory <

8. Make Sure No Item Reveals the Answer to Another Item.

9. Have All Distracters Appear Plausible.

10.Use Alternative Positions in a Random Manner for the Correct Answer.

11.Minimize the Use of “None of the Above” and Avoid Using “ All of the Above”.

12.Avoid Artificially Inflating the Reading level.

13.Limit the Use of Always and Never in the Alternatives.

14.Avoid Using the Exact Phrasing from the Text.

15.Organize the Test in a Logical Manner.

16.Give Careful Consideration to the Number of Items on Your Test.

17.Be Flexible When Applying These Guidelines.

Checklist for the Development of Multiple-

Choice Items1. Are the items clear and easy to read?

2. Does the item clearly state the problem or question?

3. Are there between three and five alternatives?

4. Are the alternatives brief and arranged in an order that promotes efficient scanning?

5. Have you avoided negatively stated items?

6. Is there only one alternative that is correct or represents the best answer?

7. Have you checked for cues that accidentally identify the correct answer?

8. Are all alternatives grammatically correct relative to the stem?

9. Have you checked to make sure no item reveals the answer to another item?

10.Do all distracters appear plausible?

11.Did you use alternative positions in a random manner for the correct answer?

12.Did you minimize the use of “none of the above” and avoid using “all of the above”?

13.Is the reading level appropriate?

14.Did you limit the use of always and never in the alternatives?

15.Did you avoid using the exact phrasing form the text?

16.Is the test organized in a logical manner?

17.Can the test be completed in the allotted time period?

Strengths of Multiple-Choice Items

Multiple-Choice Items Are Versatile

Multiple-Choice Items Can Be Scored in an Objective Manner

Multiple-Choice Items Are Not Unduly Subject to Guessing

Multiple-Choice Items Are Not Significantly Influenced by Response Sets.

Multiple-Choice Items Are an Efficient Way of Sampling the Content Domain.

Multiple-Choice Items Provide Information about the Type of Errors That Students Are Making

Weaknesses of Multiple-Choice Items

Multiple-choice items are not effective for measuring all educational objectives.

Multiple-choice items are not easy to write.

Corrections for GuessingCorrected Score = Right – Wrong / (n-1)

Where

right = number of items answered correctly

Wrong = number of items answered incorrectly

n = number of alternatives or potential answers

For true-false items it is simply calculated as :

Corrected Score = Right − Wrong

True – False ItemsIt involve a statement or question that the

student marks as true or false , agree or disagree, yes or no and so on.

Example 29 True – False with Directions

Directions : Carefully read each of the following statements . If the statement is true, underscore the T. If the statement is false, underscore the F.

1. T F In recent years, malaria has been eliminated worldwide.

2. T F The ozone layer protects us from harmful ultraviolet radiation.

Example 30 Correct – Incorrect Item with Directions

Directions : Carefully read each of the following sentences. If the sentence is grammatically correct , underscore C. If the sentence contains a grammatical error , underscore I for the incorrect.

1. C I He set the book on the table.

2. C I She set on the couch.