of 23
8/9/2019 Writing Test Items
1/23
Writing Test Items
Introduction
Essential Characteristics of Item Writers
o Knowledge and Understanding of the Material Being Tested
o Continuous Awareness of Objectives
o Continuous Awareness of Instructional Model
o Understanding of the Students for Who the Ites are Intended
o S!ill in Written Counication
o S!ill in Techni"ues of Ite Writing
General Tips
o #$%ress Ites as &recisel'( Clearl' and Si%l' as &ossible
o Include all )ualifications *ecessar' to &rovide a +easonable Basis for
+es%onding
o #%hasi,e -eneral Tas!s +ather than Sall .etails
o Avoid /argon and Te$tboo! 0anguage
o 0ocate and .elete Irrelevant Clues
o #liinate Irrelevant Sources of .ifficult'
o &lace all Ites of a -iven T'%e Together in the Test
o &re%are Ke's or Model Answers in Advance of Test Adinistration
o Arrange for Co%etent +eview of the Ites
Writing Specific Types of Items
o Multi%le1Choice Ites
State the &roble in the Ste
Include One Correct or Most .efensible Answer
1
https://www.msu.edu/dept/soweb/writitem.html#introhttps://www.msu.edu/dept/soweb/writitem.html#esscharhttps://www.msu.edu/dept/soweb/writitem.html#knowmathttps://www.msu.edu/dept/soweb/writitem.html#awareobjhttps://www.msu.edu/dept/soweb/writitem.html#awaremodhttps://www.msu.edu/dept/soweb/writitem.html#studentshttps://www.msu.edu/dept/soweb/writitem.html#wrtskillhttps://www.msu.edu/dept/soweb/writitem.html#technicshttps://www.msu.edu/dept/soweb/writitem.html#tipshttps://www.msu.edu/dept/soweb/writitem.html#precisehttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#generalhttps://www.msu.edu/dept/soweb/writitem.html#jargonhttps://www.msu.edu/dept/soweb/writitem.html#clueshttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#typehttps://www.msu.edu/dept/soweb/writitem.html#keyshttps://www.msu.edu/dept/soweb/writitem.html#reviewhttps://www.msu.edu/dept/soweb/writitem.html#specifichttps://www.msu.edu/dept/soweb/writitem.html#mcitemhttps://www.msu.edu/dept/soweb/writitem.html#stemhttps://www.msu.edu/dept/soweb/writitem.html#corranshttps://www.msu.edu/dept/soweb/writitem.html#esscharhttps://www.msu.edu/dept/soweb/writitem.html#knowmathttps://www.msu.edu/dept/soweb/writitem.html#awareobjhttps://www.msu.edu/dept/soweb/writitem.html#awaremodhttps://www.msu.edu/dept/soweb/writitem.html#studentshttps://www.msu.edu/dept/soweb/writitem.html#wrtskillhttps://www.msu.edu/dept/soweb/writitem.html#technicshttps://www.msu.edu/dept/soweb/writitem.html#tipshttps://www.msu.edu/dept/soweb/writitem.html#precisehttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#qualifyhttps://www.msu.edu/dept/soweb/writitem.html#generalhttps://www.msu.edu/dept/soweb/writitem.html#jargonhttps://www.msu.edu/dept/soweb/writitem.html#clueshttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#typehttps://www.msu.edu/dept/soweb/writitem.html#keyshttps://www.msu.edu/dept/soweb/writitem.html#reviewhttps://www.msu.edu/dept/soweb/writitem.html#specifichttps://www.msu.edu/dept/soweb/writitem.html#mcitemhttps://www.msu.edu/dept/soweb/writitem.html#stemhttps://www.msu.edu/dept/soweb/writitem.html#corranshttps://www.msu.edu/dept/soweb/writitem.html#intro8/9/2019 Writing Test Items
2/23
Select .iagnostic 2oils or .istracters
O%tions Should be &resented in a 0ogical( S'steatic Order
O%tions should be -raaticall' &arallel and Consistent with the Ste
O%tions Should be Mutuall' #$clusive
Insure that Correct +es%onses are not Consistentl' Shorter or 0onger than
the 2oils
#liinate -raatical or 3erbal Clues
&resent the &roble in *ovel Ters
Use *egativel' Stated Ites Infre"uentl'
Beware of 4*one of These(4 *one of the Above(4 4All of these(4 and 4Allof the Above4
Alter Ite .ifficult' b' Ma!ing O%tions More Ali!e or 0ess Ali!e in
Meaning
o Alternative1+es%onse Ites
Include Onl' One Idea in #ach Ite
#liinate &artl' True1&artl' 2alse Ites
#liinate S%ecific .eteriners
Insure that True and 2alse Ites are A%%ro$iatel' #"ual in 0ength
Balance the *uber of True Ites and 2alse Ites
#liinate 3ague Ters of .egree or Aount
Use Caution in Writing *egative Ite Stateents
o Matching Ites
Include 5oogeneous Material in #ach #$ercise
Include at 0east Three to 2ive but no More than #ight to Ten Ites in a
Matching Set
#liinate Irrelevant Clues
2
https://www.msu.edu/dept/soweb/writitem.html#diagnosthttps://www.msu.edu/dept/soweb/writitem.html#orderhttps://www.msu.edu/dept/soweb/writitem.html#grammathttps://www.msu.edu/dept/soweb/writitem.html#exclusivhttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#elimcluehttps://www.msu.edu/dept/soweb/writitem.html#novelhttps://www.msu.edu/dept/soweb/writitem.html#negativhttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#tfitemshttps://www.msu.edu/dept/soweb/writitem.html#oneideahttps://www.msu.edu/dept/soweb/writitem.html#parttruhttps://www.msu.edu/dept/soweb/writitem.html#specdethttps://www.msu.edu/dept/soweb/writitem.html#lengthhttps://www.msu.edu/dept/soweb/writitem.html#balancehttps://www.msu.edu/dept/soweb/writitem.html#vaguehttps://www.msu.edu/dept/soweb/writitem.html#negstathttps://www.msu.edu/dept/soweb/writitem.html#matchitmhttps://www.msu.edu/dept/soweb/writitem.html#homogenhttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#irrelevhttps://www.msu.edu/dept/soweb/writitem.html#diagnosthttps://www.msu.edu/dept/soweb/writitem.html#orderhttps://www.msu.edu/dept/soweb/writitem.html#grammathttps://www.msu.edu/dept/soweb/writitem.html#exclusivhttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#consisthttps://www.msu.edu/dept/soweb/writitem.html#elimcluehttps://www.msu.edu/dept/soweb/writitem.html#novelhttps://www.msu.edu/dept/soweb/writitem.html#negativhttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#bewarehttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#alterdifhttps://www.msu.edu/dept/soweb/writitem.html#tfitemshttps://www.msu.edu/dept/soweb/writitem.html#oneideahttps://www.msu.edu/dept/soweb/writitem.html#parttruhttps://www.msu.edu/dept/soweb/writitem.html#specdethttps://www.msu.edu/dept/soweb/writitem.html#lengthhttps://www.msu.edu/dept/soweb/writitem.html#balancehttps://www.msu.edu/dept/soweb/writitem.html#vaguehttps://www.msu.edu/dept/soweb/writitem.html#negstathttps://www.msu.edu/dept/soweb/writitem.html#matchitmhttps://www.msu.edu/dept/soweb/writitem.html#homogenhttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#includehttps://www.msu.edu/dept/soweb/writitem.html#irrelev8/9/2019 Writing Test Items
3/23
&lace #ach Set of Matching Ites on a Single &age
+educe the Influence of Clues and thereb' Increase Matching Ite
.ifficult'
Co%ose the +es%onse 0ist of Single Words or 3er' short &hrases
Arrange the +es%onses in S'steatic Order6 Al%habetical( Cronological(
etc7
o The &roof of the Ite Writing is in the Ite Aanal'sis
Introduction
Writing test ites is a atter of %recision( %erha%s ore a!in to co%uter %rograing than towriting %rose7 A test ite ust focus the attention of the e$ainee on the %rinci%le or construct
u%on which the ite is based7 Ideall'( students who answer a test ite incorrectl' will do so
because their aster' of the %rinci%le or construct in focus was inade"uate or inco%lete7 An'
characteristics of a test ite which distract the e$ainee fro the ajor %oint or focus an ite(reduce the effectiveness of that ite7 An' ite answered correctl' or incorrectl' because of
e$traneous factors in the ite( results in isleading feedbac! to both e$ainee and e$ainer7
A %oet or writer( es%eciall' of fiction( relies on rich ental iager' on the %art of the reader to%roduce an i%act7 2or ite writers( however( the tas! is to focus the attention of a grou% of
students( often with widel' var'ing bac!ground e$%eriences( on a single idea7 Such
counication re"uires e$tree care in choice of words and it a' be necessar' to tr' the ites
out before %robles can be identified7
8 To%9
Essential Characteristics of Item Writers
-iven a tas! of %recision counication( there are several attributes or ind sets that arecharacteristics of a %roficient ite writer7
Knowledge and Understanding of the Material Being Tested
At the Universit' level( the de%th and co%le$it' of the aterial on which students are tested
necessitates that onl' facult' ebers full' trained in a %articular disci%line can write concise(unabiguous test ites in that disci%line7 2urther( the nuber of %ersons who can eaningfull'
criti"ue test ites( in ters of the %rinci%les or constructs involved( is liited7 An agreeent b'
colleagues to review each others: tests will li!el' i%rove the "ualit' of ites considerabl' %rior
to the first tr'1out with students7
3
https://www.msu.edu/dept/soweb/writitem.html#singlehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#wordshttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#proofhttps://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#singlehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#reducehttps://www.msu.edu/dept/soweb/writitem.html#wordshttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#arrangehttps://www.msu.edu/dept/soweb/writitem.html#proofhttps://www.msu.edu/dept/soweb/writitem.html#top8/9/2019 Writing Test Items
4/23
Continuous wareness of !"#ecti$es
A test ust reflect the %ur%oses of the instruction it is intended to assess7 This "ualit' of a test(
referred to as content validit'( is assured b' s%ecif'ing the nature and;or nuber of ites %rior toselecting and writing the ites7 Instructors soeties develo% a chart or test blue%rint to hel%
guide the selection of ites7 Such a chart a' consider the odules or bloc!s of content as wellas the nature of the s!ills a test is e$%ected to assess7
In the case of criterion1referenced instruction( content validit' is obtained b' selecting a sa%leof criteria to be assessed7 2or content1oriented instruction( a balance a' be achieved b'
selecting ites in %ro%ortion to the aount of instructional tie allotted to various bloc!s of
aterial7 An e$a%le of a test blue%rint for a fift'1ite test is shown below7
Types of Tests %elia"ility &alidity Correlation Total
Knowledge of terms < = = = >
Comprehension of principles < ? ? ? =>pplication of principles @ ? > =
nalysis of situations = @ @ @
E$aluation of solutions 1 @ @ @
Total =< => =? >D
The blue%rint s%ecifies the nuber of ites to be constructed for each cell of the two1wa' chart72or e$a%le( in the above test blue%rint( four ites are to involve the a%%lication of the
%rinci%les of reliabilit'7
Continuous wareness of Instructional Model
.ifferent instructional odels re"uire ites of "uite different characteristics for ade"uateassessent7 2or e$a%le( a%%ro%riate ite difficult' in a aster'1odel situation ight be a
a$iu value of @D Etwent'1%ercent of the students answering incorrectl'F7 On the other hand(
ites written for a norative odel ight have an a%%ro%riate average difficult' of the order of
8/9/2019 Writing Test Items
5/23
aster' odels( ite difficult' ust not be so low as to %rovide no challenge whatever to an'
e$ainee in a class7
It is generall' easier to adjust the difficult' than to adjust the discriination of an ite7 Ite
discriination de%ends to a degree on the range of e$ainee abilit' as well as on the difficult' ofthe ite7 It can be difficult to write aster'1odel ites which do not discriinate when the
range of abilities aong e$ainees is wide7 0i!ewise( hoogeneous abilities a!e it ore
difficult to write norative1odel ites with acce%tabl' high discriinations7
*o atter what the instructional odel or the range of abilities in a class( the onl' wa' toidentif' a%%ro%riate ites is to select the on the basis of subjective judgent( adinister the(
and anal',e the results7 Then onl' ites of a%%ro%riate difficult' and discriination a' be
retained for future use7
S'ill in Written Communication
An ite writer:s goal is to be clear and concise7 The level of reading difficult' of the ites ust
be a%%ro%riate for the e$ainees7 Wording ust not be ore co%licated than that used in
instruction7
S'ill in Techni(ues of Item Writing
There are an' hel%ful hints and lists of %itfalls to avoid which a' be hel%ful to the itewriter7 This is an area where easureent s%ecialists a' be %articularl' hel%ful7 The reainder
of this hand1out will be devoted to ite1writing ti%s7
8 To%9
GE)E%* TI+S
E,press Items as +recisely- Clearly and Simply as +ossi"le
Unnecessar' aterial reduces the effectiveness of an ite b' forcing e$ainees to res%ond to the
irrelevant aterial and %erha%s be distracted b' it7 2or e$a%le( the following ite6
In carr'ing out scientific research( the t'%e of h'%othesis which indicates the direction in which
the e$%erienter e$%ects the results to occur once the data has been anal',ed is !nown as aEnF 777
could be written
An h'%othesis which indicates the e$%ected result of a stud' is called aEnF 777
Include all .ualifications )ecessary to +ro$ide a %easona"le Basis for %esponding
The ite
What is the ost effective t'%e of test iteG
ight be rewritten
5
https://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#top8/9/2019 Writing Test Items
6/23
According to #bel( the ost versatile t'%e of objective ite for easuring a variet' of
educational outcoes is the 777
The second version s%ecifies whose o%inion is to be used( narrows the tas! to consideration of
objective ites( and focuses on one ite characteristic7 The first version %oses an alosti%ossible tas!7
Emphasi/e General Tas's %ather than Small 0etails
The ite
The %roduct1oent coefficient of correlation was develo%ed b'
=7 /ohn -osset
@7 Sir +onald 2isher
8/9/2019 Writing Test Items
7/23
>7 has re%resentative nors7
The %hrase 4easures what it %ur%orts to easure4 is considered to be a easureent cliche
which would be "uic!l' recogni,ed b' students in the area7 The ite ight be rewritten6
The validit' of a test a' be deterined b'
=7 easuring the consistenc' of its scores7
@7 co%aring its scores with those of a %arallel for7
8/9/2019 Writing Test Items
8/23
>7 valid7
Eliminate Irrele$ant Sources of 0ifficulty
Other e$traneous sources of difficult' a' %lague e$ainees in addition to the ite faults
entioned above7 Students a' isunderstand the test directions if the test forat is co%le$and;or the students are not failiar with it7 When res%onse !e's are coon to two or ore
ites( care ust be ta!en that students are ade aware of the situation7 If a set of ites using acoon !e' e$tends to a second %age( the !e' should be re%eated on the second %age7 Then
students will not forget the !e' or have to turn bac! to an earlier %age to consult the !e'7
Whenever co%le$ or unfailiar test forats are used( e$ainees should have an o%%ortunit' to
%ractice res%onding to ites %rior to the actual test whose results are used for grading7 Such a
%ractice adinistration will also give the ite writer an indication of difficulties students a' be
having with directions or with the test forat7
+lace all Items of a Gi$en Type Together in the Test
-rou%ing li!e test ites allows e$ainees to res%ond to all ites re"uiring a coon ind1setat one tie7 The' don:t have to continuall' shift bac! and forth fro one t'%e of tas! to another7
2urther( when ites are grou%ed b' t'%e( each ite is contiguous to its a%%ro%riate set ofdirections7
+repare Keys or Model nswers in d$ance of Test dministration
&re%aring a !e' for objective1t'%e ites or a odel answer to essa' or short answer ites is an
e$cellent wa' to chec! the "ualit' of the ites7 If the are ajor flaws in ites( the' are li!el' tobe discovered in the !e'ing %rocess7 &re%aring a odel answer %rior to adinistering the test is
es%eciall' i%ortant for essa' or other o%en1end ites because it allows the e$ainer to develo%
a frae of reference %rior to grading the first e$aination7
rrange for Competent %e$iew of the Items
An'one who has atte%ted to %roof his or her own co%' !nows that it is uch better to have theaterial %roofed b' another %erson7 The sae %rinci%le a%%lies to %roofing test ites7 5owever(
it is i%ortant that the outside reviewer be co%etent in the subject atter area7 Unfortunatel'(
critical review of test ites is a deanding and tie1consuing tas!7 Ite writers a' a!ereci%rocal agreeents with colleagues or a' find advanced students to criti"ue their ites7 Test
construction s%ecialists a' %rovide hel%ful coents with res%ect to general ite
characteristics7
8 To%9
Writing Specific Types of Items
The reainder of this handboo! will deal with s!ills hel%ful in writing s%ecific t'%es of ites7
There is an alost infinite variet' to the fors test ites a' ta!e7 Test ites are often grou%ed
into two ain categories6 objective ites and constructed1res%onse ites7 Objective ites arethose in which the e$ainee recogni,es a best answer fro o%tions %resented in the ite7
8
https://www.msu.edu/dept/soweb/writitem.html#tophttps://www.msu.edu/dept/soweb/writitem.html#top8/9/2019 Writing Test Items
9/23
Objective ites include ulti%le1choice ites( alternative1res%onse ites and atching ites7
Constructed1 res%onse ites include restricted1res%onse ites( short1answer ites( co%letion
ites and essa' ites7 #ach t'%e of ite will be considered in turn on the following %ages7
Multiple2Choice Items
A ulti%le1choice ite %resents a %roble or "uestion in the ste of the ite and re"uires the
e$ainee to select the best answer or o%tion7 The o%tions consist of a ost1correct answer andone or ore distracters or foils7 Consider the following e$a%le7
The stateent 4Attitude toward su%%ort of %ublic schools is easured b' %erforance at the%olls4 is an e$a%le of
=7 a theor'7
@7 induction7
7 a deduction or 4if then4 stateent7
The ste is the %hrase 4The stateent Attitude toward su%%ort of %ublic schools is easured b'%erforance at the %olls: is an e$a%le of74 The nubered res%onses are the o%tions( with o%tion
nuber four being the correct answer( and o%tions one( two( three and five are foils or
distracters7 *ow let us consider soe hints for constructing this ulti%le1choice t'%e of ite7
State the &roble in the Ste The ite
Multi%le1choice ites
=7 a' have several correct answers7
@7 consists of a ste and soe o%tions7
8/9/2019 Writing Test Items
10/23
?7 ste and a correct answer7
A student who has been given the objective of recogni,ing the co%onents of a ulti%le1choice
ite will read the ste( and iediatel' !now the correct answer7 The onl' reaining tas! is to
locate the o%tion which contains the co%lete list of co%onents7
Include One Correct or Most .efensible Answer The ite below would be a good basis for
discussion but %robabl' should not be included in an e$aination7
The ost serious as%ect of the energ' crisis is the
=7 %ossible lac! of fuel for industr'7
@7 %ossibilit' of wides%read une%lo'ent7
7 cost of develo%ing alternate sources of energ'7
Such an ite ight be rewritten to focus on a ore s%ecific and;or a as%ect of the energ' crisis7
It ight also be written to focus on the o%inion of a recogni,ed e$%ert6
According to &rofessor Koenig( the ost serious as%ect of the energ' crisis is the
=7 %ossible lac! of fuel for industr'7
@7 %ossibilit' of wides%read une%lo'ent7
7 cost of develo%ing alternative sources of energ'7
Select .iagnostic 2oils or .istracters Such as 11
Cliches
Coon Misinforation
0ogical Misinter%retations
&artial Answers
Technical Ters or Te$tboo! /argon
10
8/9/2019 Writing Test Items
11/23
The ajor %ur%ose of a ulti%le1choice ite is to identif' e$ainees who do not have co%lete
coand of the conce%t or %rinci%le involved7 In order to acco%lish this %ur%ose( the foils or
distracters ust a%%ear as reasonable as the correct answer to students who have not asteredthe aterial7 Consider the following ite6
A terinal a' be defined as
=7 a final stage in a co%uter %rogra7
@7 the %lace where a co%uter is !e%t7
8/9/2019 Writing Test Items
12/23
=7 diagnostic test7
@7 criterion1referenced tests7
7 subjective test7
The e$ainee is led directl' to o%tion < b' the last word in the ste which re"uires an o%tionwith its first word beginning with a vowel7 O%tion @ is rendered ore i%lausible b' the
singular1 %lural inconsistenc'7 The ite ight be rewritten as follows6
A test( which can be scored b' a cler! untrained in the content area of the test( is said to be
=7 diagnostic7
@7 criterion1referenced7
7 subjective7
O%tions Should be Mutuall' #$clusive A !nowledgeable e$ainee ust be able to locate onl'
one o%tion which will contain the correct or best answer7 Consider the fault' ite below7
What should be the inde$ of difficult' for an effective aster'1odel test iteG
=7 0ess than =D
@7 0ess than @D
8/9/2019 Writing Test Items
13/23
8/9/2019 Writing Test Items
14/23
?7 find out if the test is reliable7
>7 evaluate the overall difficult' of the test7
The word 4anal',e4 in o%tion @ would be associated with 4ite anal'sis4( leading e$ainees to
the correct o%tion through e$traneous inforation7 The %roble could be solved( in this case( b'changing the word 4anal',e4 in o%tion @ to 4consider47 2urther( an association ight be %rovided
for un!nowledgeable e$ainees b' changing o%tion < to 4anal',e content of the test47
&resent the &roble in *ovel Ters Most often( we are interested in easuring co%le$ ental
%rocesses such as a%%lication of %rinci%les rather than ere eor' of factual !nowledge7 If weatte%t to easure a co%le$ ental tas! such as evaluation( but we use e$a%les with which
the students are failiar( we reduce the tas! of one of sheer eor'7
In %resenting e$aination ites in novel ters( we ust ta!e care not to a!e the ites
e$treel' difficult relative to e$a%les used in instruction7 If we use novel ite t'%es( we usta!e sure that students understand the new %rocess and that the' have had an o%%ortunit' to
%ractice the re"uired s!ill7
Use *egativel' Stated Ites Infre"uentl' There are situations in which the ost eaningful tas!
we can re"uire of an e$ainee is to identif' the e$ce%tion in a set of o%tions7 The ite ste willoften as!6
Which of the following is *OT an e$a%le of HHHHHHHHH G
A ajor %roble with a negativel'1stated ite is that students a' iss the negation when
reading the ste7 A negativel'1stated ite does re"uire an e$aine to switch his or her ind setfro that of loo!ing for the best answer to that of locating the ost definite non1answer7 Ites
with negativel' stated stes can often be rewritten as effective %ositivel'1stated ites7
2or e$a%le( the negativel'1stated ite
Which of the following is *OT a ethod of deterining test reliabilit'G
=7 Coefficient of e"uivalence
@7 Coefficient of stabilit'
7 Test1criterion intercorrelation
a' be re%hrased as a %ositivel'1stated ite7
Which of the following is a ethod of deterining the validit' of a testG
=7 Coefficient of e"uivalence
14
8/9/2019 Writing Test Items
15/23
@7 Coefficient of stabilit'
7 Test1criterion correlation
The correct answer to each of the two above ites is o%tion >7
Beware of 4*one of These(4 *one of the Above(4 4All of these(4 and 4All of the Above74 Theo%tions 4*one of these4 and 4*one of the above4 should not be used when the e$ainee is to
select the best( but not necessaril' absolutel' correct answer7 The' should onl' be used when one
of the o%tions would be agreed u%on b' e$%erts as absolutel' correct7 The ite below illustratesthe ina%%ro%riate use of the 4*one of the above4 o%tion7
What is an ideal level of difficult' for an objective test iteG
=7 =D
@7 @D
7 *one of the above
The correct answer would de%end on the test odel used as a frae of reference7 The ite ight
be i%roved b' rewriting it as follows7
What level of ite difficult' would allow for a$iu discriinationG
=7 =D
@7 @D
D
?7 D
>7 *one of the above
Ma$iu discriination could occur onl' at the level of difficult' of >D7 Therefore( 4*one of
the above4 is a ore a%%ro%riate o%tion than in the earlier version of the ite7
4All of these4 and 4All of the above4 tend to be less useful o%tions than the 4*one of these4 t'%e
of o%tion7 When 4All of these4 or 4All of the above4 are used with a five1o%tion ulti%le choiceite( an e$ainee has onl' to recogni,e an' two of the four o%tions as correct to be led to the
15
8/9/2019 Writing Test Items
16/23
correct answer7 Conversel'( the e$ainee needs onl' to recogni,e one of the o%tions as incorrect
in order to reject the 4All of the above4 o%tion7
If *one of the above4 and 4All of the above4 are to be used as o%tions( the' ust be used
occasionall' as the correct o%tion7 If the' are seldo or never !e'ed as correct( the e$aineeswill soon recogni,e the as 4fillers4 and ignore the7
Alter Ite .ifficult' b' Ma!ing O%tions More Ali!e or 0ess Ali!e in Meaning Ite o%tionswhich are ore ali!e in eaning %rovide a ore difficult choice than do those which are ore
obviousl' different7 Consider the three following ites7
E#asiestF The "ualit' of a test which indicates how consistentl' the test easures is called
=7 objectivit'7
@7 reliabilit'7
8/9/2019 Writing Test Items
17/23
An alternative1res%onse ite is a s%ecial case of theulti%le1choice ite forat7 There are an'
situations which call for either1or decisions( such as deciding whether a s%ecific solution is right
or wrong( whether to continue or to sto%( whether to use a singular or %lural construction( and soon7 2or such situations( the alternative res%onse ite is an ideal easuring device7 Since onl'
two o%tions are %ossible( alternative1res%onse ites are generall' shorter( and( therefore( re"uire
less reading tie7 Students a' res%ond to ore alternative1res%onse ites than other t'%es of
ites in a given length of tie7
A ajor disadvantage of alternative1res%onse ites is the fact that students have fift'1fift'
%robabilit' of answering the ite correctl' b' chance alone7 When carefull' written and
%retested( alternative1res%onse ites a' be written which e$ceed ulti%le1 choice ites in
abilit' to discriinate7 -enerall'( however( it is considered necessar' to have a larger nuber ofalternative1choice ites than of other t'%es of ites in order to achieve a given level of test
reliabilit'7
There are two ain t'%es of alternative1res%onse ites7 One t'%e is essentiall' a two1o%tionulti%le1choice ite7 An e$a%le follows7
To deterine the degree of relationshi% between two continuous variables( one ust co%ute the
=7 %roduct1oent coefficient of correlation7
@7 ran!1order coefficient of correlation7
A second t'%e of alternative1res%onse ite is the co%lete stateent forat( the ost failiar
e$a%le of which is the true1false ite7
To deterine the degree of relationshi% between two continuous variables( one ust co%ute the
ran!1order coefficient of correlation7
=7 True
@7 2alse
The correct answer to the two1choice version is o%tion @( and the correct answer to the co%lete1stateent version is o%tion @( 2alse7
A ajor distinction between the co%lete1stateent( true1false t'%e of ite and ites in
ulti%le1choice or two1choice forats( is that the co%lete1stateent ite contains no criterion
for answering the ite7 The criterion is outside of the ite( rooted in the characteristics and
e$%eriences of each individual e$ainee7 #ach e$ainee ust as! the "uestion( true or falsewith res%ect to whatG It follows that each co%lete1stateent( true1false ite ust be
une"uivocall' true or une"uivocall' false7 It sees intuitive that %ro%er wording and the
eliination of e$traneous clues are ore crucial with the true1false t'%e of ite than with an'other ite forat7 2ollowing are soe %oints to consider in writing ore effective alternative1
res%onse ites7
Include Onl' One Idea in #ach Ite Alternative1res%onse ites testing two ideas siultaneousl'%rovide little useful feedbac! to the e$ainee or to the e$ainer7 If such 4double1barreled4 ites
17
8/9/2019 Writing Test Items
18/23
are answered incorrectl'( it is not certain whether one of the ideas or both of the are res%onsible
for the confusion7 A two1idea ite is given below7
Test validit' is a function of test reliabilit'( which can be i%roved b' using fewer ites7
=7 True
@7 2alse
The answer to the ite is false since ore ites would be needed to i%rove test reliabilit'7 Theite should be s%lit into two true1false ites7
#liinate &artl' True1&artl' 2alse Ites 0ac! of absolute truth or falsit' of an ite generall'
results fro an ite writer:s failure to consider all %ossible fraes of reference fro which anite a' be answered7 Consider this e$a%le7
To be valid( a test ust be content1balanced7
=7 True
@7 2alse
When considered fro the %oint of view of an e$ainer of student achieveent( the ite is true7
5owever( an e$ainer interested in student a%titude would li!el' consider the ite false( sincehe or she would be %riaril' interested in accurate %rediction of a criterion( not on the content of
the test7 The solution for the sa%le ite above is si%le7
To be valid( an achieveent test ust be content balanced7
=7 True
@7 2alse
The ite a' now be !e'ed true7
#liinate S%ecific .eteriners In the real world( it is difficult to find things about which we can
a!e absolute stateents( such as 4It is never true4 or 4It is alwa's true74 3er' often( we need to
"ualif' our stateents( with words li!e often( as a rule( soe ties( a'( and so on7 Students usethe following generali,ations when answering true1false ites7 If an ite contains an absolute
s%ecific deteriner11such as alwa's or never11ar! the ite false7 If an ite contains a
"ualif'ing s%ecific deteriner11such as soeties or %robabl'11ar! it true7
Writers of alternative1res%onse ites ust be e$treel' cautious in words which a' serve ass%ecific deteriners7 In fact( ite writers should atte%t to use s%ecific deteriners in a wa'
which will cause test1wise but un!nowledgeable students to answer the ite incorrectl'7 An
e$a%le of an ina%%ro%riate use of a s%ecific deteriner is given below7
All valid tests are reliable tests7
18
8/9/2019 Writing Test Items
19/23
=7 True
@7 2alse
On the basis of the s%ecific deteriner( 4all(4 an e$ainee would answer the ite false( the
!e'ed answer7 A ore a%%ro%riate version of the ite is given below7
All valid a%titude tests are reliable tests7
=7 True
@7 2alse
In this case the s%ecific deteriner 4all4 would lead a test1wise e$ainee to select the 4false4
answer( but the !e'ed answer is 4true74
Insure that True and 2alse Ites are A%%ro$iatel' #"ual in 0ength Since we often have to
"ualif' stateents in order to a!e the une"uivocall' true( test1wise students often use ite
length as an e$traneous clue7 #fforts should be ade to write false ites of about the saelength as true ites7
e7 Balance the *uber of True Ites and 2alse Ites The nuber of true and false ites shouldbe a%%ro$iatel' balanced so that test1wise students will a%%roach an ite on the basis of its
content rather than on the %robabilit' of its being true or false7 Soe authors argue that the
%ro%ortion of false ites should e$ceed the nuber of true ites7 The' argue that( in general(
res%ondents tend to agree with stateents rather than disagree with the7 Thus the false iteswill tend to be soewhat ore difficult and the total test will be soewhat ore reliable7
#liinate 3ague Ters of .egree or Aount Words li!e 4fre"uentl'4 and 4seldo4 are
es%eciall' o%en to inter%retation in true1false ites( which have no built1in frae of reference7 Itis generall' %ossible to edit such vague ters out of true1false ites7 5ere is an e$a%le7
+eliabilit' is fre"uentl' deterined b' the s%lit1halves ethod7
=7 True
@7 2alse
This difficult1to1answer ite a' be rewritten as follows7
The s%lit1halves ethod is used to deterine test reliabilit'7
=7 True
@7 2alse
The revised ite a' be !e'ed =7 True7
19
8/9/2019 Writing Test Items
20/23
Use Caution in Writing *egative Ite Stateents The use of negative alternative1res%onse ite
stateents involves the sae %erils as does the use of negatives in ulti%le1choice ite stes7
#s%eciall' to be avoided is the double1negative( which soeties %o%s u% in alternative1res%onse ite stateents7 An e$a%le is given below7
There is no advantage in not using s%ecific deteriners in alternative1res%onse ites7
=7 True
@7 2alse
The ite should be rewritten7
S%ecific deteriners should be balanced between true and false ites7
=7 True
@7 2alse
The ite is now !e'ed =7 True7
Matching Items
A atching ite consists of two coluns6 one colun of stes or %robles to be answered( and
another colun of res%onses fro which the answers are to be chosen7 Traditionall'( the colun
of stes is %laced on the left and the colun of res%onses is %laced on the right7 An e$a%le isgiven below7
.irections6 Match the data gathering %rocedures in the ite colun on the left with the nae of
the data gathered in the res%onse colun on the right7 &lace 'our answer in the blan! to the leftof each %rocedure7 #ach answer a' be used onl' once7
0ata Gathering +rocedure Type of 0ata
EaF =7 Adinister two fors of a test a7 Coefficient of e"uivalence
EdF @7 #stiate reliabilit' on the basis of ite data b7 Coefficient of stabilit'EcF
8/9/2019 Writing Test Items
21/23
*ote that( in the above e$a%le( it is necessar' to answer the ulti%le1choice ite in order to
answer the %arent atching ite7 *ote also that the res%onses Eite co%onentsF in the list at the
right have a EsF added to each res%onse in order to eliinate singular1%lural e$traneous clues7
Because of the nature of the atching tas!( naes with events( for e$a%le( it is clear thatatching ites often easure recognition of factual !nowledge rather than higher level ental
%rocesses7 5ere are soe hints for writing atching ites7
Include 5oogeneous Material in #ach #$ercise Of all %ossible ite t'%es( atching ites are
the ost li!el' to contain e$traneous clues to the correct answers( es%eciall' when heterogeneousaterial is included in one atching ite7 Consider the following e$a%le7
EaF =7 Measures factual !nowledge a7 Matching ite
EbF @7 One ste and several o%tions b7 Multi%le choice ite
EdF
8/9/2019 Writing Test Items
22/23
These rules are illustrated b' the ite revision shown below7
.irections6 Match the data gathering %rocedures in the ite colun on the left with the nae of
the data gathered in the res%onse colun on the right7 &lace 'our answer in the blan! to he left of
each %rocedure7 #ach res%onse a' be used ore than once7
0ata Gathering +rocedure Type of 0ata
EaF =7 Adinister two fors of a test a7 Coefficient of e"uivalence
EdF @7 #stiate reliabilit' on the basis of ite data b7 Coefficient of stabilit'EcF
8/9/2019 Writing Test Items
23/23
&rinted handouts are available fro the Scoring Office( ==? Co%uter Center( which describe the
ite anal'sis %rint1out and how it a' be used to guide and i%rove instruction( and to evaluate
and i%rove ites7
23