ITEM ANALYSISARDHIAN SUSENO
CHOIRUL RISA PRADANA P.
ITEM ANALYSIS
is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole
SCOREPAK®
PURPOSE1. To improve items which will be used
again in later tests. 2. To eliminate ambiguous items in a
single test administration.3. To increase instructors' skills in test
construction.4. To identify specific areas of course
content which need greater emphasis or clarity.
Method 1) ITEM DIFFICULTY2) ITEM DISCRIMINATIO ITEM
CHARACTERISTIC CURVE3. Item validity (point biserial method)4. Effectiveness of distractors
I. ITEM DIFFICULTY• item difficulty is determined by the number of people
who answer a particular test item correctly (p).• For example, if the first question on a test was answered
correctly by 76% of the class, then the difficulty level (p or percentage passing) for that question is p = .76. If the second question on a test was answered correctly by only 48% of the class, then the difficulty level for that question is p = .48. The higher the percentage of people who answer correctly, the easier the item, so that a difficulty level of .48 indicates that question two was more difficult than question one, which had a difficulty level of .76.
Method of Item Difficulty
a) Method for Dichotomously Scored Item
b) Method for Polytomously Scored Item
c) Grouping Method
a). Method for Dichotomously Scored Item
Method for Dichotomously Scored Items
p is the difficulty of a certain item. R is the number of examinees who get that item
correct. N is the total number of examinees.
NRP
Example 1
There are 80 high school students attending a
science achievement test, and 61 students pass item
1. Please calculate the difficulty for item 1.
Answer : 0.7
b). Method for Polytomously Scored Items
maxXXP
X , the mean of total examinees’ scores on one item
maxX , the perfect scores of that item
Example 2 The perfect scores of one open- ended item is 20
points, the average score of total examinees on this
item is 11 points. What is the item difficulty?
Answer : 0.55
c). Grouping Method (Use of Extreme Groups)
• Upper (U) and Lower (L) Criterion groups are selected from the extremes of distribution of test scores or job ratings
2LU PPP
UP is the proportion for examinees of upper group who get the item correct.
LP is the proportion for examinees of lower group who get the item correct.
Example 3 There are 370 examinees attending a language
test. Known that 64 examinees of 27% upper
extreme group pass item 5, and 33 examinees of 27%
lower extreme group pass the same item. Please
compute the difficulty of item 5.
Answer : 0.49
II. ITEM DISCRIMINATION• Item discrimination refers to the degree to
which an item differentiates correctly among test takers in the behavior that the test is designed to measure.
• item discrimination determines whether those who did well on the entire test did well on a particular item.
Method of Item Discrimination
a). Index of Discrimination
D = pH - pL (7.5)
We need to set one or two cutting scores to divide the examinees into
upper scoring group and lower scoring group. pH is the proportion in the upper group who answer the item correctly and pL is the proportion in the lower group who answer
the item correctly. Values of D may range from -1.00 to 1.00.
ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.
Another example
50 Examinees’ Test Data on 8-Item Scale About Job Stress.
Question: There are 140 students attending a world history test. (1)
If we use the ratio 27% to determine the upper and lower group, then how many examinees are there in the upper and lower group separately? (2)If 18 examinees in upper group answer item 5 correctly, and 6 examinees in lower group answer it correctly, then calculate the discrimination index for item 5.
ITEM VALIDITY :POINT-BISERIAL METHOD
ILLUSTRATION BY GGS INFORMATION SERVICES. CENGAGE LEARNING, GALE.
• Another way to determine the
discriminability of an item is to determine the correlation coefficient between performance on an item and performance on a test, or the tendency of students selecting the correct answer to have high overall scores. POINT-BISERIAL METHOD
Distractor (Incorrect Alternatives)
• Analyzing the distractors (e.i., incorrect alternatives) is useful in determining the relative usefulness of the decoys in each item. Items should be modified if students consistently fail to select certain multiple choice alternatives. The alternatives are probably totally implausible and therefore of little use as decoys in multiple choice items. A discrimination index or discrimination coefficient should be obtained for each option in order to determine each distractor's usefulness .