Epm5153 Assessment in Education - based on subject topic and finding the index distracter and...

7/30/2019 Epm5153 Assessment in Education - based on subject topic and finding the index distracter and discrimination

1/27

EPM5153 ASSESSMENT IN EDUCATION

1.0 INTRODUCTION

The item analysis is an important phase in the development of an exam program. In this phase

statistical methods are used to identify any test items that are not working well. If an item is too

easy, too difficult, failing to show a difference between skilled and unskilled examinees, or even

scored incorrectly, an item analysis will reveal it. The two most common statistics reported in an

item analysis are the item difficulty, which is a measure of the proportion of examinees who

responded to an item correctly, and the item discrimination, which is a measure of how well the

item discriminates between examinees who are knowledgeable in the content area and those who

are not. An additional analysis that is often reported is the distractor analysis. The distractor

analysis provides a measure of how well each of the incorrect options contributes to the quality

of a multiple choice item. Once the item analysis information is available, an item review is oftenconducted.

Item analysis is a method that is used in education to evaluate test item. This can ensure that

questions are in appropriate standard and measure the effectiveness of individual test item. Item

analysis is purposed to improve test items and identify unfair or biased item. Item analysis is

conducted to avoid the chance of tests that are too difficult (and have an insufficient floor) tend

to lead to frustration and lead to deflated scores, and tests that are too easy (and have an

insufficient ceiling) facilitate a decline in motivation and lead to inflated scores.

Item analysis can be analyzed by computing: Difficulty Index, Discrimination Index, Validity

Coefficient, and Effectiveness of Distraction.

1.1 Difficulty Index

According to Wilson (2005), item difficulty is the most essential component of item analysis.

Item difficulty is determined by the number of people who answer a particular test item

correctly. It is important for a test to contain items of various difficulty levels in order to

distinguish between students who are not prepared at all, students who are fairly prepared, and

students who are well prepared.

UNITAR International University 1
http://www.proftesting.com/test_topics/steps.phphttp://www.proftesting.com/test_topics/steps.php


2/27


To compute level of difficulty we use the formula:

Difficulty Index (p) = C/T

p = Difficulty Index

C = the number of students who answer item X correctly

T = the number of total students who answer item X

For example:

There are 12 students who answer an item X, 4 of whom can answer the item correctly. So that,

the level of difficulty is:

Level of difficulty (p) = 4 / 12

= 0.33

The highest score forp is 1.0 and the lowest score is 0

p always has positive value

The higher score forp the easier item

The lower score forp the harder item

According to Allen & Yen (1986) to avoid a test is too difficult or too easy, each items in a test

should have a difficulty range from 0.3 to 0.7. It is used to differentiate between individuals

level of knowledge, ability, and preparedness.

1.2 Discrimination Index



3/27


Discrimination goes beyond determining the proportion of people who answer correctly and

looks more specifically at who answers correctly. In other words, item discrimination determines

whether those who did well on the entire test did well on a particular item. An item should in fact

be able to discriminate between upper and lower scoring groups.

To compute discrimination index, the first thing we have to do is dividing test-takers into three

groups, upper group, average group and lower group. Then using this formula:

Discrimination index (D) = CU CL

(1/2)TUL

D = Discrimination index

Cu = Number of Upper 25% students who answered the item correctly

CL = Number of Lower 25% students who answered the item correctly

1.3 The Effectiveness of Distraction

Multiple choice tests have one question and several options of answer. Among the options, there

is only one answer which is correct, and the other options are wrong answers. Those wrong

answers in multiple choice tests are called distraction. A distraction is called effective when there

are a lot of students choosing it. According to Fernandez (1984) a distraction can be called as a

good distraction when there are 2% of test takers choosing it.

1.4 Summary

In the item analysis phase oftest development, statistical methods are used to identify potential

item problems. The statistical results should be used along with substantive attention to the item

content to determine if a problem exists and what should be done to correct it. Items that are

functioning very poorly should usually be removed from consideration and the exams rescored

http://www.proftesting.com/test_topics/steps.phphttp://www.proftesting.com/test_topics/steps.php


4/27


before the test results are released. In other cases, items may still be usable, after modest changes

are made to improve their performance on future exams.

2.0 TOPIC TEST FOR ENGLISH LANGUAGE SUBJECT



5/27


SUBJECT English Language

THEME World of knowledge

TOPIC Family

CLASS 5 Perdana

AGE 11 years old

NUMBER OF STUDENTS 12 students ( 6 boys and 6 girls)

STUDENTS ABILITY Mixed Ability

LEARNING OUTCOMES At the end of this topic, students will be able to :

1 Answer the comprehension questions based on the text given.

2 Complete the sentences with the suitable words.

3 Write an essay using the word provided.

LEARNING SKILLS 3.8 Read and understand simple factual texts for main ideas, supporting

details, sequence and cause and effects.

4.3 Complete texts with the missing word, phrase or sentences.

4.4 Construct simple and compound sentences with guidance and

independently.



6/27


2.2 OBJECTIVE QUESTIONS PAPER

Name :______________________________ Class : _______________________

Paper 1 : SECTION A

Each question in this paper is followed by four possible answers : A, B, C and D. For each

question, choose only one answer.



7/27


1. Pak Kadir uses a _______________ to cut the long grass in his garden.

A hoe

B rake

C sickle

D wheelbarrow

Answer : C (Level of knowledge : Knowledge domain, level 1)

2. The goat and its ____________ are walking to the field.

A kids

B cubs

C foals

D calves

Answer : A (Level of knowledge : Knowledge domain, level 1)

3. Puan Ani gives a sharp knife to the maid and keeps the _______ one in the drawer.

A fresh

B blunt

C neat

D new

Answer : B (Level of knowledge : Knowledge domain, level 1)



8/27


4. _______________ Rahman ____________ Meng Choo came to the meeting yesterday.

A If .. so

B Either . or

C Neither norD Although . Because

Answer : C (Level of knowledge : Knowledge domain, level 2)

Choose the sentence with the correct punctuation.

5. A Mei Ling and her parents are going to singapore.

B Mei Ling and her Parents are going to Singapore.

C Mei Ling and her parents are going to Singapore.

D Mei Ling and her parents, are going to Singapore.

Answer : C (Level of knowledge : Analysis domain, level 2)

Questions 6-10

Read the dialogue below carefully and answer the questions that follow.

Eiman : Grandma, where does Uncle Latif live ?

Grandmother : He is in Australia with his family.

Anyss : Why do they live so far away ?Grandmother : Thats because he is married to an Australian who owns a restaurant there.

Anyss : How many children do they have ?

Grandmother : Three. So, you have three cousins who live far away.

Eiman : What about Aunt Maznah ?



9/27


Grandmother : She works for an oil company in Miri and lives there with her family.

Anyss : I hope Aunt Maznah wont leave us.Grandmother : I hope so too, but she

has just got a job in Kuching after completing her studies at University

Malaysia Sarawak.Eiman : I want to study at a university too. I want to be a scientist.

Anyss : I want to be an English teacher.

Grandmother : Thats good. If both of you work hard, you will be successful.

Eimanand and : Thank you, Grandma. We will study very hard and we wont disappoint

Anyss you.

6. How many children does grandmother have ?

A Two

B Three

C Four

D Five

Answer : B (Level of knowledge : Analysis domain, level 2)

7. Who is grandmother talking to ?

A Her grandchildren

B Her children

C Her parents

D Her cousins

Answer : A (Level of knowledge : Comprehension domain, level 2)

8. What is Uncle Latifs wife ?

A Scientist

B English teacher

C Restaurant owner



10/27


D Oil company worker

Answer : C (Level of knowledge : Comprehension domain, level 1)

9. The word disappointin the dialogue means

A feels proud

B be a loving person

C happy and enjoyable

D makes someone feels sad

Answer : D (Level of knowledge : Comprehension domain, level 2)

10. Eiman wants to be a scientist because he wants to

A work in an oil company.

B study at a university.

C invent new things.

D live in Australia

Answer : D (Level of knowledge : Comprehension domain, level 1)



11/27


2.3 ITEM ANALYSIS OF 10 MCQs

Question 1

Computation and Interpretation of Item Difficulty and Discrimination Indices

Students Options

C* A B D

Difficulty Index Dicrimination Index

Upper 25% 3 3 0 0 0

P = CT

= 6/12= 0.5 (50%)

D = CU CL(1/2)TUL

= 3 1

(1/2) (3+3)

= ___2__ = 2 = 0.67(1/2) (6) 3

Average 50% 6 2 2 1 1

Lower 25% 3 1 2 0 0

Total 12 6 4 1 1

Distracter Index

0.33 0.08 0.08

Comments Option A is a betterdistracter

The item is moderatelydifficult.

The item has low discrimination.



12/27


Question 1

Computation and Interpretation of Item Distraction Index

Students Options

C* A B D

Index Interpretation

Upper 25% 3 3 0 0 0 0.00 0.19 Very low distraction

Average 50% 6 2 2 1 1 0.20 0.39 Low distraction

Lower 25% 3 1 2 0 0 0.40 0.59 Moderate distraction

Total 12 6 4 1 1 0.60 0.79 High distraction

Computation 0.33 0.08 0.08 0.80 1.00 Very high distraction

Comments Option A has low distraction but is a better distracter compared to B and D

Question 2


Students Options

A* B C D


Upper 25% 3 3 0 0 0

P = CT

= 8/12= 0.67 (67%)

D = CU CL

(1/2)TUL

= 3 1(1/2) (3+3)

= ___2__ = 2 = 0.67

(1/2) (6) 3

Average 50% 6 4 1 0 1

Lower 25% 3 1 0 1 1

Total 12 8 1 1 2

Distracter Index

0.08 0.08 0.17

Comments Option D is a betterdistracter

The item is easy The item has high discrimination



13/27


Question 2


Students Options

A* B C D






Computation 0 .08 0.08 0.17 0.80 1.00 Very high distraction

Comments Option D has very low distraction but is a better distracter compared to B and C

Question 3


Students Options

B* A C D


Upper 25% 3 3 0 0 0P = C

T= 4/12

= 0.33 (33%)

D = CU CL

(1/2)TUL

= 3 0

(1/2) (3+3)

= ___3__ = 3 = 1(1/2) (6) 3

Average 50% 6 1 0 0 5

Lower 25% 3 0 0 0 3

Total 12 4 0 0 8

Distracter Index

0. 00 0.00 0.67


The item is difficult item The item has very highdiscrimination

Question 3




14/27


Students Options

B* A C D




Lower 25% 3 0 0 0 3 0.40 0.59 Moderate distractionTotal 12 4 0 0 8 0.60 0.79 High distraction


Comments Option D has very high distraction but is better distracter compared A and C..

Question 4


Students Options

C* A B D


Upper 25% 3 2 0 1 0P = C

T= 2/12

= 0.17 (17%)

D = CU CL

(1/2)TUL

= 2 0(1/2) (3+3)

= ___2__ = 2 = 0.67(1/2) (6) 3

Average 50% 6 0 0 6 0

Lower 25% 3 0 0 3 0

Total 12 2 0 10 0

Distracter Index

0.00 0.83 0.00

Comments Option B is a betterdistracter

The item is very difficult . The item has high discrimination

Question 4




15/27


Students Options

C* A B D






Comments Option C has very high distraction but is a better distracter compared to A and D

Question 5


Students Options

C* A B D


Upper 25% 3 2 1 0 0P = C

T= 5/12

= 0.42 (42%)

D = CU CL

(1/2)TUL

= 2 1

(1/2) (3+3)

= ___1__ = 1 = 0.33(1/2) (6) 3

Average 50% 6 2 1 0 3

Lower 25% 3 1 0 0 2

Total 12 5 2 0 5

Distracter Index

0.17 0.00 0.42


The item is moderatelydifficult

The item has low discrimination

Question 5




16/27


Students Options

C* A B D






Comments Option D hasModerate distraction but is a better distracter compared to A and B

Question 6


Students Options

B* A C D


Upper 25% 3 1 2 0 0P = C

T= 1/12

= 0.08 (8%)

D = CU CL

(1/2)TUL

= 3 1(1/2) (3+3)

= ___2__ = 2 = 0.67(1/2) (6) 3

Average 50% 6 0 6 0 0

Lower 25% 3 0 3 0 0Total 12 1 11 0 0

Distracter Index

0.92 0.00 0.00

Comments Option A is a betterdistracter

The item is very difficult The item has high discrimination

Question 6


Students Options Index Interpretation



17/27


B* A C D






Comments Option A hasVery high distraction but is a better distracter compared to A and C

Question 7


Students Options

A* B C D


Upper 25% 3 3 0 0 0P = C

T= 8/12

= 0.67 (67%)

D = CU CL

(1/2)TUL

= 3 1

(1/2) (3+3)

= ___2__ = 2 = 0.67(1/2) (6) 3

Average 50% 6 4 2 0 0

Lower 25% 3 1 2 0 0

Total 12 8 4 0 0

Distracter Index

0.33 0.00 0.00


The item is easy The item has High discrimination

Question 7


Students Options

A* B C D




18/27







Comments Option B has Low distraction but is a better compared C and D

Question 8


Students Options

C* A B D


Upper 25% 3 2 0 1 0P = C

T= 8/12

= 0.67 (67%)

D = CU CL

(1/2)TUL

= 2 0

(1/2) (3+3)

= ___2__ = 2 = 0.67(1/2) (6) 3

Average 50% 6 2 0 4 0

Lower 25% 3 0 1 2 0

Total 12 4 1 7 0

Distracter Index

0.08 0.58 0.00


The item is easy The item has High d iscrimination

Question 8


Students Options

C* A B D




19/27







Comments Option B has Moderate distraction but is a better compared A and D

Question 9


Students Options

D* A B C


Upper 25% 3 1 2 0 0P = C

T

= 7/12= 0.58 (58%)

D = CU CL(1/2)TUL

= 2 1(1/2) (3+3)

= ___1__ = 1 = 0.33

(1/2) (6) 3

Average 50% 6 0 2 1 3

Lower 25% 3 0 1 0 2

Total 12 1 5 1 5

Distracter Index

0.25 0.00 0.17

Comments Option A is a better

distracter

The item is Moderately

difficult

The item has very low discrimination

Question 9


Students Options

D* A B C







Comments Option A has Low distraction but is a better distracter compared to B and C



20/27


Question 10


Students Options

C* A B D


Upper 25% 3 1 1 1 0P = C

T= 1/12

= 0.08 (8%)

D = CU CL

(1/2)TUL= 1 0

(1/2) (3+3)

= ___1__ = 1 = 0.33(1/2) (6) 3

Average 50% 6 0 1 5 0

Lower 25% 3 0 1 2 0

Total 12 1 3 8 0

Distracter Index

0.25 0.67 0.00


The item is very difficultitem

The item has low discrimination

Question 10


Students Options

C* A B D









21/27


Comments Option B has high distraction but is a better distracter compared to A and D.

3.0 SUBJECTIVE QUESTION AND ANSWER SCHEME/RUBRIC

3.1 Question 1

You are advised to spend about35 minutes on this question.

Rahmans hobby is collecting butterflies. With the help of the pictures and notes given below,write a composition in not less than 100 words.

(40 marks)

____________________________________________________________________________

____________________________________________________________________________


catch butterfly own garden use butterfly net long stick

use net catch butterfly handle gently not tear out

pour drop methylated spirit

onto head preserve insect

arrange types, sizes, colours

fun cheap hobby


22/27


____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

3.2 Subjective answer

Rahman has an unusual hobby. It is collecting butterflies. He catches the butterflies in his

own garden which is full of beautiful flowers. In order to catch the butterflies, Rahman uses a

butterfly net. He uses a long stick with a net attached to one end.

First, Rahman catches a butterfly with his net. He must handle the butterfly gently so that he

does not tear out its wings. Then, he pours a small drop of methylated spirit onto the butterflys

head. This is to preserve the insect. When the butterfly is dry, he pins it onto a board.

Rahman arranges the butterflies according to types, colours and sizes. He enjoys collecting

butterflies because it is fun and is also a cheap hobby.



23/27


4.0 ASSESSMENT REPORT FOR EACH STUDENT

NO. STUDENTS' NAME MARKS GRED REPORTS COMMENTS

1 Andrea Helena Ejin 90 A Outstanding performance Keep up the good work

2 Marytheresa Raj 85 A Very good performance

Need to improve a littl

more

3 Ezron Zabion 80 A Very good performance

Need to improve a littl

more

4 Jerrel Jammy 75 B Satisfactory performance

Need to work harder to

improve

5 Daphne Sharlene Albert 70 B Satisfactory performanceNeed to work harder toimprove

6 Eldressa Peter 68 B Satisfactory performanceNeed to work harder toimprove

7 Marcella Bansing 68 B Satisfactory performanceNeed to work harder toimprove

8 Allyana Hamzah 60 B Satisfactory performance Need to do more exerc

9 Aizzat Mohd Arif 60 B Satisfactory performance Need to do more exerc10 Beyreand John 58 C Weak performance Need to do more exerc

11 Mohd Izzan Omar 45 D Very weak performanceNeed to attend extraclasses

12 Emil Yussuf 45 D Very weak performanceNeed to attend extraclasses



24/27


5.0 CONCLUSION

Evaluation / testing requires some systematic process from beginning to end. With this

systematic steps will the formation of a set of quality tests. This is due to give the test is part of

the teaching and learning process. In conducting the test, the teacher must identify the questions

are constructed to meet all the objectives of teaching and learning that have been implemented.

If you look at the results of Difficulty Index and Discrimination Index of paperwork we provide

this course, the items / questions are constructed is good and suitable for use by students for

testing. This is because both the index does not show any difficulty / extreme differences.

Therefore, every teacher must have sufficient knowledge in the construction field test the pupils /

students can master the skills learned to be more effective, and the items were appropriate and

challenge the ability of the student to be able to grow their thinking.



25/27


REFERENCES

Anderson, L. W., & Krathwohl, D. (Eds.). (2001).A taxonomy for learning, teaching, andassessing: A revision of Bloom's taxonomy of educational objectives. New York:

Longman.

Clark, D. (2010). Bloom's taxonomy of learning domains: The three types of learning. Big Dog

& Little Dog's Performance Juxtaposition. Edmonds, WA: Author. Retrieved

fromhttp://www.nwlink.com/~donclark/hrd/bloom.html

Forehand, M. (2005). Bloom's taxonomy: Original and revised.. In M. Orey (Ed.),Emerging

perspectives on learning, teaching, and technology. Retrieved

fromhttp://projects.coe.uga.edu/epltt

Huraian Sukatan Pelajaran (2003). English Language Year 5. Kuala Lumpur : Pusat

Perkembangan Kurikulum.

Lian, C. (2011). Temanku Ceria Bahasa Inggeris Year 5, Grammar and Writing. Selangor : Info-

Didik

Mertler, Craig A. (2001). Designing scoring rubrics for your classroom. Practical Assessment,

Research & Evaluation,

Moskal, Barbara M. (2000). Scoring rubrics: what, when and how? Practical Assessment,

Research & Evaluation

Yap, J (2011).Zoom In Topical Drill English Year 5. Petaling Jaya : Setia Emas

http://www.nwlink.com/~donclark/hrd/bloom.htmlhttp://www.nwlink.com/~donclark/hrd/bloom.htmlhttp://projects.coe.uga.edu/epltt/http://projects.coe.uga.edu/epltt/http://www.nwlink.com/~donclark/hrd/bloom.htmlhttp://projects.coe.uga.edu/epltt/


26/27


APPENDIX 1

Holistic Rubric

Grade Score Criteria

A (90-100) The A argument essay is exceptional in every way. The essay is well organized

and all claims are supported. It begins with a solid introduction that contains a clear

thesis, is followed by body paragraphs that contain clear topic sentences with clear

and detailed support, and ends with an effective conclusion. Content is thorough and

lacking in no area. There are no (or few) errors in tone, format, mechanics, grammar,

and content.

B (80-89) The B essay is above adequate in most areas. In the areas where it is not above

adequate, it is still entirely acceptable. The majority of the essay is clear, focused,

and well detailed, but there may be a few areas requiring further development.

While it may contain a few errors with tone, mechanics, grammar, and/or content,

these errors are not egregious enough to detract from the overall point being made.

C (70-79) The C essay is adequate in most areas, but exceptional in none. The thesis is clear

although probably lacking in both control and command. Organization may be a

slight problem but can be fixed. The paragraphs provide support but are generally

underdeveloped. There may be multiple errors in tone, format, mechanics, grammar,

and content, but these errors do not, for the most part, detract from the overall

writing.

D (60-69) The D essay is lacking in a majority of areas. It is generally unorganized and

unfocused. The thesis is neither clear nor controls the entire essay. Most of the

essay is underdeveloped. There are frequent errors in tone, format, mechanics,

grammar, and/or content that distract from the content being provided. Its only

saving grace is that, despite all of the errors, there appears to be a legitimate effort put

forth by the writer.

F (0-59) The F essay generally needs little explanation. There are significant problems

throughout. The thesis is often lacking, and the argument, if there is one, wanders

and is unorganized. The essay shows no understanding of basic essay structure, and

there are significant errors in tone, format, mechanics, grammar, and/or content. The

effort on the part of the writer is questionable, at best.



27/27



Date post:	14-Apr-2018
Category:	Documents
Upload:	tondutosuau
View:	215 times
Download:	0 times

Epm5153 Assessment in Education - based on subject topic and finding the index distracter and...

Documents