8/3/2019 4C Standadizing Student

1/9

8/3/2019 4C Standadizing Student

2/9

8/3/2019 4C Standadizing Student

3/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

3

33 SEMESTER 2 STUDENTS OF A HIGHER LEARNING

INSTITUTION WITH LOW ACHIEVEMENT IN ENGLISHLANGUAGE AT POINT OF ENTRY INTO INSTITUTION

No of Students

0

5

10

15

20

A A- B+ B B- C+ C C- D+ D D- E

Grades Awarded

2. CLASS BX - . SUBJECT: ENGLISH LANGUAGE

Overall achievement in progressive evaluation

No of Students

0

2

4

6

8

10

A A- B+ B B- C+ C C- D+ D D- E

Grades AwardedAs it is seen in graphs 1 and 2 above, students with similar

achievements (low) in English Language at the point of entry into the

institution are in two different classes, handled by two different tutors .Their

final achievement in progressive evaluation shows such a vast difference in

grades that have been awarded.

Graphs 3 and 4 below shows the number students, having almost

similar achievement in Pendidikan Islam at the point of entry into the

institution are in different classes, handled by different tutors. Again, their

final achievement in progressive evaluation shows such a notable difference

in grades that have been awarded.

3. CLASS CX - SUBJECT: PENDIDIKAN ISLAMOverall achievement in progressive evaluation

40 SEMESTER 1 STUDENTS OF A HIGHER LEARNINGINSTITUTION WITH AVERAGE ACHIEVEMENT IN PENDIDIKAN

ISLAM AT POINT OF ENTRY INTO INSTITUTION

No of Students

0

5

10

15

20

30 40 50 70 80 90 100

Grades Awarded

4. CLASS DX - SUBJECT: PENDIDIKAN ISLAMOverall achievement in progressive evaluation

No of Students

8/3/2019 4C Standadizing Student

4/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

4

0

5

10

15

20

25

30 40 50 70 80 90 100

Grades Awarded

B. RATER MISS-CONCEPTIONA QUESTION OF RELIABILITY!

1. IN TEACHING PRACTICEEachrater observes 6 students. Each student undergoes 6 - 8 observations.Grade A takes second opinion for confirmation by selected panel

FINAL SCORE

Student No ofobservation

Finalgrade

2nd

opinionGrade

Remarks

A (Boy) 8 A A Sustained

B (Girl) 8 A C +

Unacceptable

degrading

C Girl) 6 B + --- Maintained

D (Girl) 6 B+ --- Maintained

E (Boy) 6 B --- Maintained

F (Boy) 6 C+ --- Maintained

A small portion of discussion in a teaching practice process looks at the

second opinion by a selected panel. The task of the panel is to confirm if it

is a clear-cut A. However, as seen in table B1, the panel member

proposed a C+ for Student B, instead of A that had been awarded by the

supervisor. If that is the case, what would be the status of Student A, and

what will happen to the grades awarded for Students C, D, E and F?

2. SUPERVISOR VS MENTOR

Student Supervisor Mentor

1 Grade A - Score 80 Grade A - Score 85

2 Grade A Score 82 Grade B + - Score 77

3 Grade A Score 83 Grade A Score 95

4 Grade B Score 71 Grade C- Score 58

5 Grade B Score 73 Grade A Score 82

Key:

80 - A; 75 - B+; 70 = B; 65 - C+; 60 - C; 55 - D+; 50 - D;

8/3/2019 4C Standadizing Student

5/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

5

Evaluer Lang

8 marks

Content

8 marks

Organzn

4 marks

Total

20 marks

1 5 5 3 13

2 5 5 3 13

3 5 4 2 11

4 5 5 2 12

5 4 4 3 11

6 5 4 3 12

7 6 6 4 16

8 4 5 2 11

Difference of 5 / 20 or 25 % 5 grads excluding A

The table above represents a session on testing a student from a higher

learning institution on his English Language speaking skills. Based on a

simple 3-criteria scheme of marking, the marks awarded differ between 11

and 16 upon 20. If the mark is converted into a percentage of 100, it runsalong a range of 5 grades.

Perhaps, it is pertinent to mention that there was a session for

marking an essay under the same marking scheme which ended up having a

range of difference from 11 to 17.

2. RECITAL OF SHORT VERSES (Pendidikan Islam)

Penilai Tajwid Kelancaran Fasohah Jumlah

10 mrkh 10 markah 5 markah 25 marksh

1 6 7 2 15

2 7 8 4 19

3 7 8 3 18

4 7 9 3 19

5 8 8 4 20

6 6 7 2 15

7 7 8 3 18

8 6 6 3 15

9 7 7 3 17

10 6 6 3 15

Difference of 5 / 25 or 20 % 4 grads excluding A

Table 2 below shows a similar outcome to table 1 above, for a test on

reciting a short verse from the Quran.

3 Results and Discussion

The graphs and tables in chapter 2 have, to a certain extent,

provided basic facts about the existence of problems in rater-reliability.

8/3/2019 4C Standadizing Student

6/9

8/3/2019 4C Standadizing Student

7/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

7

existence of such differences that kill client-centered in and educational

system, must be deeply explored. The educationists and the professionals

with similar responsibilities should not leave this issue unturned. The

possible reasons should be identified and viable measures should be

proposed, at least to narrow the gap of such differences.

Nevertheless some of the factors that have led to rater differences

and some possible measures to reduce rater disparity are proposed. It is

believed that this would be another beginning to produce true quality human

capital among students of higher learning institution.

FACTORS THAT INFLUENCE RATER DIFFERENCES

(INTER-RATER FACTORS)

1. INTERPRETATION OF THE MARKING SHEME

2. INTERPRETATION OF STUDENTS ANSWER

3. CONTENT KNOWLEDGE

4. EXPERIENCE

5. ATTITUDE TOWARDS EXAMINATION

6. ATTITUDE TOWARDS THE CANDIDATES

7. PRECONCIEVED IDEA

8. STUDENT ATTITUDE TOWARDS THE SUBJECT TESTED

9. STUDENT ATTITUDE TOWARDS THE EXAMINER

10. STUDENT GENDER

11. STUDENTS RACE (with non-prejudiced sensitivity)

12. STUDENT-EXAMINER RELATION

13. STUDENT REAL ABILITY VS CURRENT PERFORMANCE

14. OTHERS

INTER-RATER-DIFFERENCES THAT REDUCE

RATER - RELIABILITY

No. RATER X RATER Y

1. RIGIT WITH THE MARKINGSCHEME

FLEXIBLE WITH MARKING

SCHEME

2. FIRM WITH EVEN MINORSTUDENT MISTAKES

FLEXIBLE WITH MINORSTUDENT MISTAKES

3. WELL-VERSED WITH SUBJECTMATTER (CONTENT

KNOWLEDGE)

SHAKY / WEAK IN SUBJECTMATTER

4. WIDE EXPERIENCE IN MARKING NEW TO THE FIELD OFMARKING

5. SERIOUS IN/WITH EVALUATION RELAXED WITH EVALUATION

6. CONSIDERS TESTING /EVALUATION TO PICK ONSTUDENT WEAKNESS

CONSIDERS TESTING /EVALUATION TO PICK ONSTUDENT STRENGTH

7. TAKES THE TEST TO BE

DIFFICULT

TAKES THE TEST TO BE EASY

8. TAKES STUDENT TO BE HARD-WORKING AND CLEVER

TAKES STUDENT TO BE LAZYMISCHIEVIOUS

9. TAKES STUDENT TO BE WELLMANNERED AND RESPECTFUL

TAKES STUDENT TO BE ILLMANNERED ANDDISRESPECTFUL

10. STUDENT CLEVER BUT ERRED IN

THE CURRENT TEST HELP

STUDENT CLEVER BUT ERRED

IN THE CURRENT TEST DONTHELP

11. FEELS - ACHIEVEMENTPERCENTAGE MUST BE PRE-SET

FEELS - ACHIEVEMENTPERCENTAGE MUST BECONCURRENT WITH TEST

PROBABLE APPROACH TO STANDADIZATION

TO NARROW THE GAP IN RATER DIFFERENCE

8/3/2019 4C Standadizing Student

8/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

8

No. PROBLEM SOLUTION

1. INTERPRETATION OF THEMARKING SHEME ANDSTUDENT ANSWER

DISCUSS, FINALIZE ANDEVERYONE ABIDES BY THEDECISSION MADE

2. TESTER KNOWLEDGE ANDEXPERIENCE

UPDATE CURRENTKNOWLEDGE

(3 R APPROACH)read, research and react

3. ATTITUDE TOWARDS

TESTING

DISCUSS AND FINALIZE

(ABIDE)

4. ATTITUDE TOWARDSSTUDENTS

DO NOT BE BIAS OR INCLINED

5. PRE-SET IDEA / MIND DO NOT HAVE PRE-CONCIEVED IDEAS /

NEGATIVE MIND-SET

6. STUDENT ATTITUDE AND

ABILITY TOWARDS THESUBJECT

EVALUATE ON THE CURRENT

PERFORMANCE(NOT THE GENERAL ATTITUDE

/ ABILITY)

7. RACE AND GENDERBIASNESS

DO NOT HAVE RACIAL ORGENDER BIASNESS /INFLUENCE

8. TUTOR STUDENTRELATIONSHIP

NO INFLUENCE FROM T-SRELATION

9. VETERAN FRESHIE

ATTITUDE

I KNOW WHAT IM

DOING

MIND YOUR OWN

BUSINESS

I BOSS OR YOU BOSS?

OVERCOME NEGATIVE. USE

OF

VETERAN FRESHIE

ATTITUDE

BE PROFESSIONAL &

TELERANT IN SOLVINGPROBLEMS

10. THE BUSINESS OFSEGREGATION MY

STUDENT YOUR

STUDENTS ARECOLLECTIVELY

THE CLIENTS OF ONE

STUDENT ORGANIZATIONTHINK POSSITIVE ABOUTCOLLECTIVE WELL-BEING

11. STUDENT STANDARD IN THESUBJECT AT WHAT

LEVEL?

DECIDE THE SUBJECT AS

THE CORE OR SUPPORT

12. HIDDEN GOVERNMENTPOLICY

FOLLOW YOU ARE THESERVANTS

Acknowledgement

The paper has been developed with high academic sensitivity and

sincerity. Data has been collected and compiled over a long period of time,

grabbing opportunities to dwell into the root of the problems related to

evaluations. Data collected and provided is authentic. It is a true reflection

of the existing situation. Relevance of sources in relation to data may be

sensitive and it may lead to a certain level of disgust disgrace to some

institutions. Therefore, it is very much regretted that none of the identical

sources would be revealed to the readers / audience. Many of these

suggestions may sound to be an approach to solution within (intra-raters) aninstitution. Thus, the ball has been laid to be rolled by the most respected

educationists in their own field of professional tasks, in an attempt to

provide solutions for inter-raters (between institutions). Thank you.

References

8/3/2019 4C Standadizing Student

9/9

Persidangan Pembangunan Pelajar Peringkat Kebangsaan 2008

Universiti Teknologi Malaysia, 22-23 Oktober 2008

9

1. R.C. Bogdan and S.K. Biklen. Qualitative Research for Education.

Boston. Allwyn and Bacon. 1982

2. K. Hodgkinson. Student Perception of the Personal Relationship Involved

in Teaching Practice. Louhborough University of Technology. 1993

3. K. Zeichner and J. Gore. Using Action Research as a Vehicle for Student

Teacher Reflection. Oxford, Pergamon. 1995.

E. Saunders and C. Sauders. The Assessment of Teaching Practice. Ulster

polytechnic, Faculty of Educatiuon. A report of Work Carried Out Between

1975 - 1987