Training-Measurement & Evaluation

8/14/2019 Training-Measurement & Evaluation

1/17

Measurement &Evaluation

Foundation Education& Business Solutions Ltd

(FEABS)25, McNeil Road, Sabo Yaba, Lagos

[email protected], 01-3425462

By


2/17

Introduction

Measurement is a process of assigningnumbers to the individual members of a set

of objects or persons for the purpose ofindicating differences among them in thedegrees to which they possess thecharacteristic being measured.

On its own, evaluation is the process of

making value judgements for the purpose ofdecision-making. It is simply a processthrough which value judgements ordecisions are made from a variety ofobservations or test results. It also involvesthe inspection of all available information

concerning the student, teacher and entireeducational programme for the purpose ofmaking valid judgments about the degree ofchange in students and the effectiveness ofthe educational programme.Training Focus

The focus of this training is therefore toexpose teachers to rudiments ofmeasurements and evaluation so thatwhatever judgements or conclusions to bedrawn about the educational enterprisegenerally will be valid as much as possible.

8/9/2009 2Foundation Education &Business Solutions Ltd.


3/17

THE INSTURCTIONAL OBJECTIVES

Instructional objectives clarify the learning

outcomes expected to result from teaching.

It is a statement that describes in behaviouralterms what the students should be able to do,the conditions under which the task is to beperformed and the criterion for acceptable

performance. Thus, the teacher usesinstructional objectives to describe studentsterminal behaviour after a given unit ofinstruction.

Classification of Objectives

According to Blooms Taxonomy of objectives,objectives can be classified into three broadareas namely:

Cognitive Domain

Affective Domain

Psychomotor Domain



4/17

S/N CLASS OFOBJECTIVES

AREA USEFUL VERBS

1 Cognitive Domain -Knowledge

-Comprehension

-Application

-Analysis

-Synthesis

-Evaluation

- Define, list, reproduce, state, name, identify, enumerate,

etc.)- Convert, explain, defend, distinguish, generalise, rewrite,reorder, predict, etc.- Change, discover, operate, demonstrate, produce, relate,solve, compute modify, calculate, etc.

- Break down, distinguish, deduce, separate, identify,differentiate, illustrate, infer, point out, select, sub-divide, etc.- Create, generate, rearrange, reconstruct, reorganize, plan,revise, compile, devise, design, modify, etc.

- Appraise, compare, contrast, evaluate, decide, interpret,

justify, conclude, support, relate, discriminate, etc. )

2 Affective Domain -Receiving

-Responding

-Valuing

-Organization

-characterisation

-ask, choose, describe, follow, give, hold, identify, locate,point to, select, sit erect, reply, use, name, etc.-answer, report, perform, recite, write, greet, practice, report,comply, perform, present, select, conform, label, read, tell,etc.-complete, differentiate, initiate, propose, share, explain,invite, read, study, etc.

-identify, adhere, compare, generalise, order, synthesise, alter,complete, identify, organise, arrange, defend, integrate,prepare, combine, explain, modify, relate, etc.-display, act, propose, serve, discriminate, modify, solve,display, perform, question, use, influence, practice, revise,verify, etc.

3 PsychomotorDomain

-Gross bodilymovement

-finely coordinatedmovements

-Non-verbalcommunicationbehaviour

-Speech behaviour

-throw, catch, jump, match, kick, run, swim, dive, box, dance,etc.

-hand-finger coordination, hand-eye coordination, hand andear coordination, and hand, eye, foot and ear coordination-using the hand and arms to communicate with the deaf,convey a given message to a panel by gestures, etc.

drama, speech, foreign language and speech correctionalprogrammes. Objectives include the ability to recite a poem orproduce specified musical note.

Classification of Objectives



5/17

Characteristics of well-stated Instructional Objectives

From what has been said so far, well-stated instructional objectives must possess the

following features:

They must describe in behavioural terms what the students will be able to do when

the instruction has been completed.

They must describe the conditions or circumstances under which the learned

behaviour will occur.

They must describe the extent to which the specified behaviour can be expected,

i.e. they suggest an acceptable criterion level for performance.

Examples:

At the end of the lesson, students should be able to:- define economics- explain some of the basic terms associated with economics- list measures of central tendencies.

Questions

&

Discussions



6/17

Test as A Measuring Instrument

A test is a set of standardized questions or inventories

administered to an individual for the purpose of measuring or

obtaining quantitative information about several aspects of the

individuals behaviour.

It can also be defined as an instrument, a device or a procedure

that proposes a sequence of tasks which a student is to respond,

the results of which are used as a measure of a specified trait.

Tests generally are useful for prediction, selection,

classification and evaluation, hence it is important to ensure that

it is highly reliable.

TES

T

CRITERIA FOR A GOOD TEST



7/17

CLASSROOM TESTS

Teacher-Made Achievement TestsTeacher-made tests are the tests constructed, administered andscored by the classroom teacher or possibly by a committee or

several teachers in the same school.The teacher-made tests are constructed by a teacher to measurelearning outcomes of students in his classroom.The items are built around definite objectives specifically taughtfor in the class.Teacher-made tests are usually based on the content of thecurriculum of a particular course or school.

Planning Teacher-Made TestsIn planning for teacher-made achievement tests, the followingprocedures or activities are important.



8/17


9/17

Example of A Test Blueprint for an Objective Testin Economics

DEVELOPING A TEST BLUEPRINT

Before developing test items, there is need to develop a testblueprint (or table of specification). The test blue-print is a 2-waytable that relates the units of lesson or content area to the levels

of cognitive domain objectives at which these contents have beenlearnt. It provides the operational blueprint that guides the testbuilder and ensures that he builds a test that will do what it set todo. The purpose of test blueprint is to define as clearly as possiblethe scope and emphasis of the test and to relate the objectives tothe content. The test blueprint helps to achieve content validityand provides groundwork for building achievement tests.

The blueprint specifies precisely what weight to give each topic

and each instructional objective. There is no fixed rule that can beprescribed to the teacher to use in determining the weights to beassigned to the various cells in the table of specifications. Usually,the assignment of weight depends on the teachers subjective

judgement in terms of the relative importance, time devoted toteaching the topic and the relative emphasis used by the teacherwhen he taught the course. The Table below shows an example ofa test blueprint



10/17

MEASURABLE VARIABLES

VARIABLE DEFINITION CHARACTERISTICS

STANDARDISEDACHIEVEMENTTESTS

An achievement test is a testthat measures the extent towhich a person has achievedsomething, acquired certain

information, or masteredcertain skills usually as aresult of planned instruction ortraining. Achievement testsattempt to measure what anindividual has learned, i.e. hispresent level of performance.

-It is constructed in accord with detailedspecifications-Items are selected after tryout forappropriateness in difficulty and

discriminating power,-It is accompanied by a manual givingdefinite directions for uniformadministration and scoring,-It is provided with relevant anddependable norms for scoreinterpretation.-They are those tests whose procedurefor administration and scoring has beenmade uniform or standard.-By and large, standardised achievement

tests have, almost without exception,substantial reliabilities near, or into, 0.90range.

APTITUDETESTDifferentialtestWork-sampleaptitude testAnalogous testComponentability test

Aptitude tests are those teststhat measure an individualspotential to achieve in a givenactivity or to learn to achieve inthat activity. The purpose ofaptitude testing is to predicthow well an individual willperform on some criterion (suchas school grades, teachersratings or job performances)before training or instruction isbegun or selection or placementdecisions are made.

-It involves both innate and acquiredabilities-It measures non-deliberate orunplanned learning-It measures a persons capacity to excelin the future based on presentperformance.-Aptitudes are based on psychologicalfactors known from empirical studies toaccount for good performance in theactivity under consideration.

INTERESTVocational

InterestInventories

Interest can be defined astendencies to participate in an

activity. It is a preferentialtreatment given to a particularactivity.

The vocational interestinventories comprise theactivities and performance ofpeople in different occupations.

-The individual is required to indicate hisinterest in those activities

-This is followed by the assignment of anempirically determined weight to theindividuals responses-The performance of the individual onthe inventory is then compared withthose who have been successful in theoccupations.

PERSONALITYTESTPaper-pencil

self-reportinventoriesProjectivetechniques.

Personality refers to theaffective or non-intellectualaspects of behaviour. It is the

organized system of behavioursand values that characterize agiven individual and account forhis particular manner of

-Personality tests are instruments formeasuring the affective or non-intellectual aspects of behaviour for

personal counselling-They are used to measure such aspectsof personality as emotional stability,friendliness, motivation, dominance,



11/17

DevelopingEssay Tests- A mode ofAchievement Tests

Construction Guidelines

The following guidelines are useful in settingessay tests.

Describe the specific objectives which thequestions are to measure.

Avoid open-ended questions.

Word the questions in such a way that thecandidates will interpret them in the sameway.

Give a clue as to what you expect from thestudents to make for uniformity.

Be mindful of time. Attempt answering thequestions you set.

The essay items should be of moderatedifficulty, not too easy, but difficult enough

to pose a challenge.

Give adequate time and thought to thepreparation of essay questions.

The questions should be written so that itwill elicit the type of behaviour to measure.

Phrase the questions with the action verbsappropriate to the relevant instructionalobjective in the cognitive domain.

Do not provide optional questions on anessay test or better still be minimised.

Use a relatively large number of questionsrequiring short answers rather than just afew questions involving long answers.

Do not start essay questions with suchwords as list, who, what, and whether.

These words tend to elicit responses thatrequire only a regurgitation of factualinformation.

Adapt the length of the response and thecomplexity of the question and answer tothe maturity level of the students.

Prepare a marking scheme for scoring theanswers.

Definition

Essay tests consist of a list ofquestions for which thesubject (student) is requiredto write out the answer.

An essay item is a questionor situation with instruction,which requires the testee toorganise a complete thoughtin one or more writtensentences. The testee isgiven freedom to generateresponses which must beassessed by a scorer who isknowledgeable in the subjectarea.

Essay tests are best suitedfor measuring studentsability to originate andintegrate ideas, their depthof knowledge andunderstanding, verbalexpression, creativity andhigher thought processes.

Choice questions are oftenprovided so that all thetestees do not necessarilyhave to answer the same setof questions.



12/17

Developing Objective Tests- A mode of Achievement TestsDefinition

Objective tests are tests in which every question is set in such a way as to have only one right answer. The items areconstructed in a way as to have one, predetermined correct answer. Objective tests are called objective becausesimilar answers by different testees are given the same marks, no matter who did the scoring.

Type Definition Guidelines

1. Short-AnswerItems

The short-answer item(also called the supplyanswer or completionitem) presents a task ina sentence in which aword, a number, a

symbol, or a series ofwords has been omitted.The items call for onlyone response for a blankor a specific series ofresponses for a series ofblanks.

1. For computational problems, the teachershould specify the degree of precision andthe units of expression expected in theanswer.

2. Omit only essential or important words in asentence.

3. avoid excessive blanks in a single item. Theteacher should not eliminate so manyelements of a statement that the itembecomes ambiguous and confusing.

4. The blanks are typically placed at the end ofa statement rather than at the beginning.When the blank is placed at the beginning ormiddle of the sentence, the essential point tothat question may be overlooked orforgotten by the time the student reads the

item.5. Make all the blanks the same size regardless

of the answer.6. To test for the knowledge of definitions

and/or the comprehension of technicalterms, use a direct question in which theterm is given and a definition is asked for.

7. Avoid giving irrelevant clues to the correctanswer in the structure of the item.

2. TheAlternateChoiceItems

In the alternate choiceitem, the students aregiven two options tochoose one. Suchoptions includeyes-no,true-false, right-wrongand correct-incorrect

Avoid the use of specific determiners, i.e. wordsthat serve as special clues to the answer.Avoid verbatim quotes from textbooks.Each statement should be entirely true orentirely false.Avoid trivial details in order to make astatement false.Avoid negative statements as much as possible.Avoid controversial statements, or else quoterelevant authority.Avoid a pattern of answers. Let the number of

true statements be approximately equal to thenumber of false statements.8/9/2009 12

Foundation Education &Business Solutions Ltd.


13/17

Developing Objective Tests (Contd)- A mode of Achievement Tests

Type Definition Guidelines

3. TheMatchingItems

The matching itempresents two lists usuallycalled the premise andresponse. The premiseconsists of the questionsor problems to beanswered, while theresponse list contains theanswers. Examples ofsuch pairs of sets arenations and capitals,nations and major

exports, terms anddefinitions, dates andevents, etc.

1. Use only homogenous premises and homogenousresponses in each matching exercise, i.e. allshould refer to date, all to names, all to places,and so on. Be sure the student knows the basison which the terms should be matched.

2. Avoid having equal number of premises andresponses.

3. Arrange the numbers in systematic fashion suchas alphabetical order, dates and numbers, eitherin ascending or descending pattern.

4. Avoid extraneous irrelevant clues.5. Maintain grammatical consistency.

6. Every response in one column should be aplausible answer to every premise in the othercolumn.

7. All items and options for a given matchingexercise should be on a single page.

4. TheMultiple-Choice

Items

The multiple choice itemsconsist of a stem and abranch. The stem

presents the problem aseither an incompletestatement or a question,while the branch presentsa list of suggestedanswers (responses oroptions). There areusually four or fiveoptions. Among theoptions, only one is thecorrect answer (called thekey). The incorrect

options are calleddistracters. A distracter isa plausible but wronganswer designed toconfuse the student whodoes not know the correctanswer.

1. State the instructional objectives and content tobe covered.

2. Provide at least four options but not more than

five.3. There should be one correct response.4. The question to be answered must emerge clearly

from the stem.5. All distracters should be plausible and attractive

to students who do not know the correct answer;yet they should be incorrect.

6. Each item should be independent. One itemshould not aid in answering another item on thetest.

7. Avoid irrelevant clues to the correct answerprovided by response length, repetition of key

words, common associations or grammar.8. Avoid a pattern of answers. Each letter should

appear almost the same number of times on thekey, the letters being in random order.

9. The stem and options must be linkedgrammatically.

10. Positive rather than negative stems ought to beused or negatively stated items should beminimised.

11. Minimise the use of all of these, none of these,as options. Avoid using them both for the sameitem.

12. Avoid verbatim quotes from textbooks, lecturenotes and journal articles even though test itemsmay be based on materials from these.



14/17

Objective TestObjective tests are easy to score. They can be hand-scored or machine-scored. For machine scoring however, special answer sheets must be

used along with special pencils for answering.

Models of Scoring Objective Tests

Many models have been developed for scoring objective tests but themost commonly used in Africa are:

Rights onlyHere each question on the test carries equal marks, usually one point. A

candidates final score is simply the number of items marked right.

Correction-for-guessingThis system is based on assumptions such as :

I. that students who are ignorant of some objective test itemsobtain some correct answers through guessing;

II. that all the options per item are equally attractive to theignorant students

III. that the more a student knows, the less he guesses.

Since guessing introduces an error score, the items obtained throughguessing are to be deducted from examinees marks. This penalty forguessing is carried out by the application of the correction-for-guessingformula which states:

S = R W/(n-1) where:

R = number of questions marked rightW= number of questions marked wrongn = number of options per itemS = score after correction

This model is applicable only to multiple-choice and true-false (n=2)tests and to the number of questions actually attempted by thecandidates. Whenever this correction is to be applied,, students shouldbe warned in advance with test instructions such as:

attempt only those questions you are sure ofno wild guessing allowed, etc.

SCORING CLASSROOM TESTS



15/17

Essay Test

The essay test should be restricted as much as possible to

the measurement of the educational objectives for which theobjective tests are inadequate. These include the evaluationof verbal expression, depth of knowledge and understanding,originality, creativity and higher thought processes.

It should be noted that teaching which emphasiseswholeness of learning and insightful understanding oflarge concepts are best evaluated with essay tests while the

connectionist approach which emphasises learning in smallparts (frames) is most adaptable to objective examinations.

The major uses of the essay are in areas where the objectivetest tends to be unsuitable and particularly at universitylevels. It is also used for appraising attitudes and as aprojective technique for appraising some personality factors.Guidelines for Essay Test

1. Essay questions are best suited for assessing higherthought processes for which the objective test isweak.

2. Ensure that every question is relevant to the course-content.

3. Each question should be of moderate difficulty levelfor the examinees.

4. Be mindful of the time available for testadministration and the scope for each questionshould be accordingly limited.

5. Avoid vagueness. Each question should pose adefinite problem, be brief and be expressed in wordswithin the examinees vocabulary.

6. Prepare a marking scheme for each question as anaid to marking and reducing vagueness.

7. Optional questions should be avoided or minimised.8. Some leading useful leading verbs in setting essay

questions are: compare, contrast, distinguish,explain, list, solve, demonstrate, differentiate,evaluate illustrate.

SCORING CLASSROOM TESTS (Contd)



16/17

Test interpretation entails assigning meaning to scores, explaining thecircumstances that gave rise to such results and the probableconsequences of such results.

Guidelines for interpreting test results

1.The counsellor must know the test thoroughly.2.Explore the clients reason for wanting tests and experiences withtests. The clients expectation are important data to be considered,since he may be expecting much more than the tests can reasonablybe expected to perform.3.Structure the test interpretation session to prepare the client formeaningful, undistorted information. The counsellor must ascertain theclients knowledge of the limitations of testing as well as the values forgiving various self-information.4.The meaning of the scores should be established early in thediscussion, i.e. the client should know clearly what type of measure isbeing discussed interest, aptitude, achievement or personality.5.The frame of reference of the test results should be specified clearly,i.e. the client must know at all times which form group is being used sothat he can have some yardstick for judging how high a high score

really is.6.Test results, not scores, should be give to clients. The counsellorshould not use exact scores in interpretation. Rather, he should usedescriptive phrases.7.Test results should always be verified. The counsellor should neveruse a score in isolation. Test results should be woven in with other casedata to check on the validity as well as the reliability of the test score.8.Counsellor neutrality in imparting test results. The test should speakfor themselves without the counsellors personal value projections.

9.Meaningful and clear interpretation should be the counsellorsconstant aim.10.Accurate prediction is the principal aim of testing. Test resultsshould be stated in terms of statistical predictions.11.Client participation and evaluation should be encouraged.Interaction between client and counsellor is valuable as it gives thecounsellor constant information regarding how the client is receivinginterpretation.12.The interpretation of low scores to normal clients is one of the mostdifficult problems the counsellor faces. However, the counsellor may

test the clients readiness to accept low scores by asking the client topredict what he thinks his relative ranking might be on the trait inquestion.

TEST INTERPRETATION



17/17

Questions

&

Discussions

8/9/2009 17Foundation Education &

Date post:	30-May-2018
Category:	Documents
Upload:	onibon-ademoa
View:	222 times
Download:	0 times

Training-Measurement & Evaluation

Documents