DOCUMENT.RESUME

ED 113)395

AUTHOR Aleamoni, Lawrence M.TITLE Typical Faculty Cpncerns About StudentJByaluation of

Instruction.. _4

PUB DA7E 24 Mar 74NOTE 23P.; Prepared for presettation,at the ',Symposium"

Methods of Improving'University,Teaching at theTee Inion, srael Institute of Technology, (Haika,0`Idrael March 24i 1974)

/

HE 006 776..

4

EDRS PRICE 11-$0.76 HC-$1.58 Plus PostageDESCRIPTORS College_Students; Conference Reports; *Course

Evaluation; Faculty Evaluation;.*.Higher Education;Participant. Satisfaction; .Questionnaires; Student,Attitudes; Student Evaluation; *Student Opinion;Summative Evaluation; *Teacher Rating; *teachingQuality; Teaching Techniques -

IDENTIFIERS *Illinois Course Evaluation Questionnaire

ABSTRACTSeven typical facUlt concerns about the

'appropriateness of using ratings of ins uctor and instruction are'examined. These are summarized in terms of common observationgfrequently expressed by faculty: (1) Students cannot make, consistent

1 judgments because of their immaturity, lack of experience, andcapriCiousness; and a widely held belief is that only Colleagues withexcellent publication' records and qxperienbe are qualified toevaruate their peer's instruction. (2) Most. student rating schemesare nothing more than a popularity contest. /3) Students are 'not ableto make accurate judgments until they have,been away from the courseand possibly from the university for several years. (4) The studentrating forms are unreliable and invalid. 5) There are extraneousvariables or conditions that can affect student ratings, includingclass size, Schedule, term, student sex, major, and level. (6) Thegrade a student receives in the course is highly correlated with hisrating of the course and the instructor. (7) A, question fiequentlyraised is how student evaluations can be used to improve instruction.Research addressing these problems is cited that makes use of the-Illinois Course ,Evaluation Questionnaire (CEQ) . (LBH)

4

*.***************************************e*****************************Docubents acquired by ERIC include many informal unpublished *

* materials not available from other sources. ERIC makes every efforst ** to obtain the best copy available. Nevertheless, items of marginal .**'reproducibility are often encountered and this affects the quality *

* of the microfiche and hardcopy reproductions'EPIC makes available *

* via the ERIC Document Reproduction Service (EDRS). EDRS is not* responsible for the quality of the original document. ,Reproductions ** supplied by EDRS are the best that can be made from the original. *

**********************************************************************

4

4l

U S DEPARTMENT OF NEALTFIEDUCATION a wE0A/tENATIONAL INSTITUTE OF

EDUCATIONTHIS DOCUMENT HAS REIN REPROOuCED ExAcTCy AS RET EREO .ROMTHE PERSON OtTOR(.NiZATONATING IT POINT% e)T vt vY OR OW,. ONSSTATED DO NOT NE( ESSAR itE pkESENT OFT ( A. NAT:ONAt TT OTEDUCATON POST OT,, OP POL.Ck

.

-I

TYPICAL FACULTY CONCERNS ABOUT

1STUDENT_EVAlUATION OF INSTRUCTION

,LawTence M. Aleamoni2

1-`In the past few years there have been many propOS,Is for evaluating

instruction and afew of them were aso concurned with trying to

evalUafton to the improvement of -instruction. Most proposals suggested

the use of similar elements in ihe.evaluation procedure; These inclu e.

(a) judgment by student, peer, self, and supervisor (department- hea

and (b)-judgments;of(course material, coarse content, courde'obj ctives,

and quality of student learning: ',If, however one looks for aetual,

o

working models of instructional evalliation,0iCis iMmediatelyapparent

that schemes involving systematic ratings by peer, supervisor, or self,

oy of material, tontent, etc., are rarely actualized. More often than

not, the student ratings of instructor and instruction appear as.the

only ele ments in any of the "working models" and .there are many reasons

.

.

one could-cite for this. This paper, however, wil focus specifically1

'4 ,--,k on seven typical faculty concerns aboUtth .appro riateness of using. ,.

-stings of instructar'and instruction, at all: These are summarized

below in terms of common observations f equently expressed by faculty.

1. (a) Students cannot make consistent judgments concerning

the instructor And'nstruction because of their immatur-

ity,ity, lack ot experience, -and-capriciousnesa.

1

Ns

Conversely,

- 1An invited address- Xlrepered for presentation in the Symposium on

Methods of Improvi.Pg Liniversity Teaching at the Technion IsraelInstitute of Technology, Haifa4 Israel on March 24, 1974.

2The author is indebted, to Robert H. Davis and Joseph S. Marcusfor their constructive suggestios,s 1

5,

2.

(b) A wigely.heldbelief.ia that ,only colleagues. with exgell-

ent 'publication records and' experience are qualified to,'

evaluate their peer's instruction.

2. Most student rating schemes are netkiAg more than a popularityf

contest, with The warm, friendly, humorous, easy grading..

Akinstruc.tor emerging as the-winner.,

', :

r.

3. Students are dot able to make 'ccurate judgmenEs until they

fiAI.

have ben .amay from the course and possibly away from'tha uni-

veraity for several years.

4. The student rating forms are both unreliable and invalid.

5. There. are extraneous4Var.iablesor° conditions that can affect

* ° .

student ratings. Some of the more common

(a) t he .size of the class

(b) .the-sex of the student

P(c) the timeof. day the course was offered -

was taking the course ha a re irement

N,.

(d) Whether the stude.$11

...

, .

,. ,

. .

or on an elective basis %-.C? . N

(a) whether ,tte student was a major or .a riion-major'

,-7 (f) the term (of'semester) the coursewas offered

tgi the level of the course (freshmen, sophomore, junior,

senior, gradUate)

(h) 'the tank of the instructor (instructor, assistant

Ones are:

/

.professor, associate prOfessor, full profeseor)

6. The grade or mark a student receives ifi the course is highly

correlated with his rating the course and the instructor;

7. Finally, a question (hat is frequently raised is "How can

student ,evaluations possibly be used to improve instruction?"

. \k/

a

Noe

s

fr

A)

I

-_Surprising as it may be, answers.to'these problems and questions4

cam-be found, in a 4ethon4a. Of research' that spans' at lease 50 years.v- .

Most m; this research hailoleen conducted .using student evaluation \(rating) questionnaires similar to tpe one presently Used at the UM'-

4 varsity of Illinois at Urbana-Champaiiii-X0IUC)- and entitled the

Illinois Comxse Evaluation Questionnaire (CEQ). A copy of the-CEQ

is presented inAppendix A.

Befpre beginning to cite and summarize,the reseadfch addressing

each of the ptoblems and questions above,

) .-

CEQ will be presented ins. order tp provide

ence forth various studies.

a brief description of the7='w

a Meaningful frame. of refet-

The CEQ is an'instxjnment used to collect studenx attitudes and

opinions toward a course (Aleamoni, 1972a;Aleamoni and Spencer, 1973).:

Its purpolse is to enable faculty members to collect evaluative infor-

mation about their teach The data is collected and proceased by

course-

section but may'a so be processed by course, department, c liege,

etc. ,Extensive normativ4 data has been gathered on the CEQ over the

past 10-years and this p ovides the instructor with valuable comparisons

on the\inseructional dimensions of (a) Generil Course Attitude, (b) Course

stroction, (d) Interest and Attention of the

tructor.\ These instructional dimensions repre-

EQ. The 23 items thia:make up the aubscales

some diagnostic feedback to the..instructor

cbmparisons. For example, each instructor's

content, (c) Method of I

Students; and (e) the In

sent thesubscales of th

are used to help provide

through appropriate norm

results are compared with those of pther instructors of his own academic

C.

1

,rank, teaching at .the..dame course level and in his own department,

-college o university, as'well as'with, co44es thot have used the' CEQa,

throughout the United Stdres.

The student-responses to the CEQ ore anonyrdbusi and two copies

4.

of the quesFloPnaire6,esults (see Appendix B) along with interpretative

information are returned` only the instructor. The instructor Petty.e

decide, to submit ene copy to his department chairman for, rank, pay, an&...

tenure consideration, but the CEQ is primarily 'used to prpvide feedbek4

to instructors as to where potential problema may exist in the classroom.ra _.

. .._ . /*,. .

.

.

-'There is ample space on the CEQ form for instructors to utilize

..

).

specifically diagnostic items in identified,problem areas. Co9ferences-

J

relatille to-the interpretation ano(utilieatio° of results may be

arranged at the instructor's convenience with.consultants from the ..

Measurement-and Research Division of the Offisce of InStructional Resource's

at,UIUC. U

With that brief description of the CEQ as backgrounds responses to

the (seven concern are now presented in the order of their presentation.

above 2'

'1. (a) Students cannot vonsistentl 112181 instructor andiinstruc-.

tion. There is amp/e-evidence on thisSpoin; dating .back-to 1924,

according to 'Guthrie (1954), in which reliabilities of student ratings.

-remain in the .80 to .90 range. More reCent'literature on the subject

by Costin, Greenough add Menges (1971) and Aleamonti (1972b) pas shown

that well developed Instruments and administration procedures can

silently yield'high reliabilities in the .90's).

.J %

(b) Onl ollaa ues with excellent.publibation records and eikieri-'

5;

1.ences are qualified .to evaluate their peer's instruction'. Wherever a

diAtia;ion about or a proposal for student ratings emergrs, this tate-

6ment can usually be heard. Recently, a well known statistician presenteci

just such an agrument in The American.Statistician (Deming, 1972).

Fortunately for un fortunately for those who believe such acontention), e

a bout the time. that Professor Deming's article appeared a study had been'

campleted'addressink that very, topic. Aleamoni and Yimer (1973) found

that colleague and student ratings were not significantly relAted to the

ainstructors research productivity (the correlations were .07 and -.04,

respec;ively). In addition, there was ample evidence previously (Guthrie,. ,

1954; Stallings and Spencer, .1967; Swanson and Sisson, 1971) to show that

:colleague and student ratings were very highly correlated (from-.63 to

. .765. Theresponse:to Professor Deming's arttcle can be found' in a

eater issue'of The American Statistician (Aleamoni, 1972),.

2. MOst student rating schem p.. are nothing more thana popularity

contest. Answers to this problem are presented from published and lib-.

published studies, on the CEQ and The Advisor (a student- sonsored fotm)-

(Feldman, 1970) at UIUC. The studies conducted, in developing and uti-,

.t'

lining the CEQ subscalca, (Aleamoni. and Spencer, 1973) indicated that no

single subscale (t.e., Method Of Instruction)' completely overlapped the

other subscales. Basical this means that an instructor vho received

a high decile rating on the Instructor subscale (made up of items like

"The insfitie-terseemed to be interebted'in students as persons.") wohld

not be guaranteed high decile ratingolon the other four subscales4

qe \,

itt

a

6.

V

(General Course Attitude, Method of Instruction, Course Content, and

Interest and Attention). 'In order to more fully explaxe this problem,

the written comments made by students pn both -the CEQ aridThe Advisor

were revUwed and comOared to their objective responses. The results

indicated that students would frankly praise an instructor lor his

warm, friendly, humorous, etc., manner in the classroom, but if his

,

course was not well organized or his method of Stimulating students

to' learn was poor, the students would equally frankly criticize him,

in those areas. ,When these comments were compared to the objective

measures in the same trees, a high degree of relationship was observed.

This'evtdence, in additjon to.that presented b ostin, et al., (1911),

indicates that students are discriminating udges and not easily fooled

'by the gobd Vshowl'utm" who is lacking in the otlier.ilistructional quali-

ties. #

3. 'Students cannot make accurate judgments until they are out of

the course an away from the university fOr several years. This point

is repeatedly raised by faipulty and was recently presented by McKeachie

(1969). This problem is a very difficult one with which to deal becausef . P

longitudinal;Collow-up studies need a great deal of attention to the

question of whether samples are comparative and representative. The

sampling problem is further compounded by the fact that almost all stu-

dent attitudinal data relating to a course or instruttor is-gathered

anonymously. Most studies in this area, therefore, have relied on sur-

veys of alumni and/or grfilduating seniors.

One of the earliest studies by Drucker and Remmers (1950, 1951)

showed that alumni who were out of school five te, ten years rated the

a

SI

a.

4 order to answer this problem it Rust be divided into two portions, one

concerting the reliability of student rating forms and the other; the

a

instructors much the same as students currently enrolled. More recent

evidence by Aleamoni and Yimer (1974) ,further substantiates the earlier.

findings. This evidence seems to indicate, contrary to popular belief..,..

d speculation, that students are veit-TerCeptiVe in,thekrjudgments

and,are in'substantial agreement with peers who have been out of thea

course and away from the university for several years. ,Avery care-

fully controlled follow-up study, however, needs to be conduotCd in. ;

order to fully answer this problem.I,

4. The student rating forms are .oth -unreliable and invalid. In

1-5

validity of student rating forms.

Thereseerch ,literature it replete with studies that answer the

question of.the reliability or sudent rating forms. Almost all of the

instruments which have been carefully constructed andtesteddby pro-

fessionals yield reliabilixies at.the level of .80 ,and .90 on thea

subecales as well as the total instrument (Costin, et al., 1971).

COmpue6d on'the items and subacales makinm up the CEQ

(Aloamoni, 1972b), for example, have yielded item reliabilities rang-a

ing from .73 to .94 andubscale reliabilities ranging from .80 to

.98. It should' be'moted, however, that wherever student rating forms

are not carefully constructed with the aid of professionals, as in the

case of mast - student generated forms (Everly and Aleamoni, 1972), the

reliabilities may be no low as to-completely negate the evaluation

effect and its results.

Trying to answer the mblem of'th validity of student rating

forms is much more difficult than adtcessingthe reliability problem.

/In order to something one must have a "criterion measure for

/

comiarison. One of the criterion measures that can be-used to vali-

date student rating forms is to determine how well the items and

subscales measure what is intended (called conternt validity). This

is usually accomplished by carefully constructing the instrument so.

that it contains'items and subscales that will yield measures in the

areas that are considered necessary by an individual or group of

experts in the Field under consideration. Most of the. student rating41

forms generated,were validated by'using just this sort of approach

(C tin, et al., 1971). The use of statistical tools like Factor

Analysis have also been used to verify sub ectrvely determined dim-

ensions of the instructional setting und process. The C*() used both4

statistical (Factor Analysis) and 'subjective expert judgments in

.gener61Cing the items and sOscales that make up the form (Aleamoni

..ad Spencer, 1973),...

Many other criterion measures have been suggested by which to

vuidate st,..dent ratings. Some of those are pde (or colleague rat-

;lags), expect judb.S' ratings, student learning, etc. You will

notice that many of the problems and questions that faculty pose,

such as the seven suated above can also be interpreted as validity4

concerns. In order to avoid any redundancy of answers to the faculty

concerns, let me simply indicate that studies in which student rat-

ings were c(4p0led to (a) cglleague ratig (Guthrie, '954; Swanson

and.Sisson, 1911; Aleamoni and Yimer,,1973), (b) expert/judges'

9

ratings (Sta

..

lingo and,

Spencer, 1967), and(

9.

) student learning measures

. )roiten and erger, 1970) all indicated the existence of high .positive.

correlations which can be considered as providing additional validity

N .

evidence.

5. What extraneous variables or conditions affect student ratingd?

Studies conducted on the (a) size of the classv(Guthrie, 1954; Costin,

et al., 1971; Aleamoni and Graham, 1970, (b)" sex of the student (Costin,

et al., 1971; Aleamoni, 1972b),' (c) time of day the course was Offered

4(Aleamoni, 1972b), and (d) term (9 semester)' the course was offered

(Costin, et al., 1971; Aleamoni, 1972b), indicate that'these variables

had little or no relationship to the student ratings. The,xank of the

instructor (Guthrie, 1954; Costin, et al., 1971; Aleamoni and

1974) seems to have some effect but it isNusuallY not quite of

significant.

raham,I

tistically

On the other Dand, ra).whether the student was taking'the course ad

a requirement or od' an elective basis (Costin, 'et al., 1971; Gillmore

and Brandenburg, 1974), and (b) the level of the ourse (Costin, 1971;

Aleamoni and. Graham, 1974) yielded significant effiects on student ratings.

Such effectsk however, can be controlled through the use of appropriate,

normative data, which is an:important feature of he results reported for

the CEQ.

The one study cited by Costin, et al., -(1971) indicated that 'whether ,

the student was a major or a non7major.did not affect his-faculty rating.

This concern is presently ieins-Investigated using CEQ data

6. . The grade or mark a student receives in the course is highly

correlhted to his rating of the 'course and the instructor. There is ample

10

10.

/

evidence in'the research literature to provide a definite answer to/

this concern. In almost all of the' studies cited in ,Cotin t al.

0

(1971) and by investigators such as Guthrie (1954), Remmers c1-900)

and Weever (1960) little or no relationship has been-found/between

a student's grade plid'faculty rating. In'face, the positive corre-

lations seldom exceed .30. The evider, th7tfore, indicatep that

students ao not necessarily rate an instr4,46tor or course based upon

the grade they have or are a out to receive.e

7. How ckn student' eval tions, sibl be usedlto improve

.instruction? This cpuld well be the most important question to be .

asked concerning student evaluation of instruction. There has been

a great deal Of discussion in the research literatUre about how, when,

and where such evaluations should be used, but no clear-cut evidence

has been offered to show that it does have an effect on instruction.

The studieS by Miller (1971), Braunstein, Kle and Pa'hla (1973), /

and Centra (3,973) were inconclusive'with espec to the effect of

feedback at midterm to instructors w instr aion was again eval-

uated at the end of the term. How er, such evidence it found in a

recently completed study by Ale 'oni (1974), where student ratings, .

r- I.

gathered near the end of the erm, on thp CEQ, were presented to Uni-, /I" .

versity instructors along Nliti the opportunity of discussing their

results with a nipasurement and evaluation expert. The CEQ ratings

were again gathered at the end of the same tourees (one semester to%

.-. a year later) taught by the name instructors. The results revealed

that there was a significant increasein the student'ratings of0

these faculty on the two lowest. rated CEQ subscales that were discussed

/ in the meetings.(with the measurement and evaluation expert. Op the-

- 1 I

11.

4

r

F.

other hand, the group of faculty who was, not able to emil themigevss,

0,f expert consUliatAn but did receive ,the' CEQ results, remained un-

changed in their subscale ratings. Even thoilgh this study needs to

be replicated, it represi.nts the first tangible evidence that,student

-ratings can be used to improve instructidli.

It should be obvious,by now that the problems and questions that

faculty typically raise about the appropriateness of using student

ratings of instructor and instruction have-very definite answers which.

.can be interpreted as eing highly supportive for using such ratingd.. .

...

- ...

.

Interestingly enough,oany faculty will atill"disregard the evidence.

. 4(

and will' maintain ei.at there still is notenotighevidence or that_there

are still too any unaoswered6qUestions and pioblens to take student .--

ratings seriously. However, these same faculty would stoutly tend

their own methods of evaluating Students even though they might not be

-able t6 present any evidence to substantiate theft. claims.

In conclusion, therefore, it might be= useful to reverse the situa-

tion and ask faculty members to consider the seven concerns posed.above

from the point of view of faculty 'evaluating students! Hoy" much evi-0

<fence would' or could be pro -tided to convince_ the students that:

4: (a) Faculty can consistently judge student learning?, and4

0)- the performance of studpts wi h excellent ability is not usedcit '

to set the standard for the rest of the students in the course?

2.- Most faculty grading schemes are not affected by the at;entive,

polite, conforming, and non-creative student?

3. What was taught in the course is useful in other courses or

outside that university?

I

-41

4.. The course examinations are-reliable and valid?

5. An iinstructor'S markll or grades are not affecfed,by:

(a), the size of the class?'A'

CO. the sex of the student .

thetime of day the course was offered?

ghether.the student was taking the coUrge.as a required

r an elective?

(e) 'whether the student was a\majorOr a-on-majorT,'

(f) the term (or seme r).thg course was offered?

(a the level of the course?

(h) his professional rank?

6. Irlstriactors who had a partic rlyrOugh time when they were.

in college 0 :not tend to be just ag rough\On their s.tudentsl

7.. The courseaexaminations are useful. in .improving their learning?

ct.

13.

Reference

Aleamoni, L. M. Illinois course evaluatio questionnaire (CEQ) results

interpretation manual form 66 and form 32. Restarch'Report No. 331.

Urbana, Illinois: Measurement and Resea ch Division, Office ofi7

InstrUctional Resources; University of Illi

Aleamoni, L. M. A review of recent reliability

the Illinois course evaluation questionnaire

ois, 1972d.

validity studies on

CEQ). Research Memotan,

dum No. 127. Urbana, Illinois: Measurement a d Re,carc4 Divisfon,

Office of Instructional Resources, University of Illinois, 1972b.

Aleamoni, L. M. 'Response to Professor W. Edward De

teaching." The American Statistician, 1972

Aleamoni, I. M. The usefulness of student e.ral tions in improving

collSge teaching: Research Report No. 356. Urbana, Illinois:

g's "memorandum on

.4

Measurement and Research Division, Office of Instructional Resources,.

Uniyersity-of Illinois, 1974.

Aleamoni, L. M. & Grajam, M. H. The, relationship between CEQ ratings ando

instructor's rank, class size and course level. Journal of Educationalo

Measurement, 1974 (accepted for publication Jule,26 ,1973).

Aleamoni, L. M. & Spencer, R. E. The Illinois course evaluation questionnaire:4

A description of its development and a report of some of.its results.

Educational and Psychological Measurement, 1973,-33, 669-684,..

AleamonWL. M. & Yimer M. An investigation of the relationship between

colleague rating, student ratting, research productivity, and academl.c

rank in rating instructional effectiveness. Journal of Educational

Psychology, 1973, 64, 274-277.

14.

Aleamoni, L. M. & Yimer, M. Graduating senior ratings lelationship to

colleague rating, student rating, research productivity and academic

rank in rating instructional effectiveness. Research. Report No. 352.

Urbana Illinois: Measurement'and Research Division, Office of

InstruCtional Resources, University'of Illinois, 1974.

Braunstein, D.N., Klein, Se'Pachla, M. Feedback expectancy and0

shifts in. student ratings of college faculty. Journal of Applied

Psicholpgy, 1573, 58, 254-258.

Centre, J. A. Effectiveness of student feedback in miodifying college

instruction. Journal. of Educational Psychology, 1973, 65, 395-401.

::Cohen, S. H. & Berger, W. G. DiMensions of student's wings of college,

instructors underlying subsequent achievemenf on course examinations.

Proceedings;-78th Annual Convention, American Psychological.Associa-.

tion,:1970, 605-606.

"Cdstin,' F.,'Greedaughi W.T., & Menges, k.Y. Student ratings of college

teaching: Reliability, validity, and usefulness. Review of Edur.

dational Research, 1571, 41, 5.11-5A35,-

1"

Deming, W. E. Memorandum on:teaching. The'American Statistician, 1972,

26(1) 47.

Drucker, A. J. & Remmers, H. H. Do alumni and students differ in their

attitudes toward ihstructors? Purdue University Studies in Higher

Education, 1950, 70, 62-64. T

Drucker, A. J. & Remmers, H. H. ,D4."2.Suaihi and students differ in tair

artituded toward instructors? 'Journal of Educational Psychology,

1951, 42, 129-143.C

15.

ji\.

,SVerly, J. -C..& Aleamoni, The rise and fall of, the advisor . .

students atteMpttb evaluate their i?tstructors. Journal of the

National As.soeia ion of Colleges and Te ers of Agriculture,

1972, 16,, 43-A5i.

Feldman, S. The AdviSor. Research Report No. 325. Urbana, Illinois:. .

teasurement-andrteseareh Division, Office of Instructional ResourCes,.

University of. inois, 1970,

Gillmore G. M. & Bra denburg, D. C.. Wouldthe proportion of students- ,

taking a class a a requirement effect the stutlent 1`' ting of the

ittcourse? Research Report No. 347. Urbana, Illinois:r Measurement

ti

end Redear DitSion, Office of Instructional Resources, Univ sity

oc Illinois, .194.

Guthrie, E. R. 'The evaluation .of A progress report. Seattle,

4

Washington: UniVersity of Washington, 1954.

MoKedchie, W. J. Student ratings of fadulty. AAUP Bulletin, .1969,

'439-444.

Miller, M. T. 'Instrpctok attitudes tCwwYd, and their use of, student

ratings of teachers.' J.ournal of Educational Psychology, 1971, 62,

235-239.

mmers, Manual,.,of instruction -for the Purdue rating scale for

instruction. Westafayettee Indiana: University Book Store,

360 State, Street, 1960.

Stallings, W. M. & Spencer, R. E. Ratings of instructors in Accountancy.

.roi from video-tape clips. Redearch*ReporeNo. 265. Urbana,

1

Illinois: Measur4ment and Research Division, Office of 'Instructional

,esources, University of Illinois., 1967.

It 4

16.

Swanst"-R. A. & Sisson, D. J. The deV ,tlopment evaluation, and utiliza-.

tion of a deartmental faculty appraisal system. Journal 8Y

'Industrial Teacher EdUcation, 1971, 9, 64-79.

Weaver, C. H. Instructor rating by college students. Journal of

Educational Psychalogy,.1960, 51, 1-25.

WM.

0+;

. ,

O

/

1

4

V

S

4

et 0 .

4.1

I

Appendix:4

18

t

4

v.

17.

C L

PLEA ;F l'SE NE F' THE FIRM PSIJ youR PERSONAL COMMENTS ON TEACHER urrEcTrvENE''s AND GENERAL COURSE VALU.

THEN TJRN T '-=, ANL A'.';%,ER T,g ,,,JrcT,vr .),,Etilow:, ON THE r THER ',IDE, HS,NE:.PENCIL .oNty. YOUR INSTRUCTORIs. . .'"---eir-

WILL NOT ',FE CoMcLFTII, EVA:NATION )N1'4*, AFTER F INALi ADEN ARE %N FOR YDNR CONRSE. .

, %__ dikr J

cpuRsE CONTENT . t.

PLEASE GIVE ,,,,- 17,NIMENT', ON THE coNR r (FNTENT, .,H.JECT MATTERSANP ANY PARTICULAR RELEVANCE TH1', COURSE HAS

(

HAD TO YOUR AREA OF STUDY.

/fs.. . /.

A

IPi

.

INSTRUCTORS

WRITE. THE NAME OF YOUR PRINCIPAL INSTRUCTOR T.A.

E WHAT ARE V +1R GENERAL COMMENTS AHOUT THE iN',TRUCTOR N THIS COURSE?

I

OSTRUCTIONALornJECT!VE7,$fERE THE iNsTRvCTIONAL OHJECTtVU, tLEARLY

ASTATED FOR TH1', COURSE? Y

PAPERS AND HOMEWORK

CommCNT oN'THE VALUE OF HOORI4W HOMEWORK,#ANO PAPERS (IF ANY) IN THIS COuRSE,

EXAMS

Cot ENT ON THE EXAMS (oNipES, PRACJICALS AS TO DIFFICULTY, rk-,RNESS, ETC.;

GENERAL

1, WHAT IMPROVEMENTS IN THIS COURSE WOULD YOU SUGGEST?

2, PLEA1E GIVE YOUR THOHGHTFNL EVACUATION OF THiS,COURSE WITH COMMENTS. ARF YOU SATISFIED WITH WHAT YOU ,

GOT OUT. OF THIS COURSE? D YOU CONSIDER it A VALUAULE

A'REOHIREMEtaT?. C1R A DliApPCINTMCNT? PLEASE COMMENT.

U P 31-1173

EDUCATIONAL EXPERIENC?

ft

PLEA$E'FILL OUT THE OTHER SIDE

iLMPLY A. MEANS OF PASSING

est.t OPTICAL SCANPIIPM COCPOCATION

FORM 73 ILLINOIS COURSE EVALUATION QUESTIONNAIRE CE6tiniverslt y of Illinois, 1972Mednurrment Arvi senr. h Div 181,1. lit t 1,11.41 Rrs ,ur, in

0W W NW

IF YOU STRIl.,10' AGRE 7,0"ICCWITH THE EM ''0131-D

>41,`"z0 fF;'-19-"r

MARK.SA

A

D

SD

',0U4V2PLL MODENA Tt L IWITH THE 1TE m

IF YOU DISAGREE MODERATELY',./Ii.g*H THE ITEM

IF YOu 5TRONC,4Y D'SACIREIE,WITH THE ,TEM

WJrv'Cr 0

>-

Lu

SAMPLE 04,1. MARK ONLY O(j) MAR KS 0

' uRESPONSE PER ITEMUSING PENCIL ONLY a) I

t

tn..- tu

2. ERASE CHANGED, rc

ANSWERS CLEANLY 0AND COMPLETELY. b) I U

.

c

i"gn w

O oN IA

O 0cc ,xO wIft CC

U3

Z

Z zcc a ..../0 .L

:4It was dude billing,

..c z r .ALIA LTA .

Es

Overall the course was Rood.4....

E

'

UQz0YU'<Li

D*0 0>-u

in a.-CC

9 4a

Booed ut

D(C O

I would takemother course that was taught this

The instructor see 'led to be

0.

U

4440

.4

aim

N

derested in students as per saps

it

WCO U.' Ut

LL,

LL, z

,!:,

- u .

r, 41 r

I oi. Id h. Preferred another method of teacii7ng ul this Course.

It was easy to remain attentive.

The instrti NOT syntheslie,Itegrale of 4t.,7.1rIze effectIvelv.\

NOT much ''..)s gained by taking this course.

:The instructor ericour,gfd the develop:iment oi new viewpoints and appreciations.

I leaf n more when other teaching methods are'ClVed.

4, The curse material seemed worthwhile.

The tryctor was excelient.

The ins ructor demonstrated a thorough knowledge of the subject matterI,

1"Nould r)ther NOT lake another course from this instructor.

It was a v¢iy fVorthwhnle course.

SOIne things were NOT explained very .veil.

The course material was too difficult.

This was One of my poorest courses'. C.

The Instrtictor seemed to considergeaching. as a (horn ot mutine activity.

It was quip:cresting,

V TO

I think that the course' was taught quite well.

TPte course conteri was excelledt.

Some days I eras Ne very interested in this course.

4J2114-nzooUL./

, 4 1

24 A .s4 A 0 f,,E1

1-11

,t

B r

G

N

A

L A

T ' A A LA TAO

',A A !D (3!

T,A1 A C., T;(11

- -

A D SO

*,,A, A 0 ;0

T sn A C) SE)]

3 ',A A C3 SU

SA A .0 SDI

I SA A !, SC I SA AC

JAS -4-'

5.A, A CI SO,EM"0.,2/ A

SS. 4

E sr, A

C

T ! ',A A

;-.1.1

,AL A I, SA A ci r4r10N 4

A A I ' I I. SA A U *,0 I.

*IA A 0 ct)

"44, A L3 fl);

'3,A A U 's !

_

`.A A C) **IC)!

,0 , d SA A D $D

. I

5A A D -SD

I,I ..A A 1.) t,0

.._

. D

'A A D

a. 4A A 0 50

OPTIONAL ITEM SECTION II i! . 01) A -13 (: 0 E ,1 A 11 C L), I ',', A, ii C D L O t 4 A 0 f. 0 r i- A a C 0 at ji.,ABCDf 71A I3CCIE

4

-

;1 A B C b i iv A R is D t 80 A 11 L. f) F '' 1 A o (') o i i A ft C 6 E 83 A tIl C. 0 E 64 A ro c p i pr, A II f: 5 E'I. 0.., .. .. .

; PLEASE FILL OUT THE OTHER SIDEC *iiipO .**1.4.., OPTICAL SCANNING CORPORATION 0 9-. .4.49

40,

er

It

"5-

r

r

Appendix B

r

A

4

A

19.

""

4

I. -I 4171" -'

._-

* * * MERMAC 1-- TEST ANALY IS Al 0 tJUE S I N AI E

,d1

.. .{FEESULTS FOR, THE OBJECTIVE ITEMS ()N )'HE CEO FORM 73 Q0,EST,113NrIAIRE -\

4

'

4 tnoiN os v fti 4 4 . e J : I f I'l t.71 `1 f.

STATUS.

.FR tbSli Steli Ji, SR URAL) U 1 HER Uilli0.87 . 0.13 / 0.00 0.00_ 0.00 . 0 .00 0.00 - -

.f _ -7 P EU ,I EX EL-1 G R A DI

o' ' . . A 8 C 0 E OMIT0.40 0:53 0.07 o.uo . 0.00 0.00,,t ,

- PASS-FAILi YES NO OMIT

-

\o-.01:-) C). ell 0.1.3/. t

COURSE OPTION

,....,..--......,-

RV(,) tLECT OMIT"2" . 0.40 0.47 0.13

' MAJUk-NNUR..0,MAJOR MINOid. OTHER OMIT.... 0.13 1).00 J.60 0.07

.

.b. -SEX*

- FEMALE MALE OMIT ,

. .

0,..,. _ 0.0 / 0.31(0.07

/I -,

.. .

CC NT E HT IIAT I NGMEL V Glr G004 FA1a POUR V PR OMIT

a 0.27, 0.47 0.20 0.00 0.00 0 ...GO 0.07,

i \ I Hs I i*, RATING ' - .

EXCEL V GU 6,6Ut) F-Al R . POOR V PR OMIT,, . 0.80 0.07 0.07, 0.00 0.00 0 ... 0 0 0 .07

,. ,it

- ,CUURS -4 TI NG

EXCEL V GO 60.6U7-FA IR _POOR V PR OMIT.. 0.27 0.60 0.b.17 0.00 0.00 0.00 0.O1va

w.SEMESTLP.

,FALLSPRlivoSUALR atilt

, .o.ou 0.60 0.00 0.20.

.

ITEM SA A O' SD UMIT AST MEAN ID. JECL 01234567891. 0,67 0 33 0.00 0.00 0.00 SA 3.67 0.49 9.. 2. 0.93 J.07 0.00 0.00 0.00 . SA 3.93 0.16 9

'" .3. 0 J.27 0.27 0.47 0.00 SD 3.20 0.80 6 *4. .67 0.33. 0.00 0:00 0.00 .SA 3.67 0.49 9 *

4

_---v.5'. 0.00 J. Ot1 0:33 04.0 r 0.00 Sil 3.61 0.49 4-; IL

6. 0.00 0. OU 0C33 0.67 U. U0 SO 3.:37 0.49 87. 0.33 0.53 0.13 0.03 ilipt) SA 3.21) 0.68 6.. . 8. 0.00 0.21 .47 a. .I I isKio d0.47 -I M: 3.00 0. to 6

.

t

* * * MEA4AC -- TEST AcIALYSIS. ANL) wJESTILo!4AIRL PACNAt;'7:

9. 0.60 U.40 0.0U 0.30 0.u010. 0.93 0.0! 0.00 0.0, 0..jJ

11. 0.t7 J.13 0.0u 0.0..E u.JJ12. .0.0U. J. JU 3.20 .0.JJ O.J.j

-13. 0.47 J.)3 0.0J O.U. ,J.UU

14. p.ou J.13 0.4u 0.-t/ 0.JO15. 0.00 0.00 0.53 0.47 0.j016. 0.00 0.07 J.)3 6.ou i.0)17. 0.00 J.JO 0.27 .0.0 0.0018. 0.53 3.47 0.0U U.00 J.JJ19. 0,53 .U.471`,40.00 U.Ou 0.Uu'20. 0.33 3.00-}0.07 0.00 0.0021. 0.00 3.20 0.40 0.4u u.Uu22. 0.00 J.00 U.20 0:(,0 u.JU23. 0.73 0.27 0.0u '0.00 U.OU

sA 3.60 0.51SA 3.93 0.26

3.87. 0.35.sO 3.t30 0.411SA 3.47sO 3.33 0.7.

3.47 .523.53 C.04'

sD 3.73 0.46SA 3.53 0.52JA 3.53 0.52SA 3.27. 0.59SO 3.20 0.77s0 3.80 0.41SA 3.73 0.46

9

9.6

rs

9.

9

0

8

9

9

9

--SUBSCORE-- ITEMS RESP MEAN S.D. REL RANK/

GtNcRAL ATTITUDE 4 1.00.1 3.6U .)..) 0.35METHOD. 4 1.00 3...li 0-33- 0.93 /8CONTENT 4 1.0C 3.42 J.45 0.86 . 9

INTEREST 4. 1.00 3.3D u.4u 0.77 5'

INSTR:GENERAL 2 1.00 3.07 0.30 0.70 9

INSTR:SPECIFIC 5 1.00 3.06 .i.27 0.49 9

ro rAL 23 1.up i.JJ .:,.3.) 0.90 9

SV4PLE1 SI l' = 15

LEVEL

99

:9.

5

9

'9

9

INSTI COL-r,-GF

OVER-ALL

/ .0.1NE

//'d

/ 8' JCNE ,8

-- 9 -MCNE 9

9 r,C,E

9 Nillii 9

9 NIA9 i:UNt

qty