DOCUMENT RESUME

ED 292 433 HE 021 318

AUTHOR Rossmann, Jack E.; E1- Khawas, ElaineTITLE Thinking about Assessment: Perspectives for

Presidents and Chief Academic Officers.INSTITUTION American Association for Higher Education,

Washington, D.C.; American Council on Education,Washington, D.C.

PUB DATE Jun 87NOTE 28p.AVAILABLE FROM Division of Policy Analysis and Research, American

Council on Education, One Dupont Circle, N.W.,Washington, DC 20036-1193 (single copies free; 2 ormore, $1.00 each).

PUB TYPE Reports Descriptive (141)

EDRS PRICE MF°1/PCO2 Plus Posta9e.DESCRIPTORS Accountability; College Administration; College

Students; *Educational Assessment; EvaluationCriteria; *Evaluation Methods; Higher Education;*Outcomes of Education; Program Development; *StudentEvaluation

IDENTIFIERS *College Outcomes Assessment

ABSTRACTAssessment in higher education is shifting from the

state house to the college campus. Information is provided to helpcampus administrators make the best decision regarding theirinstitutions. A general perspective is offered to senioradministrators with little previous experience in systematicassessment. Section I, "Assessment: What Is It All About?" definesand explains the concern with assessment. Section II, "Should WeDevelop an Assessment Program?" considers the benefits of assessment,the way to balance assessment with other goals, and how to deal withexternal constituencies, "How Should We Start?" in section III,focuses on the leadership role, organizing for assessment, collectingdata, determining costs of assessment, and establishing a schedule.The accompanying figures offer information on different approaches toassessment, state mandates on assessment, possible components of anassessment plan, and possible roles for presidents and chief academicofficers in student assessment; and present a checklist of possibleassessment costs, examples of a planning schedule, and advice fromassessment pioneers. Twenty-two references are given and a concludingpage is included, entitled, "Using Assessment to Greatest Effect."(SM)

***********************************************************************Reproductions supplied by EDRS are the best that can be made

from the original document.***********************************************************************

Single copies of this report are available free of charge. Two or more copies are$1.00 per copy. ALL ORDERS MUST BE PREPAID. (payable to: American Councilon Education).

Please enclose a self-addressed mailing label.

Send orders to:Division of Policy Analysis and ResearchAmerican Council on EducationOne Dupont Circle, N.W.Washington, D.C. 20036-1193

June 1987

This report is not copyrighted. It may be reproduced in whole or in part in the in-terest of education.

AMERICAN COUNCIL ON EDUCATIONRobert H. Atwell, President

The American Council on Education, founded in 1918, is a major national associ-ation serving higher education.

1

Thinking About Assessment:PERSPECTIVES FOR PRESIDENTSAND CHIEF ACADEMIC OFFICERS

Jack E. Rossmann Elaine El-KhawasMacalester College American Council on Education

JUNE 1987

American Council on Educationand the American Association for Higher Education

Washington, D.C.

ACKNOWLEDGEMENTS

We are grateful to many college administrators whotook the time to offer advice and suggestions about thispublication, including: Peter Armacost, President, EckerdCollege; George Ayers, President, Chicago State University;Martha Church, President, Hood College; James Firnberg,Chancellor, Louisiana State University; Jerry Gaff, Dean ofthe College, Ham line University; Rick Heydinger, Assistantto the President, University of Minnesota; Jack Lindquist,President, Goddard College; Charles McClain, President,Northeast Missouri State University; Russell Warren, VicePresident for Academic Affairs, James Madison University;and Barbara Uehling, Chancellor, University of California,Santa Barbara.

Thanks are also due to several members of ACE staff,Barbara Turlington, Douglas Whitney, Madeleine. Green,and Andrew Malizio.

We owe special thanks to Patricia Hutchings and Theo-dore Marchese, of AAHE, for their effective collaboration onthis project.

INTRODUCTION

Assessment in higher education is currently a "hot topic." Some state legisla-tures and governors are mandating that colleges and universities develop a.sess-ment plans. Regional and professional accrediting associations are sharpeningtheir focus on the assessment of student achievement. And conferences on assess-ment abound. Why is there so much interestand what do presidents and chiefacademic officers need to know about the topic?

Although assessment, as a buzzword, may fade away, the pressure for betterinformation regarding student achievement is likely to continue. Based on a surveyof state higher education officials in January 1987 (Boyer, et. al.), at least two-thirds of the states have formal initiatives that are labeled assessment. Signifi-cantly, however, the survey also reported a "strong trend among state authorities. . . to consider the design and conduct of assessment a matter of institutionalprerogative."

With the assessment focus apparently shifting from the state house to the cam-pus, the American Council on Education authorized this essay as a way to helpcampus administrators review some of the issues surrounding assessment so thatthey might make the best decisions for their institutions. Thi3 essay, cosponsoredby the AAHE Assessment Forum, has three purposes:

to offer perspective on what the assessment debate is all about:G to suggest some considerations for deciding whether a campus should com-

mit itself to the development of new assessment procedures: andto hiLhlight some issues relevant to deciding how to approach assessmentof student learning.

This is not a "how-to" manual. It does not review specific assessment techniquesor offer guidance for how a campus committee might proceed to develop an assess-ment program. The American Association for Higher Education, through itsassessment Forum, is sponsoring a number of publications designed to offersuch guidance. The Southern Association of Colleges and Schools and the Officeof Educational Research and Improvement, U.S. Department of Education, areamong the agencies that are developing detailed guides to available assessmentmeasures. This essay, in contrast, seeks to offer general perspective for senioradministrators who have little or no previous experience with systematic ap-proaches to assessment.

6

.e,t))

pool 1

tt

tI

Section I:Assessment: What is it all about?

In higher education today. assessment typi-cally has ambiguous meaning. The term, whicharises from the lexicon of psychological measure-ment and testing, has taken on a much broadermeaning not unlike a closely related word, evalu-ation. In higher education. its meaning is some-times associated with a few, well-publicized ap-proaches to assessment at the college level. Todate, assessment has focused almost entirely onundergraduate education.

DEFINING ASSESSMENT

While there is no single, commonly accepteddefinition of assessment. the current debate overits value for higher education reflects at least twocritical aspects of its meaning:

assessment tries to determine what stu-dents actually achieve in their collegestudy: and

assessment liiiks educational objectives (ofa course, a program. a field of study. or aninstitution) to some measures of studentachievement.

The key purposes of assessment are to askimportant questions about student learning. toget some meaningful information on these ques-tions. and to use the information for academicimprovement. An assessment program need notbe expensive to achieve these goals. Nor is itnecessary that elaborate research instrumentsand procedures be developed.

A typical pattern of assessment is the use ofseveiai measures of what students are learning.administered periodically. Often. this involvesputting existing campus information to new andmore systematic use. Examples of such informa-tion include registrar's information on course-taking patterns, on student completions and fail-ures in specific courses, and on student progresstoward a degree.

In other instances, campuses have begun col-lecting new data. often including proficiency

"The key purposes ofassessment are to askimportant questions aboutstudent learning, to getsome meaningfulinformation on thesequestions, and to use theinformation for academicimprovement."

MI.

8

AIMMONIM

3

Figure 1. Different Approaches toAssessment: Some Examples

LIMITED APPROACHES(These involve a specific program or assess anaspect of student performance.)

Assessment linked to efforts to increasestudent retention

Senior-year projects, to demonstrate whatseniors have learned and their ability toapply their knowleage

Comprehensive exams based on the gen-eral education curriculum

Evaluation of services to support andmonitor remedial/developmental educa-tion

Student development outcomes assessedduring the college years

For selected academic majors, statementsof program objectives developed along

' with ways to evaluate them

Results on job plar lent and employersatisfaction

Surveys of graduating seniors or ofalumni/ae to learn their opinions about thecollege's programs

COMPREHENSIVE APPROACHES(These include several different components,assess students at several points or cover theentire curriculum.)

Student surveys plus standardized testsadministered as students enter and againas they complete college

Competency-based curriculum in all occu-pational and professional programs

New general education curriculum, witheach course having defined objectivesand criteria for assessing their achieve-ment

Use of assessment center techniques,which measure the performance of eachstudent on specific learning objectives,usually at several times and by several dif-ferent methods

4 A

exams for writing skills and testing of achieve-ment levels of entering students. New attentionhas also been given to assessments by graduatingstudents of their college experience and to ques-tionnaires sent to alumni/ae, employers, or grad-uate schools. As Figure 1 indicates, campuseshave developed quite different approaches to as-sessment. based on each institution's own pri-orities and mission.

WHY THE CONCERN ABOUT ASSESSMENT?

A logical question to ask is "Why is all thisgoing on?" A related question follows. "Can we ig-nore it?" Answers range widely and are quitespeculative. Some common explanations for thecurrent move toward assessment include:

Perceived weaknesses of higher educa-tion. Some observers contend that current collegeinstructional practices, rooted in the early 1970s,are shoddy, lenient and out-of-date. This expla-nation, which provided the underlying rationalefor most of the national reports on higher educa-tion that have been issued in the last few years,focuses on weaknesses in the academic curricu-lum and, sonietimes, blames shortcomings onexcessive responses to student pressure for cur-ricular flexibility. Such arguments contend thatthe curriculum at many colleges and universitiesis uneven, lacks overall coherence, and reflectsvery little attention to what students are actu-ally learning. Others see a "backlash" againstopen-access policies, a call for returning to the"basics" and restoring stringent expectations. Arelated argument is that, because a few postsec-ondary institutions may provide low-quality pro-grams, new requirements must be imposed on allinstitutions.

Trends in the workplace. Another expla-nation looks to forces of change in the Americaneconomy. Dramatic changes are taking place inthe occupational needs of the American work-place, with a sharply increased demand for work-ers with stronger academic skills. The needs of aninformation societywith an expanding servicesector and with increasing skill levels needed forautomated, high-technology industrieshavecreatea an urgent need for workers with goodreading and mathematics skills and the ability tohandle more complex tasks than previously re-quired. Both the public school system and thecollegiate sector are, therefore, under pressure torespond. Political leaders, in a context of a rising

imbalance in foreign trade, have responded toconcerns of business and industry, in part bychampioning their need for a better educatedworkforce. Under this scenario, the greatest pres-sure is to improve the "minimum" levels of skillsthat workers possess: this pressure focuses onthe high schools and, secondarily, on communityand technical colleges.

Political pressures. The increasingly im-portant role of state government is another fac-tor. Recent eventstroubled state economies,economic development initiatives, revenue shar-ing, and cuts in federal spending programshave shifted important responsibilities to thestate level. State governments have gained in-creasing professionalism and are asking tougherquestions about how state funds are being spent.Colleges are being asked to provide evidence ofquality, in this view, as part of a generally tougherstance by the states.

There are also those who would offer a morepolitical explanation. They argue that many statepolitical leaders found perceived weaknesses inelementary and secondary education to be aneffective political issue, and are now repeatingthat political scenario with a focus on highereducation.

Despite the lack of clarity on "why" assess-ment has become a new rallying cry for reform.certain facts are quite evident: Two-thirds of thestates have taken action to expect new informa-tion on student progress and performance, andmost accrediting agencies now expect betterinformation regarding student achievement andinstitutional effectiveness. The call for im-proved assessment is not something that collegescan ignore.

Indeed, findings from ACE's most recentCampus Trends survey (Campus Trends. 1987)indicate that a large number of colleges and uni-versities ate considering ways to develop new ap-proaches to student assessment. Currently, onein four institutions reports that their states arerequiring the development of assessment proce-dures. Among the institutions without statemandates, seven in ten nevertheless expect to in-troduce some form of assessment on their cam-puses in the next few years.

"Two-thirds of the stateshave taken action to expectnew information on studentprogress and performance."

..iMEMP

1.1 9 5

Section II:Should we develop an assessment program?

Given the current climate. most college ad-ministrators may be asking this question. Yet,many find it difficult to think objectively abouthow assessment might affect their own institu-tions when, at the same time, they must pay at-tention to external requirements for assessment.If the state legislature or higher education coor-dinating hoard requires some form of assess-ment, a college has no option but to do some-thing, whether or not its leaders see the potentialvalue in it. Similarly, in regions where the ac-crediting association emphasizes the assessmentof student achievement or institutional effective-ness, colleges must take some action on studentassessment, particularly if accreditation visitsare scheduled in the near future.

BENEFITS OF ASSESSMENT

Apart from external constraints, there are po-tential benefits that may make it worthwhile fora campus to consider assessment. Among them:

Academic introspection. Perhaps thegreatest long-term benefit is that assessment,once established, makes an institution more self-conscious about what its academic programs areaccomplishing. A well-designed assessment pro-gram with strong faculty support should foster astrong collectiveand continuingfocus on howeffectively the institution is meeting its goals.

Information for recruitment. Some institu-tions have realized that prospective students andtheir parents are keenly interested in informationon the actual experiences and accomplishmentsof students. Colleges with large numbers of pre-professional students have found that applicantswant to know the college's "success rate' in pro-fessional school admissions. College viewbooksincreasingly provide such information.

A context for planning. Information onstudent progress and performance also offers afactual context for academic planning. Some in-stitutions have begun small-scale asoessments as

sfA

"A well-designedassessment program withstrong faculty supportshould foster a strongcollectiveand continuingfocus on how effectivelythe institution is meetingits goals."

27

"Institutions having someexperience withassessment report suchbenefits as greater clarityon how course sequencesfittogether, vigorous andhelpful faculty dialogue oneducational purposes, . . .

and the assurance that allstudents meet certainstandards."

8

part of a curriculum improvement project. Othersconduct assessments of the post-collego employ-ment success of graduates as a way to keep theirvocational prog, ams up-to-date.

Readiness for accreditation studies.Another potential benefit of an assessment program relates to regional accreditation. Within thepast give to ten years, all of the regional -6.ccredita-don associations have called for evidence aboutinstitutional effectiveness and the quality of stu-dent achievement. Most accrediting associationsconsider that assessing student achievementshould be an on-going and integral part of the col-lege's planning process. Development of an as-sessmeat approach may mean that, at the time ofthe college's next accreditation self-study, it al-ready will have collected a good amount of infor-mation on student achievement and incorpo-rated this into institutional decision-making.

Improvements in teaching and learning.Institutions having some experience with assess-ment report such benefits as greater clarity onhow course sequences fit together, vigorous andhelpful faculty dialogue on educational purposes.new approaches to teaching, an improved basisfor advising students about their academic prog-ress. and the assurance that all students meetcertain standards.

Other reported benefits include better stu-dent retention, improved public relations, andfund-raising gains. One university, for example.reports that the assessment process has helpedits image with both students and legislators.Another college reports that its applicant pool hasincreased since it began a comprehensive assess-ment program about a decade ago.

BALANCING ASSESSMENT WITHOTHER GOALS

With all the activity surroui !ng assessmentin recent years, it is easy to lose sight of the factthat, at best, assessment is a vehicle for academicimprovement: it is not an end in itself but ameans to an educational purpose. An institutionshould not implement assessment at the expenseof other important academic and institutionalgoals. As part of the planning process, collegesmust examine an assessment plan's potential ef-fect on other important goals. Among the issuescolleges shoul'l consider:

Current circumstances. The timing maybe wrong. There are special circumstances thatmay make it difficult to begin a new assessment

13

program: a financial crisis: a pending reorganiza-tion; retrenchment of programs; appointment ofseveral key administrators: or other criticalevents in the college's life. Sometimes, collegeleaders have found that embarking on such a newinitiative helps the campus to move beyond acrisis situation.

Minority access and opportunity. Assessment can be a supportive mechanism for minor-ity student achievement or can become a newobstacle that holds students back or discouragesthem from attending the college. If assessmentplans include standardized tests, it becomescritical to examine the likely effects of suchtesting on student access. Are there alternativeapproaches that might resolve anticipated dif-ficulties? Existing courses might be redesignedor new courses developed to ensure that studentswith academic weaknesses have assistance inpreparing for such tests. Some institutions withwell-developed assessment programs have takenthe stance that assessment is used for academicprogram improvement, not for penalizingstudents.

Freedom of choice. The design of an as-sessment plan must be consonant with the col-

"Some institutions withwell-developed assessmentprograms have taken thestance that assessment isused for academic programimprovement, not forpenalizing students."

9

14

0

U

0

"It is important that anassessment plan not 'lockin' any particular approachor work against change."

10

lege's other purposes. Many institutions, forexample, pride themselves on the degree of choicethey make available to students. Will an assess-ment program introduce pressure on students totake certain courses that "help" in ?ass ing an in-stitution-wide test? A college's programs may bedistinctive in certain fields. Is it inconsistent torequire that all students participate in the sameforms of assessment? Standardized examina-tions may be convenient to use but the testresults are only appropriate for undergraduatemajors where the curricular emphasis matchesthe subject emphasis of the exams.

Adaptation to change. An assessmentplan should be flexible so that it remains mean-ingful when curricular changes are made. Manycolleges have recently introduced a new "core"curriculum and may plan to review and changecertain elements after a few years of experience.They should anticipate parallel changes in theassessment plan. An institution may be planningto introduce new programs in the near future.Another institution may be concerned about in-stitutional change and renewal. In both situa-tions, it is important that an assessment plan not"lock in" any particular approach or work againstchange.

DEALING WITH EXTERNALCONSTITUENCIES

For many campuses, requests from gover-nors, legislators, and state agencies have beenthe starting point for looking into assessment.For many other campuses, the impetus for con-sidering assessment strategies has come from re-gional accrediting agencies.

Are these pressures dangerous for higher ed-ucation? Are they unduly coercive, reflecting in-trusion into issues that should be internal to theacademy? Representatives of these agenciesargue that their actions are a responsible callfor greater accountability. They stress, too, thatthey generally give colleges considerable leewayin determining how to respond to these newmandates.

The institutional perspective is often at oddswith this view. Even though many states have in-vited colleges and universities to develop assess-ment plans that fit their own circumstances,coercive elements often remain, especially in theform of short deadlines or specific requirements.Needless to say, some tension between the state'spurposes and institutional purposes is inevita-ble: As one observer noted, state actions typicallyfocus on "proving" something while the institu-tion's interest is in "improving" education. Even

15

or.Ass..MIL-

so, college presidents know well that there aresome real dangers in state-mandated approachesto assessment. Institutional purposes and pro-grams can become distorted; large numbers ofstudents could have their educational progresstossed into confusion or delayed.

When states require colleges and universitiesto provide specific assessment information, aparticular sore point is over the prospect thatinter-institutional comparisons will be pub-licized. In the Campus Trends survey conductedby the American Council on Education (CampusTrends, 1987), a majority of college adminis-trators argued against a requirement that in-stitutional data should be published, eventhough almost all survey respondents supportedthe use of assessment as a means of internal im-provement. A key fear is that wide distribution ofsuch comparisons invites a "rankings" game: Theinformation that gets the most attention, often inthe state's leading newspapers, is the informa-tion that is easy to use -perhaps a single statisticcompared across all institutions. Yet, such infor-mation frequently offers a distorted picture be-cause it describes only limited aspects of whateach college does and ignores many relevantfactors.

es-

". . . decisions made toaccommodate a testingmandate can conflictwith decisions thatmake academic senseinternally."

16

111=IMMEN=MMII

11

Figure 2. Facing the Prospect ofState Mandates on Assessment

Influence the direction of debate. Get in-volved. Suggest resource people andforums that will allow full discussion of as-sessment and its implications for highereducation.Pr&sa for approaches that offer flexibility,that recognize institutional differences inmission and program, and that do not un-duly penalize students.Make it clear that there are no "quick fizas-in planning and implementing an asseqs-ment plan for higher education, and thatseveral years of development time areneeded.Propose that special funding be allocatedto assist colleges and universities withthe costs of developing assessmentprocedures.

ii. . . colleges sometimeshave considerableopportunity to shapespecific aspects of what thestate agency or legislaturewill decide regardingassessment."

IMINNZMEMIN IIMMO

12

Colleges having some experience with assess-ment often prefer to present assessment informa-tion as part of a larger document that puts resultsin context, points to the college's primary con-cerns, and states what the college intends to doabout these areas of concern. In other instances,where inter-institutional compai isons are man-dated, college administrators have found thatthey can sometimes avoid undue emphasis on"rankings" by the use of multiple indicators, i.e.,information describing institutions in a numberof different ways.

Broadly focused state-mandated tests arealso a subject of major concern to college adminis-trators. Their concern is not so much with thenotion of testing as with the negative effects thatstatewide testing can have on academic pro-grams. They contend that, despite good inten-tions, mandated tests of general competenciesare likely to affect what courses are taught, thetiming and sequence of courses, or the coursechoices cf students. Furthermore, decisionsmade to accommodate a testing mandate canconflict with decisions that make academic senseinternally. Distinctive programs can be seriouslyaffected if students in those programs do net dowell on some aspect of the mandated test.

What's to be done? Options may seem lim-ited. To date, however, most colleges faced withnew state mandates have found that, althoughthey were coerced to do something, there was flex-ibility in choosing exactly what to do and whatthe timing would be. And colleges sometimeshave considerable opportunity to shape specificaspects of what the state agency or legislature willdecide regarding assessment. The advice shownin Figure 2, offered by college administrators whohave some experience with assessment, speaks tosuch situations.

Many administrators with experience in as-sessment also suggest that each college take theinitiative by planning for assessment. In advanceof a state mandate, colleges could develop proce-dures for assessment that are responsive to exter-nal concerns yet are also mear ingful for the col-lege. A technical institute may choose to focus onreactions of local employers to the quality of theinstitute's recent graduates. A liberal arts collegemay wish to demonstrate the value of its generaleducation curriculum. A university may wish toemphasize undergraduate professional programsthat already are subject to external review. Suchcampus experience with assessment would gen-erally prove useful, whatever the eventual direc-tion of state activity on assessment.

17

Section III:How should we start?

Because assessment is a tool for academicimprovement, there is no "best" place for all cam-puses to start when they decide to develop an as-sessment procedure. Figure 3 illustrates the widevariety of options to consider. The starting pointand focus can vary widely from college to college,depending on circumstances.

One of the most important tasks is to developan initial position on what assessment means forthe institution and what a process for implemen-tation should include. Both the definition andprocess are likely to evolve, becoming more spe-cific and more precisely tied to the college's ownprograms as efforts get underway. Thus, it is im-portant to give careful thought to a "startingpoint" while also recognizing that new directionswill emerge.

Implementation activities have generallystarted with a limited focus. The few institutionsthat have comprehensive approaches to assess-ment today either faced unusual circumstancesor moved quite gradually toward a comprehensiveapproach, often over a decade's time.

Generally, it makes sense to try to link as-sessment efforts to the college's current pri-orities. If a new general education curriculum willbe introduced soon, an assessment plan might bedesigned to test its effects. If improving studentretention is a top concern, assessment might belinked to these efforts. Sometimes, an upcomingaccreditation visit, whether for a particular pro-gram or for the entire institution, is a usefulfocus for planning new assessment activities.

THE LEADERSHIP ROLE

It is up to the president and chief academicofficer to set the assessment process in motion.Once they have given the green light and indi-cated their support for the idea, faculty involve-ment should begin. Because discussions overassessment procedures inevitably raise quitefundamental issues about the institution's edu-cational purposes and priorities, it is also impor-tant that the president or chief academic officertake the lead in helping the institution arrive at

Figure 3. Possible Components ofan Assessment Plan

Early in a student's college career

1. Do placement testing in key academicskill areas.

2. Test before and after remedial/develop-mental courses.

3. Design competency-based skillscouses.

4. Collect data for assessing changes instudent knowledge, values, etc.

During middle years

1. Identify desired outcomes for the"core" t.trriculum and develop ways toshow how they are achieved.

2. Identify desired outcomes in majorfields of study and develop ways toshow how they are achieved.

3. Develop methods for evaluating profi-ciency levels that students achieve inwriting, critical thinking, and other gen-eral competencies.

4. Conduct continuing or periodic sur-veys of student values and attitudes.

5. Review student transcripts, papers,examinations, etc.

At college completion

1. Develop comprehensive examinations,oral interviews, or other ways to dem-onstrate accomplishments of graduat-ing students.

2. Gather information on post-collegeplans of graduates, including employ-ment and further study.

3. Conduct final surveys of student valuesand attitudes.

After college

1. Conduct periodic surveys ol alumni/ae,employers, and graduate a:til profes-sional schools.

2. Obtain information on placement rates(into jobs or for further schooling).

1813

Figure 4. Possible Roles for Presidentsand Chief Academic Officers

in Student Assessment

Setting the direction for institutional activity1. Set possible scope (and limits)2. Indicate expectations3. Propose an initial timetable4. Demonstrate moral and financial sup-

port

Putting the idea into institutional context1. Offer an appropriate rationale for de-

veloping an assessment program2. Provide an initial definition of assess-

ment for the campus3. Distinguish between appropriate and

inappropriate reasons for developingassessment

4. Emphasize that the goal is academicimprovement

5. Respond to concerns

14

an initial working definition of assessment, arationale for taking on the effort. and a planningprocedure that is appropriate for its circum-stances. Figure 4 outlines such roles for the col-lege's leadership.

Just as important. the president or chief aca-demic officer needs to explain what importantgoals and uses of assessment are envisioned. Isit important that all students be assessed? Why?Will information be shared with students?Should assessment data be tied to the courses of-fered by individual departments? Should assess-ment information be published and, if so. forwhom?

The president or chief academic officer alsoneeds to clarify limits. The concept of assessmentcan be controversial and can engender numerousmisunderstandings. Unless the president makesit clear that assessment results will not be linkedto faculty evaluations. for example. it is likely thatmany faculty will have fears in this regard. Unlessthe president clearly explains what impact the as-sessment procedures will have -, students. con-fusion and distrust among current students arealso likely.

ORGANIZING FOR ASSESSMENT

As soon as the president has committed theinstitution to assessment as a concept. the or-ganization of an approach to assessment canbegin. Substantial faculty involvement at the out-set is essential. First steps will usually involve theformation of an assessment committee. generallyincluding opinion leaders among the faculty andthe chief academic officer. The involvement ofothers beyond this core group will vary by institu-tion, personal experience. and special expertise.This core committee must develop a process thatwill allow the college to clarify the learning objec-tives to be assessed and to consider alternativeapproaches to assessment.

An important task for the assessment com-mittee is to review all approaches to assessmentin light of other important institutional pri-orities. As plans begin to take shape. the commit-tee should consider what the effects of a planmight be in a few years. Unintended (and poten-tially damaging) outcomes are difficult to pin-point. but a considerable degree of insight aboutthem can be gained from a full discussion of theissues.

19

If, for example. departmental comprehens;veexaminations are to be required as students ar-completing their studies, there must be assur-ances that there is sufficient faculty time avail-able to develop and conduct the examinationsand that there are clear next steps available forstudents who fail. If assessment will rely on astandardardized test, the institution must decidewhether results will be used solely for diagnosticinformation for programs or, instead, whether aspecific passing score is to be expected of all stu-dents. If the latter, the institution must be pre-pared to justify this decision, including thechoice of passing score and the fit between thetest and the college's academic program.

COLLECTING DATA

In planning for assessment, the committeeshould consider from what sources informationwill be collected and how it will be analyzed.Often, a fair amount of relevant information al-ready exists in campus data files and just re-quires further analysis. Other potential measureswill require collecting new data.

Among the potential sources of existing data:

Student transcripts. If an institution as-serts that students receive a high-quality generaleducation, is that assertion supported by ananalysis of the transcripts! If it is assumed thatstudent "good sense" combined with t :live fac-ulty advising will assure that students takecourses in such core areas as a second language,mathematics, exposure to other cultures, histor-ical and ethical perspectives. etc., to what extentis this assumption supported by the transcriptevidence? Some institutions are also exploringways in which standard sources of student evalu-ation like course examinations or papers can be"re-evaluated" as part of the college's broader as-sessment effort.

Student retention studies. A focus on stu-dent retention can be an important part of the as-sessment activity. Of the students who enroll asfirst-time, full-tirw freshmen, what proportionreceive their degrees within a reasonable time?How does this percentage relate to the retentionrate at comparison institutions? Most institu-tions also could compare students who graduatewith students who withdraw from the institutionon such factors as cumulative grade-point-

"Often, a fair amount ofrelevant informationalready exists in campusdata files and just requiresfurther analysis."

'' 015

,11M....WM=MIM

"Academic achievementneed not be the only factormeasured by standardizedtest instruments."

average and characteristics upon entrance (e.g.high school rank, aptitude test scores, careergoals, financial needs). These comparisons helpthe college understand better why students areleaving the institution. For example, if highachieving students are more likely to leave, is itbecause these students are not challengedacademically?

Among the new approaches to consider:

Standardized tests. One of the most basicmeasures of student learning are tests of theiracademic skills. Examples include the adminis-tration of pre- and post-tests for remedialcourses: the use of what are sometimes called"rising junior" tests in basic skill areas or ingeneral education: and the assessment of higherorder skills in the areas of critical thinking, quan-titative problem-solving, oral communication.and writing.

Use of standardized tests for internal aca-demic planning is another option. Major testingagencies are in the midst of developing new in-struments for flexible use in undergraduate as-sessment. An advantage of using standardizedinstruments is that the results can be comparedacross departments and colleges within a singleinstitution and, if appropriate, across institu-tions. Care must be taken that such comparisonsare appropriate and that available instrumentsmeasure what the college wants to havemeasured.

Academic achievement need not be the onlyfactor measured by standardized test instru-ments. A number of institutions recently havefound it helpful to collect data related to the qual-ity of student effort in a variety of areas (e.g., con-tacts with faculty, use of the library).

A critical factor in the interpretation of testresults lies with the motivation of students tak-ing them: a dilemma is that, if the tests don'tcount on the students' records, students may notmake their best effort (or even show up for thetests) but, on the other hand, if the tests docount, it becomes critical that the content of atest closely parallel the objectives of the programthat is being assessed.

Assessment centers. One unique and com-prehensive approach to assessing studentachievement is the assessment center model.This approach focuses on individual student

42 I

learning, with assessment based on student per-formance in simulated activities that draw on thestudent's knowledge as well as skill in applyingthat knowledge. Persons externa' to the institu-tion are asked to determine whether each studenthas achieved certain specific learning objectives.Successful adoption of this powerful approach toassessment requires a total institutional commit-ment and a long period of implementation.

Departmental senior exams. In assessingachievement in a student's major field, somefour-year colleges have developed a senior com-prehensive examination, in some cases using ex-ternal examiners. Some fields already have a pro-fessional licensing examination. Both of these 1

approaches can be effective components of the as-sessment of student outcomes.

Student evaluations. Periodic assessmentof their educational experience by students andformer students can provide helpful informationfor planning. Some institutions have developed apattern of collecting questionnaire data fromgraduating students every year and from a sam-ple of alumni/ae every three to five years. Thesequestionnaires can include items that ask the re-spondent to assess the extent to which she/he hasdeveloped competence in a variety of areas (e.g.,written and oral communication, quantitativeskills, solving complex problems). While the usualcaveats that accompany self-assessment informa-tion are required, this approach can provide help-ful insights regarding institutional strengths andweaknesses.

Employment information. Other ap-proaches to the collection of information involvedata from the institution's career developmentcenter and from employers of graduates. Four-year institutions often seek information from thegraduate and professional schools in which theirgraduates enroll.

As yet, it is difficult to find methods and in-struments that are ideal for each institution andprogram. A good approach, used by most institu-tions that have developed assessment proce-dures, is to use several related measures and lookfor a consistent pattern of results before deci-sions are reached about weaknesses in an aca-demic program.

Figure 5. Checklist ofPossible Assessment Costs

Start-up CostsConsultant visitsConference attendanceCampus workshopsFaculty and staff timeDevelopment of assessment instruments

Continuing CostsComputer timePurchase of books and related materialsConduct and analysis of surveysTest purchase and scoringFaculty and staff time

66

. . . some of the "pioneer"institutions have developedlow-cost assessmentprograms largely throughreallocation strategies."

18

DETERMINING COSTS OF ASSESSMENT

Potential costs of assessment are a majorsource of uncertainty. How much does it cost totest or survey students, to administer anaanalyze questionnaires, to develop an assessmentcenter? Will assessment become a new budgetitem requiring large amounts of institutic al re-sources? Where will the money come from?

In fact, as Picture 5 suggests, several differenttypes of costs must be anticipated. During plan-ning, for example, funds may be needed to coverconsultant visits, trips to conferences or to othercolleges, or special campus workshops. To put anassessment plan in place, there may be costs forextensive faculty participation in the develop-ment of new tests or examinations; purchaseand scoring of standardized examinations;adaptation of existing computer informationsystems; or conduct of surveys among studentsor alumni/ae.

Once an assessment plan is in place, ongoingcosts may include costs for periodic surveys ortesting of student learning as well as necessarystaff support for the assessment activity. Othercosts, to be expected at all stages, include: re-leased time for a small number of faculty; staff as-sistance from the office of institutional researchor the office of the chief academic officer; addi-tional computer time to assemble and analyzeexisting information on student performance;and costs to purchase relevant books and otherresource materials.

How will all this be financed? Colleges thathave already developed assessment plans havetaken different approaches. There may be finan-ci.al assistance with planning and start-up costs,whether from a special grant from the state, afoundation or other source. Some institutions inVirginia and New Jersey received special fundingfrom state agencies, for example. Even a smallgrant from a foundation or private donor pro-vides an important catalyst for getting started.

On the other hand, some of the "pioneer" in-stitutions have developed low-cost assessmentprograms largely through reallocation strategies.They may give released time to committee mem-bers and assign new responsibilities to severaladministrators. The reallocation approach worksbest with aspects of an assessment plan thatbuild on existing activities. An alumni/ae affairsoffice may have conducted an occasional survey

23

of alumni/ae and now might plan for a survey ona regular basis. Existing information on place-ment test results may be given more detailedanalysis and wider circulation than before. Acurriculum committee's mandate might bebroadened to include review of assessmentresults.

Student fees are another funding source,especially approp -fate to cover costs for tests orassessments that offer specific benefits to stu-dents. One college, which has an extensive andindividualized assessment program, chargeseach student a one-time fee of $50 to coverassessment.

Most colleges have found that, in addition tothese special sources of funding, costs for assess-ment also must be budgeted on a regular basis.Among colleges that currently have programs inplace, costs typically have been $10 to $15 perstudent enrolled. There is no easy formula for es-timating these costs, although key factors in-clude the extent to which existing personnel cancoordinate the effort, and the nature and scope ofthe assessment plan. If the college already ham anextensive institutional research or testing pro-gram in place, a new assessment program simplymay require a redirection of effort. If assessmentactivities will be focused within academic depart-ments as part of ongoing planning or departmen-tal review efforts, additional costs may be modest.If an entirely new, multi-staged assessment isplanned, or if all students are to be assessed atseveral stages of their academic progress, costscould be substantial.

A valuable part of any planning committee'swork is to discuss the costs of various assessmentalternatives. The committee's charge mightspecify that any proposal must be cost-effectiveand realistic for the college's financial circum-stances over the next decade.

A budgetary framework might be establishedat the outset. Indeed, some approaches have verydifferent cost implications that are not im-mediately obvious: A "home-grown" test de-veloped by the college's faculty may appear to bea low-cost option, but might prove to be very ex-pensive because of the personnel costs involved.Different choices might be made that keep costsdown without losing good information. A small-scale survey of a sample of students may be asuseful as a canvas of the entire student body.

"Student fees are anotherfunding source, especiallyappropriate to cover costs

jor tests or assessmentsthat offer specific benefitsto students."

9419

Figure 6. Examples of aPlanning Schedule

EXAMPLE 1. During the first year, there mightbe a campus-wide discussion of the concept ofassessment, what it means at this institution,possible approaches to assessment, etc. It isat this point that all the doubts and uncertain-ties need to be aired. By the end of the firstyear, however, there should be a good degreeof consensus regarding the general ap-proaches to assessment which the institutionwill be using. During the second year, the as-sessment committee would make decisionsabout specific processes (instruments, re-ports, etc.) that the institution will use. Thecommittee keeps the entire campus informedthroughout this decision-making process.Third-year activities would focus on pilot-test-ing instruments (tests, questionnaires, etc.)and analyzing these preliminary data. By theend of the third year, the institution shouldknow what modifications it needs in the basicplan for assessment, so that the plan can befully implemented in the fourth year.

EXAMPLE 2. Another approach to planning forassessment is less structured and may bemore typical for college campuses. First, thepresident and chief academic officer set theplanning process in motion, and appoint an as-sessment committee, comprised mainly of fac-ulty members. During the first year, the as-sessment committee generates several ideasand sponsors faculty forums for discussion ofpossible approaches to assessment. Duringthe second year, the committee sponsors ex-perimental or "pilot" projects that try out differ-ent assessment approaches. Each of theseprojects may focus on a single academic de-partment, a small numbor of students, or a par-ticular method of assessment. During the thirdyear, the committee and the faculty review theresults of the pilot projects. They recommendsuccessful projects, or aspects of them, for in-stitution-wide use and suggest other newideas to try.

20

Some colleges have introduced aspects of as-sessment as part of their planned activity for ac-creditation self-studies. Phasing of assessmentactivitiesand their budgetary requirementsmay be feasible: Individual schools and depart-ments may be involved in assessment on differenttime schedules; surveys of alumni/ae, or of em-ployers in the local community, may be con-ducted every few years rather than annually.

Budgetary needs for assessment should alsobe viewed in broader perspective. Assessment in-formation can be of great benefit to the college,offering internal committees and administratorsvaluable planning information as well as provid-ing external constituentspublic officials, legis-lators, the media, students, and parentswithspecial assurances about the worth of programsoffered. Administrators at several colleges havestated that an unexpected benefit has been addi-tional funding, whether from state sources orother sources, that is tied to the fact that thecollege was able to demonstrate the academic per-formance of its students and the strength of itsprograms.

ESTABLISHING A SCHEDULE

The development of effective assessment pro-grams takes time. Colleges and universities thattoday are recognized as leading institutions inthe field of assessment have been developingtheir assessment programs for a decade or more.And their programs continue to evolve.

Recognizing then, that a high-quality assess-ment program is not likely to emerge within sixmonths or a year, the president, in consultationwith an assessment committee, should establisha timeline for implementationprobably involv-ing activity over three to five years.

Two quite different examples of a three-yeartime schedule are shown in Figure 6.

No single model would be appropriate for allcolleges. Some institutions may require an addi-tional year or two for planning and exploratoryactivities. Other institutions may need to imple-ment a plan in less than three years; in suchsituations, it is important to allow sufficient flex-ibility in an assessment plan so that later modifi-cations can be made.

25

Conclusion:Using assessment to greatest effect

In the months ahead, most colleges and uni-versities are likely to be influenced by the debateover assessment. As the focus of activity moves tothe development of assessment plans on indi-vidual campuses, colleges and universities havethe opportunity to engage in assessment ac-tivities that make sense for their institution. Thedanger is that hasty reactions to mandates is-sued by state boards or other external agencieswill not be useful, either for the institution or forits students.

As Figure 7 suggests, the process of develop-ing an approach to assessment is much like otherefforts directed to academic improvement. Fac-ulty involverric..it is critical. Choices have to bemade from among many alternatives. An explor-atory or experimental stage is often necessary.The commitment of campus leaders is necessary,and fledgling efforts need both financial andmoral support. Communication with all affectedparties is also important, particularly when an ef-fort will require several years of planning.

Thoughtful leadership from presidents andchief academic officers along with meaningfulfaculty involvement are crucial if an assessmentplan is to benefit students and improve the in-stitution. A key value of any assessment ap-proach lies in the way it raises fundamental ques-tions about the effectiveness of undergraduateeducation. When campus leaderspresident.chief academic officer, administrators and fac-ulty leaders alikeconsistently raise such ques-tions and consider ways to improve the academicprogram based on reliable and appropriate infor-mation, the assessment effort will be functioningeffectively.

Figure 7. Advice fromAssessment "Pioneers"

Involve faculty at the outset. Tap their knowl-edge and opinions. Rely on faculty to pose thekey questions and to oropose ways to answerthose questions.

Provide sufficient resourcesespecially at thedepartment level.

Communicate with students about assess-ment plans. Be sure they understand how as-sessment activities will (or won't) affect them.

Use plain English. Don't get hung up onpsychometric or other jargon.

Get started, try something, expecting somechange in direction as you learn more.

BUT

Don't scramble to do just anything. Don't runfor the first test you hear about.

Start modestly if necessary, but be clear thatthe effort is not intended to be short-term.Recognize that most successful comprehen-sive assessment efforts have taken years todevelop.

Don't rely on any one test or measure for aca-demic decisions that affect students or pro-grams. Multiple measures provide a strongerbasis for decisions.

Don't let assessment become just another as-signment for some administrative office.

Stress that the process should lead to im-provements in the educational experience forstudents. Don't mix assessment procedureswith procedures to evaluate faculty.

26

SELECTED REFERENCES ON ASSESSMENT

Adelman, Clifford (Ed.) Assessment in Amer-ican Higher Education: Issues and Contexts. Of-fice of Educational Research and Improvement,U.S. Dept. of Education, Washington, D.C.,1986.

Alverno College Faculty. Assessment atAlverno. Milwaukee, Wisc.: Alverno CollegeProductions, 1979.

Assessing the Outcomes of Higher Educa-tion. Proceedings of the 1986 ETS InvitationalConference. Educational Testing Service, Prince-ton, New Jersey, 1987.

Astin, Alexander W. "Why Not Try Some NewWays of Measuring Quality?" Educational Rec-ord 1982, 63, 10-15.

Bok, Derek. "Toward education of quality"Harvard Magazine, MayJune, 1986 (49-64).

Bok, Der!k. "Toward higher learning"Change, NovemberDecember, 1986, 18, no. 6,(18-27;.

Boyer, Carol M., Ewell, Peter T., Finley, JoniE., and Mingle, James R. "Assessment and Out-comes Measurement: A View from the Sta vs."AAHE Bulletin, 39, no. 7, 1987 (8-12).

Definir and Assessing BaccalaureateSkills: Ten ease Studies. American Associationof State Colleges and Universities, Washington,D.C., 1986.

El-Khawas, Elaine. Campus Trends, 1987.American Council on Education, Washington,D.C., 1987.

Ewell, Peter T. (Ed.) Assessing EducationalOutcomes. New Directions for Institutional Re-search, no. ,. 1. Jossey-Bass, Inc.: San Francisco,1985.

Ewell, Peter T. Assessment, Accountabilityand Improvement: Managing tne Contradiction.National Center for Higher Education Manage-ment Systems, Boulder, Colorado, May 1987.

Ewell, Peter T. and Dennis P. Jones. "TheCosts of Assessment," in Adelman, Assessmentin Higher Education: Issues and Contexts, pp.33-46.

Fincher, Cameron, Jones, Larry G., and

Placek, Joyce (Eds.). Higher Education in Geor-gia: Assessment, Evaluation, and Accredita-tion. Proceedings of the 1986 Conference. TheUniversity of Georgia (Institute of Higher Educa-tion, Office of Institutional Research and Plan-ning), Athens, Georgia.

Fong, Bobby. The External Examiner Ap-proach to Assessment. Association of AmericanColleges, June 1987.

Hutchings, Patricia. Six Stories: Implement-ing Successful Assessment. American Associa-tion for Higher Education, June 1987.

Lewis, Darrell R. and Wasescha, Anna M."Costs and benefits of assessment in post-secondary education." University of Minnesota,Minneapolis, Minnesota. Unpublished manu-script presented at the annual meeting of theAssociation for the Study of Higher Education,San Diego, California, February 16, 1987.

Marchese, Ted. "Learning about Assess-ment." AAHE Bulletin, September, 1986, 38, no.1, (10-13).

Mentkowski, Marcia and Georgine Loacker,"Assessing and Validating the Outcomes of Col-lege," in Fwell, Assessing Educational Out-comes, pp. 47-64.

Nettles, Michael T. "The Emergence of CollegeOutcome Assessments: Prospects for EnhancingState Colleges and Universities," Working Paper87-1. New Jersey State College Governing BoardsAssociation, Inc., Trenton, New Jersey, March1987.

Northeast Missouri State University. In Pur-suit of Degrees with Integrity: A Value-AddedApproach to Undergraduate Assessment. Ameri-can Association of State Colleges and Univer-sities, Washington, D.C., 1984.

A Proposal for Program Assessment at KeanCollege of Ne... :ersey. Kean College, Union, NewJersey, May 1986.

Read, Joel. Assessment/Evaluation andCosts. Alverno College Occasional Paper no. 5,Milwaukee, Wisconsin, 1985.

ACE BOARD OF DIRECTORS 1986-1987

Executive CommitteeFrank H. T. Rhodes, President,, Cornell

University, ChairJudith S. Eaton,, President, Community College

of Philadelphia, Vice ChairPhilip H. Jordan,, Jr., President, Kenyon College

Immediate Past Chair

Harold W. Eickhoff, President,, Trenton StateCollege, Secretary

Robert H. Atwell, American Council on EducationPresident

Janet D. Greenwood, President, University of BridgeportEdward T. Foote II, President, University of MiamiAlfredo G. de los Santos Jr., Vice Chancellor

for Educational Development, MaricopaCommunity Colleges

Association RepresentativesR. Jan LeCroy,, Chancellor, Dallas County

Community College District,, AmericanAssociation of Community and Junior Colleges

Stephen Horn, President, California State University,Long Beach, American Association of StateColleges and Universities

Arnold Thackray, Director,, Center for the History ofChemistry, University of Pennsylvania, AmericanCouncil of Learned Societies

James H. Daughdrill,, Jr , President, Rhodes CollegeAssociation of American Colleges

John S. Toll, President,, University of MarylandCentral Administration at Adelphi, Association ofAmerican Universities

Francis J. Kerins, President, Carroll College (Montana),Association of Catholic Colleges and Universities

The Rev. Michael G. Morrison, S.J.,, PresidentCreighton University, Association of JesuitColleges and Universities

Robert A. Reichley, Vice President, UniversityRelations, Brown University, Council forAdvancement and Support of Education

Robert V. Cramer, President, Carroll College (Wisconsin),Council of Independent Colleges

R. R. Reid, Treasurer, Whitman College, "ationalAssociation of College and University BusinessOfficers

William P. Hytche, Chancellor, University of Maryland,Eastern Shore, National Association for EqualOpportunity in Higher Education

Joab M. Lesesne, Jr., President, Wofford College,National Association of Independent Colleges andUniversities

John A. DiBiaggio, President, Michigan State University,National Association of State Universities andLand-Grant Colleges

Class of 1987Roscoe C. Brown, Jr., President, Bronx Community

College,, City University of New YorkWilliam M. Fulkerson, Jr., President, Adams State

College

Janet D. Greenwood, President,, University of BridgeportWilliam J. Maxwell, President, Jersey City

State College

Patsy H. Sampson, President,, Stephens College

James J. Whalen, President, Ithaca College

Class of 1988Robert L. Albright, President,, Johnson C. Smith

University

David W. Breneman, President,, Kalamazoo CollegeEdward T. Foote II,, President, University of Miami

William P. Gerberding, President, University ofWashington

Bruce E. Whitaker, President, Chowan CollegeBrunetta R. Wolfman, President, Roxbury

Community College

Class of 1989Sister Magdalen Coughlin, CSJ, President, Mount

St. Mary's College

Alfredo G de los Santos Jr ,, Vice Chancellor forEducational Development, Maricopa CommunityColleges

James 0. Freedman, President, Dartmouth CollegeWilliam H. Harris, President, Paine College

Stanley 0. Ikenberry, President, University of IllinoisBetty Lentz Siegel, President, Kennesaw College

2

7187