1

ASSESSMENTASSESSMENT

2

WHAT IS ACADEMIC WHAT IS ACADEMIC ASSESSMENT ?ASSESSMENT ?

THE SYSTEMATIC COLLECTION, REVIEW, AND USE OF INFORMATION ABOUT ACADEMIC PROGRAMS IN ORDER TO IMPROVE STUDENT LEARNING

Source:Catherine PalombaBall State University

3

WHAT IS ACADEMIC WHAT IS ACADEMIC ASSESSMENT ?ASSESSMENT ?

THE APPLICATION OF MANAGERIAL AND EDUCATIONAL CONCEPTS TO THE DESIGN, ANALYSIS, AND IMPROVEMENT OF CURRICULUM DEVELOPMENT AND LEARNING PROCESSES.

4

EVOLUTION OF ASSESSMENTEVOLUTION OF ASSESSMENT

PRIVATE COLLEGESPUBLIC UNIVERSITIESREGIONAL & PROFESSIONAL

ACCREDITATION AGENCIESSTATE LEGISLATURES AND

BOARDS OF EDUCATION

5

PRIVATE COLLEGESPRIVATE COLLEGES

MAJOR PROBLEM– THEY WERE BEING SQUEEZED OUT OF

THE UNDERGRADUATE MARKET AND WERE GOING BROKE

– THEIR BOARDS, MADE UP MAINLY OF BUSINESS AND INDUSTRY LEADERS, DEMANDED PROMPT AND POSITIVE RESULTS

6

PRIVATE COLLEGESPRIVATE COLLEGES

CONCLUSIONS– UNDERGRADUATE EDUCATION IS AN

ACADEMIC AND A SOCIALIZATION PROCESS

– HAD TO SHOW PROOF OF ADDITIONAL VALUE ADDED IN ORDER TO WARRANT TUITION RATES 4-5 TIMES STATE UNIVERSITY RATES

7

PRIVATE COLLEGESPRIVATE COLLEGES

CONCLUSIONS– THEY WERE IN THE KNOWLEDGE

BUSINESS; NOT THE RESEARCH, TEACHING, AND SERVICE BUSINESS

– THEIR FOCUS SHOULD BE ON THE UNDERSTANDING/ORGANIZATION/ STRUCTURING AND DISSEMINATION PARTS OF THE KNOWLEDGE MODEL OF HIGHER EDUCATION

8

PRIVATE COLLEGES PRIVATE COLLEGES

SOLUTIONS– EMPLOYED ABILITY BASED LEARNING

• STUDENTS SHOULD BE ABLE TO DO SOMETHING WITH WHAT THEY KNOW (Handout 1)

– REVISED CURRICULUM DESIGN• CHANGED FROM “MY COURSE MUST BE

IN THE PROGRAM BECAUSE IT IS WONDERFUL AND OUR DEPARTMENT NEEDS THE ENROLLMENT”

9

PRIVATE COLLEGES PRIVATE COLLEGES

SOLUTIONS– REVISE CURRICULUM DESIGN

• TO A PROCESS THAT INVOLVED – AN ANALYSIS AND DETERMINATION OF THE KNOWLEDGE

SETS, SKILLS, AND VALUES (KSVs) NEED TO BE A SOCIALLY AND ECONOMICALLY VIABLE GRADUATE

– DEVELOPMENT OF THE LEARNING OUTCOMES REQUIRED TO ACCOMPLISH THE KSVs

– ORGANIZATION OF KSVs AND ASSOCIATED LEARNING OBJECTIVES INTO COURSE OR MODULE UNITS

10

PRIVATE COLLEGESPRIVATE COLLEGES

SOLUTIONS– CHANGED MISSION STATEMENTS AND

REWARD SYSTEMS TO REFLECT EMPHASIS ON ORGANIZING AND DISSEMINATION OF KNOWLEDGE

• UTILIZED BOYER’S IDEAS ABOUT SCHOLARSHIP

11

PRIVATE COLLEGESPRIVATE COLLEGES

SOLUTIONS– ADAPTED AND USED BUSINESS

MANAGEMENT CONCEPTS/METHODS • FUNCTIONS OF MANAGEMENT• SYSTEMS THEORY• CONTINUOUS IMPROVEMENT

12

MANAGEMENT CONCEPTS MANAGEMENT CONCEPTS

MANAGEMENT FUNCTIONSSYSTEMS THEORYCONTINUOUS IMPROVEMENT

13

MANAGEMENT FUNCTIONSMANAGEMENT FUNCTIONS

CONTROLLING

DIRECTING

PLANNING

ORGANIZING

14

PLANNINGPLANNING

EXTERNAL & INTERNAL ANALYSISMISSION & VISIONGOALS, OBJECTIVES &

STRATEGIESDEVELOP ACTION PLANSALLOCATION OF RESOURCES TO

PLANS

15

ORGANIZINGORGANIZING

STRUCTURE– FUNCTION– GEOGRAPHIC– PRODUCT– CUSTOMER– MATRIX

STAFFING

16

DIRECTINGDIRECTING

ADMINISTRATIVE & LEADERSHIP SKILLS

COMMUNICATIONMOTIVATIONGROUPSCHANGE

17

CONTROLLINGCONTROLLING

ESTABLISH STANDARDSMEASURE PERFORMANCECOMPARE PERFORMANCE

AGAINST STANDARDSDETERMINE AND APPLY PROPER

CORRECTIVE ACTION

18

MANAGEMENT FUNCTIONSMANAGEMENT FUNCTIONS

DIRECTING

PLANNING

ORGANIZING

1.Establish Standards2. Measure Performance3. Compare Performance To Standards4. Take Corrective Actions

19

SYSTEMS THEORYSYSTEMS THEORY

A SYSTEM IS DEFINED AS A GROUP OF OBJECTS THAT ARE INTERRELATED AND INTERDEPENDENT WITH EACH OTHER AND THEIR ENVIRONMENT SO AS TO FORM A WHOLE.

20

SYSTEMS THEORYSYSTEMS THEORY

1. OBJECTS/PARTS AND THEIR ATTRIBUTES2. INTERRELATIONSHIPS/INTERDEPENDENCIES3. BOUNDARIES4. GOAL SEEKING5. TRANSFORMATION PROCESS - INPUTS & OUTPUTS6. SYNERGISM/GESTALT7. HIERARCHY OF SUBSYSTEMS8. REGULATION9. ENTROPY/DISORDER

21

FUNCTIONS OF MANAGEMENT - FUNCTIONS OF MANAGEMENT - SYSTEMS PERSPECTIVESYSTEMS PERSPECTIVE

INTERNALANALYSIS

EXTERNALANALYSIS

GOALSOBJECTIVESSTRATEGIES

MISSIONVISION

STANDARDS,MEASUREMENT &

CORRECTIVE ACTION

IMPLEMENTACTION PLANS

COMMITMENTOF

RESOURCES

LONG RUNPLANS

- - - - - - - - - - - SHORT RANGE

PLANS

E

22

CONTINUOUS IMPROVEMENTCONTINUOUS IMPROVEMENT

PART OF THE QUALITY MOVEMENT ANY PROCESS CAN BE IMPROVED KEY METHOD TO REMAIN COMPETITIVE IN

TODAY’S CHANGING BUSINESS WORLD PROCESS REENGINEERING IS NEEDED

WHEN INCREMENTAL IMPROVEMENT IS NOT SUFFICIENT

23

PUBLIC UNIVERSITIESPUBLIC UNIVERSITIES

LAND-GRANT CONCEPTEMERGENCE OF STATE RESEARCH

UNIVERSITIESQUESTIONS ABOUT CURRICULUM &

UNDERGRADUATE EDUCATION (Handouts 2 & 3)

ACCOUNTABILITY MOVEMENT

24

PROFESSIONAL PROFESSIONAL ACCREDITING AGENCIESACCREDITING AGENCIES

AACSB– MISSION DRIVEN STANDARDS– ONE OF THE FIRST TO REQUIRE

LEARNING OUTCOMES AND ASSESSMENT

– MUST HAVE PROCESSES IN PLACE TO ASSURE HIGH QUALITY AND CONTINUOUS IMPROVEMENT

25

REGIONAL ACCREDITING REGIONAL ACCREDITING AGENCIESAGENCIES

SACS (Handout 4)

– ESTABLISH CLEARLY DEFINED INSTITUTIONAL PURPOSE

– FORMULATE EDUCATIONAL GOALS– EVALUATE ACHIEVEMENT OF GOALS– USE RESULTS TO IMPROVE PROGRAMS– MEASURES TO EVALUATE PROGRAMS

AND GENERAL EDUCATION

26

STATE LEGISLATURES AND STATE LEGISLATURES AND STATE BOARDS OF CONTROLSTATE BOARDS OF CONTROL

LEGISLATURE– HOT BUTTONS - ACCESS, AFFORDABILITY &

ACCOUNTABILITY– MORE THAN 40 STATES REQUIRE ASSESSMENT– PASS LAWS THAT REQUIRE STATE HIGHER ED

INSTITUTIONS TO TAKE SPECIFIC ACTIONS - SB 148

THECB (Handout 5)

– EACH INSTITUTION MUST REVIEW AND EVALUATE ITS CORE CURRICULUM AND REPORT THE RESULTS TO THECB

27

WHY DO WE HAVE TO DO IT?WHY DO WE HAVE TO DO IT?

LEGISLATURE– TOLD US TWO TIMES - THEN PASSED A LAW

THECB – IMPLEMENTED THE LAW (SB148) BY REQUIRING

EACH INSTITUTION TO REVIEW AND EVALUATE ITS CORE CURRICULUM AND REPORT THE RESULTS

SACS – REQUIRES A PROCESS FOR ASSESSMENT AND

CONTINUOUS IMPROVEMENT OF PROGRAMS AND GENERAL ED CORE

28

WHY SHOULD WE DO IT?WHY SHOULD WE DO IT?

HELPS US CLARIFY OUR GOALS FOR THE CORE

SHIFTS OUR THINKING FROM INPUTS TO PROCESSES AND OUTPUTS

A PROVEN TOOL TO HELP US IMPROVE THE CORE AND THE LEARNING ENVIRONMENT

29

GENERAL EDUCATION COREGENERAL EDUCATION CORE

UNT’S PLAN FOR ASSESSMENT AND

CONTINUOUS IMPROVEMENT

30

WHAT IS EXPECTED OF US ?WHAT IS EXPECTED OF US ?

AS THE STEERING COMMITTEE, WE ARE RESPONSIBLE FOR:

1. DESIGNING AN ASSESSMENT AND CONTINUOUS IMPROVEMENT PROCESS FOR THE UNIVERSITY’S GENERAL EDUCATION CORE,

2. DEVELOPING A PLAN FOR IMPLEMENTING THE PROCESS, AND

3. IMPLEMENTING THE PLAN AND MONITORING THE PERFORMANCE OF THE A&CI PROCESS.

31

DEVELOPING A PLANDEVELOPING A PLAN

WHAT NEEDS TO BE DONE ? HOW WILL WE DO IT ?WHO WILL DO IT ?WHEN WILL IT BE DONE ?

A good place to start is with the answers to the following:

32

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHAT ARE WE GOING TO ASSESS ?

HOW ARE WE GOING TO CONDUCT ASSESSMENT - WHAT TOOLS OR METHODS WILL WE USE ?

WHO IS GOING TO DO THE ASSESSMENT ?

33

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHEN AND HOW MUCH ?

HOW WILL THE RESULTS BE USED TO IMPROVE THE CORE CURRICULUM ?

WHO IS GOING TO PREPARE THE REPORTS FOR THECB AND SACS ?

34

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHAT KIND OF DOCUMENTATION WILL WE NEED ? – PROCESS ITSELF– RESULTS OF THE PROCESS– EVIDENCE TO PROVE THAT THE ASSESSMENT

RESULTS ARE USED BY CURRICULUM PLANNING OR OTHER PROCESSES TO IMPROVE THE CORE

35

CURRICULUM PROCESSCURRICULUM PROCESS

INTERNALANALYSIS

EXTERNALANALYSIS

GOALSOBJECTIVESSTRATEGIES

MISSIONGEN ED CORE

STANDARDS, MEASUREMENT,ANALYSIS AND

CORRECTIVE ACTION

IMPLEMENTGEN ED CORE

COMMITMENTOF

RESOURCES- - - - - - - - - - -

E

LEARNINGOBJECTIVES

GEN ED CORECURRICULUM

36

ASSESSMENT TOOLS ASSESSMENT TOOLS

SURVEYSTESTSPORTFOLIOSINSTITUTIONAL DATACAPSTONE COURSE PROJECTSCLASSROOM ASSESSMENTCURRICULUM REVIEW

Source:Catherine PalombaBall State University

37

SURVEYSSURVEYS

ADVANTAGES– COVER MANY DIFFERENT TYPES OF

INFORMATION AND GROUPS– SOMETIMES ENABLE NATIONAL COMPARISONS

DISADVANTAGES– TAKE TIME TO DESIGN – RESPONDENTS MAY NOT BE REPRESENTATIVE

Source:Catherine PalombaBall State University

38

TESTSTESTS

ADVANTAGES– PROVIDES INFORMATION ABOUT COGNITIVE

OUTCOMES– CAN COVER A WIDE RANGE OF CONTENT– CAN ASSESS A LARGE GROUP– GOOD FOR LONGITUDINAL DATA– GOOD FOR BEFORE AND AFTER MEASURES– NATIONAL EXAMS ARE AVAILABLE

Source:Catherine PalombaBall State University

39

TESTSTESTS

DISADVANTAGES– NATIONAL EXAMS

• MAY NOT MATCH CURRICULUM• ARE USUALLY EXPENSIVE

– LOCALLY DEVELOPED EXAMS• CAN BE TIME CONSUMING TO DEVELOP• MAY BE DIFFICULT TO GET CONSENSUS• LACK OF EXTERNAL NORMS

Source:Catherine PalombaBall State University

40

PORTFOLIOSPORTFOLIOS

ADVANTAGES– CAN BE INTEGRATED WITH COURSEWORK– PROVIDES INFORMATION OVER TIME

DISADVANTAGES– RAISE MANY PRACTICAL QUESTIONS– CAN BE DIFFICULT TO EVALUATE

Source:Catherine PalombaBall State University

A collection of student work generated over time

41

INSTITUTIONAL DATAINSTITUTIONAL DATA

ADVANTAGES– AVAILABILITY– MAY ALREADY BE IN A DATABASE

DISADVANTAGES– MAY NOT MATCH CURRICULUM – MAY BE TIME CONSUMING AND EXPENSIVE TO

ANALYZE

Source:Catherine PalombaBall State University

42

CAPSTONE COURSECAPSTONE COURSE

ADVANTAGES– PROVIDES A TIME AND PLACE TO UNDERTAKE

ASSESSMENT

DISADVANTAGES– IS NOT ASSESSMENT BY ITSELF– NEEDS TO BE PART OF AN OVERALL PLAN

Source:Catherine PalombaBall State University

43

CLASSROOM ASSESSMENTCLASSROOM ASSESSMENT

ADVANTAGE– A LOT OF GOOD DATA

DISADVANTAGE– HARD TO USE THE DATA

Source:Catherine PalombaBall State University

Use of small-scale assessment techniques conducted by teachers to determine what students are learning in particular classes

44

CURRICULUM REVIEWCURRICULUM REVIEW

ADVANTAGES– COMPARES COURSE LEARNING OBJECTIVES

TO PROGRAM OBJECTIVES– EVALUATES THE WHOLE PROGRAM, NOT JUST

THE PARTS

DISADVANTAGES– TAKES A LOT OF TIME– POLITICALLY CHARGED

Source:Catherine PalombaBall State University

45

ASSESSMENT TOOLS( )ASSESSMENT TOOLS( )

SURVEYSTESTSPORTFOLIOSINSTITUTIONAL DATACAPSTONE COURSE PROJECTSCLASSROOM ASSESSMENTCURRICULUM REVIEW

Source:Catherine PalombaBall State University

Also See SACSHandout

46

OUR A&CI PROCESS AND PLAN OUR A&CI PROCESS AND PLAN MUST:MUST:

MEET ALL THE THECB REQUIREMENTS

BE DESIGNED TO ALSO ACCOMPLISH SACS REQUIREMENTS

MEET OUR OWN NEEDS FOR A PROCESS THAT WILL ASSURE EXCELLENCE OF UNT’S GENERAL EDUCATION CORE

47

THECB REQUIREMENTS ( )THECB REQUIREMENTS ( )

Extent to which the [institution’s core] curriculum is consistent with the elements of the core curriculum recommended by the Board;

Extent to which the [institution’s core] curriculum is consistent with the “Texas Common Course Numbering System”;

SEE THECB HANDOUTS 5 & 6

48

THECB REQUIREMENTS THECB REQUIREMENTS

Extent to which the curriculum is consistent with the elements of the core curriculum component areas, intellectual competencies, and perspectives as expressed in “Core Curriculum: Assumptions and Defining Characteristics” adopted by the Board; and

49

THECB REQUIREMENTS THECB REQUIREMENTS

Extent to which the institution’s [core curriculum] educational goals and the exemplary educational objectives of the core curriculum recommended by the Board are being achieved.

50

THECB REPORT REQUIRES( )THECB REPORT REQUIRES( )

A table that compares the institution’s core curriculum with the core component areas and exemplary educational objectives of the core curriculum recommended by the Board;

A brief description of the purpose and substance of the institution’s core curriculum;

SEE THECB HANDOUTS

51

THECB REPORT REQUIRES( )THECB REPORT REQUIRES( )

A description of the processes and procedures used to evaluate the institution’s core curriculum; and

A description of the ways in which the evaluation results are utilized to improve the core curriculum at the institution.

SEE THECB HANDOUTS

52

SACS REQUIREMENTS( )SACS REQUIREMENTS( )

Establish a clearly defined purpose appropriate to collegiate education,

Formulate educational goals consistent with the institution’s purpose,

SEE SACS HANDOUT

The institution must:

53

SACS REQUIREMENTS( )SACS REQUIREMENTS( )

Develop and implement procedures to evaluate the extent to which these educational goals are being achieved, and

Use the results of these evaluations to improve educational programs, services, and operations.

SEE SACS HANDOUT

54

OUR REQUIREMENTS OUR REQUIREMENTS

Is our core curriculum an integrated whole or a collection of parts ?

Are the current goals and objectives of UNT’S general education core sufficient ?

Does the design and content of the general education core meet the needs of our students ?

55

WHAT IS EXPECTED OF US ?WHAT IS EXPECTED OF US ?

AS THE STEERING COMMITTEE, WE ARE RESPONSIBLE FOR:

1. DESIGNING AN ASSESSMENT AND CONTINUOUS IMPROVEMENT PROCESS FOR THE UNIVERSITY’S GENERAL EDUCATION CORE,

2. DEVELOPING A PLAN FOR IMPLEMENTING THE PROCESS, AND

3. IMPLEMENTING THE PLAN AND MONITORING THE PERFORMANCE OF THE A&CI PROCESS.

56

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHAT ARE WE GOING TO ASSESS ?– EXAMPLES

HOW ARE WE GOING TO CONDUCT ASSESSMENT - WHAT TOOLS OR METHODS WILL WE USE ?

WHO IS GOING TO DO THE ASSESSMENT ?

57

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHEN AND HOW MUCH ?

HOW WILL THE RESULTS BE USED TO IMPROVE THE CORE CURRICULUM ?

WHO IS GOING TO PREPARE THE REPORTS FOR THECB AND SACS ?

58

DEVELOPMENT OF OUR A&CI DEVELOPMENT OF OUR A&CI PROCESS AND PLANPROCESS AND PLAN

WHAT KIND OF DOCUMENTATION WILL WE NEED ? – PROCESS ITSELF– RESULTS OF THE PROCESS– EVIDENCE TO PROVE THAT THE ASSESSMENT

RESULTS ARE USED BY CURRICULUM PLANNING OR OTHER PROCESSES TO IMPROVE THE CORE