Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M....

Culture of AssessmentCulture of Assessment

CREATING A CULTURE OF ASSESSMENT

Dean Kenneth B. Roberts, Ph.D.Heidi M. Anderson, Ph.D.

Eleanora Bird, M.S.Jeff Cain, M.S.

Office of Education InnovationUniversity of Kentucky College of Pharmacy

Closing the Loop


2

Outline Literature “Creating a Culture of

Assessment” UKCOP Case Study

Created an office to handle the College assessment efforts

Faculty development and involvement Student involvement Documenting current assessment efforts

Future Directions at UKCOP


Creating a Culture

of

Assessment 3


4

Culture of Assessment1. Leadership must be ‘on board’

Mission, resources, use results for improvement

2. Involve all stakeholders Faculty Students Alumni


5

Culture of Assessment3. Make assessment relevant

Course reviews Surveys, rubrics

4. Focus on teaching and learning rather than assessment Focus conversations not on

assessment, but on helping students learn


6

Culture of Assessment5. Provide opportunities to learn about

assessment6. Minimize the burden of assessment on

faculty and students Provide centralized leadership,

coordination and support for assessment Provide resources Keep paperwork to a minimal


7

Culture of Assessment7. Start small, Set up clear expectations and be

flexible8. Celebrate and reward assessment efforts

Give faculty and staff written recognition Encourage leaders to commend publicly Provide special funding Honor faculty and staff who obtain external

grants that will strengthen assessment


8

Culture of Assessment9. Use assessment results appropriately

Keep faculty, students and staff informed

Don’t penalize faculty whose assessments results are less than positive

10. Don’t expect everyone to get on board


(UKCOP)

9


10

Vision Office of Education Innovation (OEI)

was developed to assist faculty Teaching Learning ASSESSMENT Technology Scholarship of Teaching

Today


11

Mission The mission of the “Office of Education

Innovation” (OEI) is to provide assistance to faculty in developing and evaluating the professional pharmacy and graduate programs within the University of Kentucky, College of Pharmacy (UKCOP).


12

Community Dean’s Leadership Team Office of Academic Affairs Office of Education Innovation (4 FTEs)

Assistant Dean Assessment Coordinator Information Technology Officer Administrative Assistant

Faculty Students Other Participants

TEAM


13www.mc.uky.edu/Pharmacy/oei


14

Who the office works with? The “Office of Education Innovation” (OEI)

works closely with: UKCOP Course Coordinators/Directors UKCOP Individual Faculty UKCOP Departments UKCOP Students External Constituents to College


15

OEI Activities and Services Curriculum

Assessment Instructional Technology Faculty Development (Enrichment) Scholarship of Teaching

Today


UKCOP Faculty Development: How, What and

Why of Assessment?

16


17

Why do assessment? Accountability

ACPE Accreditation University Review Curriculum committee requirements

Enhance student’s educational development Decision-making


18

Accreditation July 2000 accreditation standards require

assessment activities to be aligned with mission and goals: achievement indicators, formative and summative formats, outcomes assessment, and systematic assessment plan.


19

Assessment ProcessDriven by Mission

Used forImprovement Ongoing

Use Multiple Measures


20

What is the Assessment Process? “Systematic collection, review and use of

information about educational programs undertaken for the purpose of improving student learning and development”

Palumbo and Banta, 1999, p. 3


21

Levels of Assessment Institutional

Academic CURRICULUM COURSES Teaching-

learning Methods Assessment

Methods

Individual STUDENTS:

Learning Performance Attitudes Behaviors Skills Satisfaction


22

Levels of Assessment Institutional

UNIV REVIEW ACCREDITATION

Individual Faculty

SELF Teaching Portfolio

Peer Observation

Scholarly Productivity


23

What is Assessed? Student

Student Learning Outcomes

Performance Faculty

Contribution to teaching Scholarly productivity National visibility

Program Graduation rates Licensure pass/fail Curriculum Courses Teaching-Learning

Methods


24

Difference: Student Assessment vs. Program Assessment

Student Assessment is a teaching process designed to improve learning

Program Assessment is an institutional process designed to improve the program


25

Student Assessment Carefully define your student learning

outcomes If you don’t know what you’re looking for,

you will have a difficult time measuring it. Outcome statements should describe

student learning or behavior Outcome: “the result”


26

Student Assessment Start simple. Choose a limited number

of outcomes to measure at first. Begin modestly so that the data

collected can help to define and refresh accomplishments


Faculty Involvemen

t

27


28

Faculty Involvement Faculty learning about assessment:

Website developed to keep them abreast of activities

Faculty participation on committees: Assessment committee Curriculum committee


29

Faculty Involvement Self-study reports

Curricular purposes Curricular Outcomes Course Review (online submission) Course Evaluation (online)

Survey developmentPS Graduate Program

Requested peer evaluation of teaching


30

UKCOP General Outcomes

Foundation Knowledge Critical Thinking Information Management Communications Professionalism Social Interaction, Citizenship, Leadership Life-long Learning


31

UKCOP Professional Outcomes Patient Care

Medication Distribution Human Resources Management and

Operations Research Teaching


32

Faculty Involvement Focus on Teaching/Learning

Generate the appropriate assessment questions

Determine appropriate ways to measure outcomes

Decide on when to assess the students Prepare and utilize the reports


33

Faculty Involvement

Developing Questions to Ask: What do we want students to know, and do? How will we know they are learning what we

hope they will? What is the effect of our curricula, courses,

teaching, styles and educational tools? Did our changes bring about the effect we

desired?


Student Involvemen

t

34


35

Student Involvement Incoming orientation:

Explain the College’s assessment, rationale relating to accreditation standards, role of OEI

Review IRB protocol and informed consent Assessments throughout their 4 years Anonymous (code for tracking) Reports in aggregate fashion


36

Student Participation Curriculum Committee

Student Liaison Committee (SLC) meets 3x per semester

Assessment Committee Course

Performance Assessments Other

Accreditation Visit, Program Review


UKCOP Case Study:Documenting Current Effort

37


38

Documenting Current Efforts

Purpose Determine what assessments were already

taking place within the College

Develop an assessment inventory list and appropriate questions about student learning


39

Self-assessment of outcome abilities Attitudes towards peer/self assessment Performance records from courses CAPP results (integrated abilities) Student portfolios Yearly interviews Post-graduation surveys

Goal:

Develop a Complete Student Profile


40

Three Phases Phase 1: Needs Analysis

Phase 2: Recommendations and Plan Development

Phase 3: Align Assessment Plan, Outcome Indicators


41

Research Literature review was conducted to

determine types of assessment data that might be useful to gather for the program’s assessment functions Student records Program evaluations Student performance and competencies

Needs Analysis


42

Data Audit Class of 2002-2005: Student and educational

outcomes records Data obtained from various offices and files within

the college: Demographic Entry-exam scores Learning styles GPAs

Needs Analysis


43

Data Audit Data portfolios for several student cohorts

were examined Faculty Other sources:

Institutional Research Office Boards of Pharmacy State/local pharmacy associations

Needs Analysis


44

Data Audit Report


45

Success External IR officer requested to use the

college audit form as a model for the campus

Data are being collected from all audiences in a wide variety of venues

Some data are being disseminated in meaningful way

Data AuditNeeds Analysis


46

Lessons Learned from Data Audit

Amount & breadth of information was considerable

Gaps/Duplication

Some areas require more or different data


47

Lessons Learned from Data Audit

Length and frequency of surveys Faculty engagement Incorporate findings into College

assessment plan Reassess


48

Phase 2Recommendations about research

questions to assessment committee

I-E-O data management model used to classify student and program data (Astin, 1993)

I=inputs E=environments O=outcomes


49

Phase 2A. Content Analysis

B. Educational Impact

Which educational experiences/environments maximize student learning during the first year of pharmacy school?


50

Phase 2Ho: Structured Clinical Instructive Models

(SCIMs), prior to students participating in EPPE, statistically improve student critical thinking skills.

Ho: Videotaped counseling sessions statistically improve student communication skills during the first year of pharmacy school.


51

PHASE 3

Align and map curricular outcomes

Disseminate reports and use for improvement


Future Assessment Directions

52


53

Next Steps OEI reviewing literature Formulating appropriate research questions Prioritization and approval of research

questions by assessment committee Identifying appropriate measures Conducting the analysis Reporting and using the results for

improvement


54

Assessing Faculty Teaching

Classroom teaching Portfolio

Student advising Scholarship of

Teaching Publications Presentations

Supervision of clinical experiences

Supervision of graduate students

Consultation about teaching

Next Step:


Other Activities of the Office of

Education Innovation (OEI)

55


56

Instructional Technology Introduce and foster the use of

innovative technologies that enhance teaching, learning and assessment

Consultation and chair side assistance Remove obstacles


57

Assessment Software CourseEval

TrueOutcomes


58

Technology Smart classrooms Blackboard Audience Response System PDAs FLASH animations Assessment hardware/software


59

Faculty Enrichment Education Research Focus Group (2002)

Seminars Technology (primary focus) Teaching (Forthcoming)

Learning (Forthcoming)

Assessment (Forthcoming)

Website: primer materials

http://www.mc.uky.edu/Pharmacy/oei/faculty_development.asp


60

Scholarship of Teaching Publications Presentations

AACP Annual Meeting poster, presentation Other Presentations

Syllabus International Education Conference Assessment Institute

http://www.mc.uky.edu/Pharmacy/oei/faculty_assessment.asp


61

Summary Assessment evaluates the effectiveness of the

program in terms of measurable outcomes The plan consists of:

Student learning outcomes Measures of student achievement of the

outcomes (and objectives) Methods of collecting data Process for using and reporting the results Assessment of the process


62

Summary Fostering a culture of assessment:

Takes time and commitment Is challenging Involves various stakeholders at some

levels Requires resources Continuous


63

http://www.mc.uky.edu/Pharmacy/oei/faculty_assessment.asp

Click to edit Master title style

Click to edit Master text styles Second level

Third levelFourth level

Fifth level

04/18/23 1 64Culture of AssessmentCulture of AssessmentCulture of AssessmentCulture of Assessment

Questions ?

64

Date post:	16-Dec-2015
Category:	Documents
Upload:	adam-griffith
View:	216 times
Download:	0 times

Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M....

Documents