1

SUSAN BECK-FRAZIERSUSAN MORRISSEY

KRISTEN SPRINGER DREYFUSALEX SENIOR

2013

Assessment Review Committees

Training Sessions

2

Why is the assessment review process important?

ECU is committed to documenting the quality and effectiveness of all its programs and services.

Responsibility for assessment SACS Comprehensive Standard 3.4.10 – The institution places primary

responsibility for the content, quality, and effectiveness of the curriculum with its faculty

SACS Comprehensive Standard 3.3.1.2-5 - The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in: administrative and academic support services, research and public service

ECU Accountability-statewide push Our Time, Our Future 2013: Strengthening Academic Quality

3

Faculty and Staff

Unit Assessment Coordinator

Assessment Review

Committees

Institutional Assessment

Advisory Council

Institutional

Effectiveness Council

ECU Assessment Report Review Model

4

Why am I here?

All educational programs and support units on campus have developed assessment plans and annually report results, actions taken, and follow up to the previous year’s actions taken.

March 2013 the Academic Council approved the Assessment Review Committee (ARC) structure in order to facilitate the institutional effectiveness assessment process at ECU.

ARCs are chaired by representatives on the Institutional Assessment Advisory Council (IAAC).

You have been asked to be an ARC member.

5

Roles and Responsibilities

The duties of the Assessment Review Committee (ARC) include: Guiding faculty and staff in adhering to university and

internal review criteria and deadlines Reviewing and evaluating the quality of assessment

reports using the approved university’s rubric Working with assigned ARC chair to prepare the

summary report due in the spring

6

Units/ARC Chairs/OIA Resource Person

Academic Units/Divisions          ARC Chair/IAAC Member OIA Resource Person BSOM                                    Rich Franklin                       Kristen Dreyfus COE                                        Diana Lys                           Kristen Dreyfus CAHS                                     Kevin O’Brien                     Kristen Dreyfus CON                                       Mary Holland                    Kristen Dreyfus CHHP                                    Susan McGhee                Kristen Dreyfus TECS                                     Leslie Pagliari                  Kristen Dreyfus CFAC                                     Jeff Ward                            Susan Morrissey CHE                                        Ginger Woodard             Susan Morrissey SODM                                    Geri Crain                           Susan Morrissey COB                                       Shanan Gibson                Susan Beck-Frazier HCAS           

Undergraduate                 Don Bradley                        Susan Beck-Frazier Graduate                              Heather Ries                     Susan Beck-Frazier Foundations/Support     George Bailey                    Susan Beck-Frazier

Administrative Units/Divisions                                                 Academic Affairs              Karen Summey              Susan Beck-Frazier Admin. & Finance            Steve Duncan                    Susan Morrissey Chancellor’s Division/     Erin Eatman                        Susan Morrissey Division of Univ. Adv.     Kay Murphy Division of HS                     Patricia Fazzone                Susan Morrissey Division of RGS                  Ron Mitchelson                Susan Morrissey Division of SA                     Kathy Hill                             Susan Morrissey

7

What am I reviewing?

Educational Program Assessment Reports

3 Program Learning Outcomes

2 Institutional Learning Outcomes (global & leadership)

Means of Assessment Criterion for Success 2012-13 Results 2012-13 Actions Taken 2011-12 Follow-up to

Actions Taken

Support Unit Assessment Reports 3 Outcomes Means of Assessment Criterion for Success 2012-13 Results 2012-13 Actions Taken 2011-12 Follow-up to

Actions Taken

8

What should I see in the assessment report?

Educational units Program Learning Outcomes

Directly related to student learning Description of what students should know, think or do by the time they

complete a degree program (undergrad, grad, certificate, stand-alone minor, foundations)

Institutional Learning Outcomes Reminder: No rating needed for the component – Outcome Statement – for

the Global and Leadership Outcomes

Support units Outcomes

Directly related to the functions of the unit Description of what the office/center does, why it is part of ECU

9

What should I see in the assessment report?

Means of Assessment Description of how data is collected Clearly measure the outcome Direct measure must be used for educational units Multiple means of assessment must be used for at

least one outcome (direct and indirect, as appropriate) Assessment instruments attached, where appropriate

Criterion for Success Description of how well students/unit should perform Clearly matches the means of assessment

10

What should I see in the assessment report?

Results Summary of the data collected using the Means of

Assessment Clearly state whether or not the Criterion for Success

was metActions Taken (Use of Results)

Summary of what has been done, based on analysis of Results, to improve the program/unit

Follow-up to Actions Taken Description of whether or not the Actions Taken from

the previous year worked to improve the program/unit Directly linked to the Results collected in the current

reporting year

11

What am I looking at in the ECU Assessment Report (4-column report)? Educational Program

12

What am I looking at in the ECU Assessment Report (4-column report)? Support Unit

13

Demonstration of running 4 column report in TracDat

Log in to TracDat with your ECU username and passphrase

From the drop down menu, select the unit you have been assigned to review

Click Reports Choose “run” next to ECU ASSESSMENT REPORT Choose the following:

Currently Being Assessed Institutional Learning Outcome & Program Learning

Outcome (for educational programs only) Reporting Years 2011-12 and 2012-13 Open Report

14

Demonstration of Completing Rubric in SharePoint

Go to IPAR’s homepage – www.ecu.edu/iparUnder Highlights (middle column) click Assessment Rubric

Click New RubricFollow instructions

15

IPARASSESSMENT: 2012-2013 Assessment Report Review Feedback (unit name)

ECU Collaborate Workflow Notification

East Carolina University is committed to documenting the quality and effectiveness of all its programs and services. The Office of Institutional Assessment, the Institutional Assessment Advisory Council (IAAC), Assessment Review Committees and Unit Assessment Coordinators work together to ensure that an effective and timely review process is completed. This formal review is done with the use of a rubric (one for educational programs and one for support units – both attached).

Review feedback for your unit can be viewed by clicking here  If you are receiving this email as the Unit Assessment Coordinator (previously called the Responsible Assessment Person or RAP), please share the information with your faculty or office personnel. Department chairs and directors, please review the feedback information with faculty at an upcoming departmental meeting. Department chairs, directors and IAAC representatives have been copied as well.

If any part of your assessment report was rated “developing” or “acceptable”, discuss the areas of concern with faculty and office personnel and make the necessary changes in TracDat. It is recommended that the attached rubrics be used for guidance. For specific questions or points of clarification, contact the member of your Assessment Review Committee who reviewed your report. The reviewer’s name can be found at the top of the rubric.

16

Next Steps

Attend meetings of your ARC in order to contribute to discussions that facilitate authentic assessment

Contact assigned UACs before the reporting due date to remind of deadlines (August 1 for support units, October 15 for educational programs)

Review 2012-13 assessment report for assigned units and complete the rubric (by October 15 for support units, December 15 for educational programs)

Follow up with assigned UACs to verify they have received feedback and to ensure that changes are made in Tracdat

Work with ARC chair to prepare presentation for IAAC (spring 2014)

17

Frequently Asked Questions

Do I rate the Global and Leadership Outcome Statements? No because the Global and Leadership outcomes are currently

established by the institution. How do I rate outcomes with multiple means of assessment (MOA),

criterion for success (CFS), results (R) and actions taken (AT)? The lowest rating among the multiple means of assessment,

criterion, results and actions taken is the reported rating. Comments are provided for each of the multiple entries.

For example, MOA1: assessment instrument needs to be attached. MOA2:proficient. MOA3: Assessment instrument vaguely described – I do not understand what it is.

What do I do if a Follow Up is not appropriate because this is the first year of results? Select the “Acceptable” rating and provide comments that although

none is available, it is appropriate due to the fact that this is the first year of results.

18

Frequently Asked Questions

What do I do if the 4 column assessment report is blank? Contact your OIA resource person. It is possible that

the unit has turned off their current outcomes and created new ones for the upcoming year.

What do I do if I have questions regarding the assessment review of my units? Contact your ARC chair

What do I do if I have questions regarding the SharePoint technology? Contact Alex Senior (seniora@ecu.edu)

19

Helpful Hints

Review a report together for inter-rater reliability

Read through the report thoroughly before selecting outcomes to review and starting the rubric

Complete in Word for spellcheck Complete an entire assessment report

rubric at one time

20

Additional Notes

Saved rubrics can be accessed using the link that SharePoint sends once the “save” button at the bottom has been selected

Email your ARC chair with questions as soon as you have them

If you would like to practice completing a rubric, make sure you select Test College/Test Unit

21