8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 1/46
Evaluating & Assessing IPAS Projects:
Stories from the Field
2015 NERCOMP Annual Conference, 31 March 2015
D. Christopher Brooks, Ph.D.
Senior Research Fellow
ECAR
Laura Damm
Student Success Coordinator
East Mississippi Community College
Lois Smith
Vice President, Organizational Growth, Development & Effectiveness
Stanly Community College
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 2/46
Overview
What is IPAS?
ECAR IPAS Projects
Methodological Issues
Case Studies
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 3/46
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 4/46
ECAR
IPAS Research Hub
http://tinyurl.com/k95hr7t
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 5/46
IPAS Evaluation & Assessment Guide
•
Why the evaluation and assessment to IPASprojects is important
• Issues and approaches to designing an evaluation
and assessment protocol
•
Issues and approaches related to measuring andanalyzing outcomes data
• Identifying good practices in assessing and
evaluating IPAS projects
• How IPAS evaluation and assessments lead to the
identification of actionable outcomes
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 6/46
Evaluation versus Assessment
#
Evaluation
Assessment
Question
How do we know
the efforts are orhave been worth it?
How do we know if
it works or worked?
Objects
Worth or merit of a
project Impact of a project
Aspects
Procedurally-focused
Judgmental
Contextual
Outcomes-focused
Diagnostic
Empirical
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 7/46
Evaluation versus Assessment: Timing
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 8/46
Evaluation & Assessment:
Levels of Analysis
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 9/46
Assessment Designs
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 10/46
Assessment Designs
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 11/46
Case Studies
Stanly Community College
29-30 January 2015
Montgomery County Community College
2 February 2015
East Mississippi Community College
17-18 December 2015
Colorado State University
5 February 2015
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 12/46
East Mississippi Community College
Expand the Pride
March 31, 2015
Laura Damm
Student Success Coordinator
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 13/46
Campuses
Scooba- The main campus (est.1927) is
residential, host to athletics, band, and chorus inaddition to academic and limited technical
programs. 1,126 students
Mayhew- The most populous, commuter campus
(est. 1968) houses GED and Workforce training
services in addition to academic and technical
programs. 3,673 students
Four extension centers offer unique technicalprograms. Online course offerings began in 2000.
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 14/46
Demographics
EMCC enrolls about 6,900 students annually
Ethnicity is 49% White, 47% Black, 4% Other
Female (58%)
Attends full time (57%)
Average age is 23.5
Most are commuters
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 15/46
Technology solutions
1. DropGuard by Smartevals
Early Alert system driven by instructor input
Replaced homegrown process initiated 2009and stalled in 2010-11. Went live fall 2012.
2. Student Planning by Ellucian
Education planning tool, asynchronousinteraction with advisor, capability to register
with one click. Went live fall 2013.
1.
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 16/46
Project Goals
1. Increase retention of new cohort 10%
2. Early Alert used by 100% FT instructors;75% adjuncts in year 1 and 85% in year 2
3. Student Planning used with 25% new cohort inyear 1 and 50% in year 2
4. New cohort demonstrates 5% improvement in
retention, persistence, and graduation/transfer
1.
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 17/46
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 18/46
Usage of Student Planning by Students
in FTFT Fall 2013 cohort
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 19/46
This is a viable tool that allows a second
party to reach out to my students viaemail contact and phone calls. Through
this tool, students are encouraged to
meet with tutors, use technology more
effectively, and have a one-on-one
question and answer session thataddresses EMCC policy and procedures.
-S. Frey, faculty
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 20/46
CompletionCohort: First time, full
time students enrolled at
10th day audit.Q: Of those who started,
how many completed?
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 21/46
Retention/Course Completion
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 22/46
Re-enrolledCohort: First time, full
time students enrolled
at 10th day audit.Q: Of those who started,
how many re-enrolled?
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 23/46
Retention/Re-enrolled
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 24/46
WarningsStudents who received a warning for
poor test, missing assignment,
online absence, absent from class,and missing English essay.
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 25/46
Warned with Early Alert System
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 26/46
IntervenedIntervened: Defined as
on ground tutoring
due to inability to trackindividual students in
NetTutor
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 27/46
Intervened with on ground tutoring
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 28/46
Warned and Finished
How many students in the
cohort were warned and
finished the term?
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 29/46
Warned and Finished
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 30/46
Tutored and Finished
How many students in the
cohort were tutored and
finished the semester?
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 31/46
Tutored and Finished
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 32/46
Lions’ Brew opened Sept 2014. Usage
of tutoring services increased 155%.
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 33/46
Evaluating and Assessing IPAS
Projects: Stories from the Field
March 31, 2015
Lois Smith
VP Organizational Growth, Development, and
Effectiveness
Stanly Community CollegeNorth Carolina
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 34/46
Stanly Community College
!
A little about us! Small, primarily rural, community college in south
central North Carolina
! One of 58 community colleges in NC
!
SCC serves Stanly County
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 35/46
Stanly Community College
! A member college of Achieving the Dream
and of the Completion by Design NC cohort
! Curriculum student headcount of
approximately 2800
! Smallest college in the Gates FoundationIPAS Grant Program
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 36/46
Stanly Community College
! IPAS Project at Stanly• The technology selected was Student Success Plan (SSP)
• SSP is open source software that was integrated with our
Moodle LMS and our Ellucian SIS
• Used two partners to assist with implementation - Unicon and
Gateway to College
• Users throughout the institution are utilizing SSP for: Early Alert, Journaling, and My Academic Plan (MAP) for academic
planning
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 37/46
Stanly Community College
! Goals and Outcomes
Upon completion of the IPAS adoption, SCC:
! will report improvement in cross department
collaboration for the benefit of student success andcompletion
! will be able to offer students a holistic approach toadvising and intervention strategies with the goal of
student retention
!
will be able to compare course availability toprojected student needs via MAPs
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 38/46
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 39/46
Stanly Community College
! Assessment Strategy
! Long Range and Short Range Measures
! Few Examples:
!
Indirect measures – student surveys (those whoreceived early alerts or received a MAP)
! Usage reports within SSP such as # ofcompleted MAPs or # of early alerts generated
! Institutional data such as Student Persistence,
Success Rates, and Course Completion Rates
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 40/46
Stanly Community College
! Assessment Strategy - Example
! Holistic approach to advising and interventionsmeasured by tracking early alert activity and
student persistence
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 41/46
Stanly Community College
! Assessment Strategy - Example
! Student Persistence rates, are so far,
remaining steady when compared back to
baseline
Spring 2013 to Fall 2013: 52.8% (Baseline)
Spring 2014 to Fall 2014: 52.9%
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 42/46
Stanly Community College
! Assessment Strategy – Example
! Overall goal of improving student success andcourse completion measured by:
! Student Success Rates
Fall 2013 (Baseline) = 75%
Fall 2014 (Post-SSP) = 78%
! Student Course Completion Rates
Fall 2013 (Baseline) = 83%
Fall 2014 (Post-SSP) = 86%
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 43/46
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 44/46
Questions?
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 45/46
D. Christopher BrooksComing: May 2015 to
http://tinyurl.com/k95hr7t
!"#$ &'()*(+,- (-.
#//0//10-2 3*4.0
8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)
http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 46/46
Thank you!
Lois Smith
D. Christopher Brooks
@dcbphd
Laura Damm