Southeastern Association for Community College Research
2008 Conference
Using Technology to Efficiently and Effectively Gather Information from
Recent Alumni and Employers
August 2008
August 6, 2008 Southeastern Association for Community College Research 2
SACCR 2008 Conference August 2008
Alumni and Employer Surveys
Presenters
Dr. James Coraggio, Director of Institutional Research and Effectiveness
Maggie Tymms, Assessment Coordinator for Academic Programs
August 6, 2008 Southeastern Association for Community College Research 3
SACCR 2008 Conference August 2008
SPC Background
SPC, established in 1927, is the oldest 2-year college in Florida First Community College in Florida to offer
4 year degrees (2002) 9 Campuses throughout the county 2007-08 FTE: 16,086 (LD), 1,338 (UD) Opening Spring 2008 headcount: 25,754 Annual 2007-08 enrollment: 62,200
August 6, 2008 Southeastern Association for Community College Research 4
SACCR 2008 Conference August 2008
Institutional Effectiveness
“Institutional Effectiveness is the integrated, systematic, explicit, and documented processes of measuring performance against the SPC mission for purposes of continuous improvement of academic programs, administrative services, and educational services offered by the college.”
Operating ResultsBudget PlanningStrategic & Unit
PlanningEvaluations/ AssessmentsMission
“Closing the Loop”
August 6, 2008 Southeastern Association for Community College Research 5
SACCR 2008 Conference August 2008
Performance Improvement
From Compliance to Performance Improvement
Data Information ImprovementKnowledge
Improved OutcomesSPCReportsAssessment Data
CommunityCollege LeadershipProgram Directors
FacultyStudents
August 6, 2008 Southeastern Association for Community College Research 6
SACCR 2008 Conference August 2008
Agenda
Student Assessment Points Purpose of Follow-up Surveys Alumni/Employer Survey Process Dissemination of Survey Results and
Reports
August 6, 2008 Southeastern Association for Community College Research 7
SACCR 2008 Conference August 2008
Student Assessment Points
Students surveyed multiple times
August 6, 2008 Southeastern Association for Community College Research 8
SACCR 2008 Conference August 2008
Purpose of Follow-up Surveys
Recent Alumni Survey was designed to: Gain insight into alumni’s perceptions of
the quality of general education and career preparation
Determine the current activities of former students such as employment information and educational status
August 6, 2008 Southeastern Association for Community College Research 9
SACCR 2008 Conference August 2008
Purpose of Follow-up Surveys
Employer Survey was designed to: Gain insight into an employer’s perceptions of
hired SPC graduates in the areas of key general education competencies and foundational skills
Provide the employee’s major job responsibilities and wage information, as well as employer’s willingness to hire another SPC graduate
Identify employer’s willingness to support various college activities
August 6, 2008 Southeastern Association for Community College Research 10
SACCR 2008 Conference August 2008
Purpose of Follow-up Surveys
State Requirements Survey data are also collected for state
required annual Placement & Follow-up Report (FETPIP: Florida Education Training Placement Information Program) and supplemental information to meet workforce criteria
August 6, 2008 Southeastern Association for Community College Research 11
SACCR 2008 Conference August 2008
Alumni/Employer Survey Process
Previous model Labor intensive Manually tabulated paper surveys Telephone interviews conducted
using overtime hours
August 6, 2008 Southeastern Association for Community College Research 12
SACCR 2008 Conference August 2008
Alumni/Employer Survey Process
Issues with previous model Costly survey process Declining college budget Increasing demand for accountability Outdated contact information Students over-surveyed Declining response rates Conducted in isolation
August 6, 2008 Southeastern Association for Community College Research 13
SACCR 2008 Conference August 2008
Alumni/Employer Survey Process
Goals of new survey processA. Improve data collection processB. Enhance quality of survey formsC. Include program specific
informationD. Increase response rates
August 6, 2008 Southeastern Association for Community College Research 14
SACCR 2008 Conference August 2008
A. Improve Data Collection Process
Data Collection Process Eliminated telephone calls Automated the process (email merge) Employ online survey submission
(Zoomerang) Obtain accurate email addresses Use multiple points of contact
August 6, 2008 Southeastern Association for Community College Research 15
SACCR 2008 Conference August 2008
A. Improve Data Collection Process
Survey Schedule Recent Alumni Survey
Graduates surveyed six months after the end of session (Three times per year)
Employer Survey Employer of graduates surveyed at the
end of academic year (Once per year)
August 6, 2008 Southeastern Association for Community College Research 16
SACCR 2008 Conference August 2008
A. Improve Data Collection Process
First email Second email Third email
First mailing Second mailing
2 weeks
2 weeks
2 weeksMultiple Points of Contact
2 weeks
August 6, 2008 Southeastern Association for Community College Research 17
SACCR 2008 Conference August 2008
B. Enhance Quality Of Survey Forms
Improved Survey Forms Better “look and feel” of survey format Developed scannable paper forms Use online branching models to reduce
survey length Design items to gather only pertinent
data
August 6, 2008 Southeastern Association for Community College Research 18
SACCR 2008 Conference August 2008
Online Survey: Email and Form
August 6, 2008 Southeastern Association for Community College Research 19
SACCR 2008 Conference August 2008
Paper Survey: Letter and Form
August 6, 2008 Southeastern Association for Community College Research 20
SACCR 2008 Conference August 2008
C. Include Program Specific Information
Program specific information Meet with PDs Discuss requirements for program
accreditation and timelines Include their necessary program specific
items
August 6, 2008 Southeastern Association for Community College Research 21
SACCR 2008 Conference August 2008
D. Increase Response Rates
Create awareness and convey importance Involve program faculty in the process Request faculty assistance to inform students Include a flyer with graduation materials Coordinate with programs that hand-carry
surveys to local employers
August 6, 2008 Southeastern Association for Community College Research 22
SACCR 2008 Conference August 2008
Response Rates
Summer 07
August 6, 2008 Southeastern Association for Community College Research 23
SACCR 2008 Conference August 2008
Response Rates
Fall 07
August 6, 2008 Southeastern Association for Community College Research 24
SACCR 2008 Conference August 2008
Response Rates
6.3%
9.1%8.3%
5.2%
11.2%
10.1%
4.2%
9.0%
5.3%
7.6%
5.6%
4.4%
6.4%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
1st E-mail 2nd E-mail 3rd E-mail 1st Mailing 2nd Mailing
Summer & Fall 06 Spring 07 Summer 07 Fall 07
August 6, 2008 Southeastern Association for Community College Research 25
SACCR 2008 Conference August 2008
Response Rates
Overall Response Rate
13.7%
21.2%23.4%
19.0%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
Summer & Fall 06 Spring 07 Summer 07 Fall 07
August 6, 2008 Southeastern Association for Community College Research 26
SACCR 2008 Conference August 2008
Response Rates - Employer
OtherResponses
Recent Alumni Surveys Sent
Employer Surveys Sent
Is a 44% Employer Response Rate Good?
COE Responses
With Permission Responses
Without Permission
August 6, 2008 Southeastern Association for Community College Research 27
SACCR 2008 Conference August 2008
Disseminating Information
College wide access to reports http://www.spcollege.edu/Central/Surveys/SurveyList.htm
Program Specific Reports Published on the Educational Outcomes
website http://it.spcollege.edu/edoutcomes
August 6, 2008 Southeastern Association for Community College Research 28
SACCR 2008 Conference August 2008
Disseminating Information
August 6, 2008 Southeastern Association for Community College Research 29
SACCR 2008 Conference August 2008
Disseminating Information
August 6, 2008 Southeastern Association for Community College Research 30
SACCR 2008 Conference August 2008
Disseminating Information
August 6, 2008 Southeastern Association for Community College Research 31
SACCR 2008 Conference August 2008
Program Specific Reports
August 6, 2008 Southeastern Association for Community College Research 32
SACCR 2008 Conference August 2008
SPC IE Website
August 6, 2008 Southeastern Association for Community College Research 33
SACCR 2008 Conference August 2008
Lessons Learned
Do not expect that students remember their ID#
Administer the survey soon after the end of the session (six months)
Use multiple points of contact (two weeks) Find the most recent contact information Make students aware of the importance of
the survey process (survey announcements)
August 6, 2008 Southeastern Association for Community College Research 34
SACCR 2008 Conference August 2008
Lessons Learned
Partner with programs to acquire up-to-date information
Provide updates to college data systems (PeopleSoft, Alumni Foundation)
Monitor results and make changes as necessary
More accurate data (quicker turn around of results) with automated processes
August 6, 2008 Southeastern Association for Community College Research 35
SACCR 2008 Conference August 2008
Future Direction
Ultimate goal is to provide Provosts (Deans) and Program Directors ‘timely’, ‘relevant’ ‘accurate’, and ‘interpretable’ information regarding their programs and their performance outcomes Multi-measure ‘Dashboard’ style reports
(Business Intelligence/Performance Measurement)
August 6, 2008 Southeastern Association for Community College Research 36
SACCR 2008 Conference August 2008
Questions/Next Steps
Southeastern Association for Community College Research
2008 Conference
Using Technology to Efficiently and Effectively Gather Information from
Recent Alumni and Employers
August 2008