Dale R. Tampke – Dean, Undergraduate Studies, University of North Texasdale.tampke@unt.edu

Developing and Implementing a Web-Based Early Alert System

Where we’re headed today…

Our Context - UNT Early Alert as a Concept Project Scope (the tech-y part) Building Advocacy Functionality

End-user Responder

Data from 2011-12 (and what we’ve learned so far)

System improvements

University of North Texas - UNT Main campus –

Denton, TX Enrollment

35,754 total headcount

23,756 undergraduates

Moderately selective SAT 1105 ACT 23.4

11 Colleges/Schools Degrees

97 Bachelor’s 101 Master’s 48 Doctoral

Faculty 988 FT 519 PT

Median Class Size - 28

A bit more about UNT

Gender Female (56.0%)

Ethnicity White (62.2%) African American (13.2) Latino (12.8) Asian (5.5) Native American (0.7) Non-resident Alien (4.7)

Over 80% from <100 mi

25% Pell eligible 49% first-generation Students admitted

into colleges and schools

Mandatory two-day summer orientation

FTIC retention rate – 75.6% (2011 cohort)

Six-year graduation rate – 49.4% (2005 cohort)

Please respond to the following:

Describe your institution:A. Public or PrivateB. Two-year or Four-yearC. Small (999 and below), Medium

(1,000 – 4,999), Large (5,000 – 24,999), Mega (25,000 and up)

D. Residential or commuterE. Urban or rural

The Early Alert concept

Grounded in literature on undergraduate retention Student behavior can predict attrition Early intervention can change outcomes

First efforts were course-centered Poor performance Excessive absences

(Think “mid-term” grades)

Early Alert progresses…

Expansion to campus-wide availability Include psycho-social concerns Web front end E-mail back end Authentication varies Integration varies

A common issue: How many faculty use the system?

Our idea:

Integrate with student information system

We could build it ourselves Start with a focus on faculty (make it

easy for them) Designate a central receiver of the data Expand beyond “academic” issues Have a ready referral Begin a personal, caring conversation

Here’s a question:

What stakeholders would you need to include to implement an

Early Alert system on your campus?

Building Advocacy

Include stakeholders Students (8 from office staffs) Faculty (12 from Arts and Sciences) Academic Advisors (10 from all colleges) Student Services (15 areas) IT

Get feedback at the conceptual stage Be ready to adopt a good idea Create a faculty test group

Things to ask (examples)

Issues that affect student performance User access to the system Information a user would need to know

about a student How and whether to inform the student of

the alert Security and permissions Real time or batch processing Reporting (programmed, ad hoc, or both?)

Aspects of the system

Secure – authentication required Campus wide access Easy for faculty to use

Menu-driven Minimal information about the student needed

Ability to inform referred student via e-mail Timely Real-time ad hoc query capability Nightly reporting Completed in six weeks by one programmer

A question…

What student issues would be included in a drop-down menu on

an Early Alert system at your campus?

Reasons for Referral(what’s on the drop down menu)

Poor class attendance Poor performance on

quizzes/exams Poor performance on

writing assignments Does not participate in

class Difficulty completing

assignments Difficulty with reading Difficulty with math Sudden decline in

academic performance Concerns about their major

College adjustment issues

Financial problems Physical health

concerns Mental health concerns Alcohol or substance

use concerns Roommate difficulty Disruptive behavior Absent from work Student needs veterans

assistance Other concerns (text

box)

How Early Alert works

EARS 1.0 (early alert referral system) is available from the on-line class roll

Instructors of record receive an e-mail reminding them of EARS at the beginning of the term

Accessed through the faculty portal (The “Faculty Center”)

Nightly report delivered to a central office (Student Academic Readiness Team – START)

Follow up within one day of receiving

Other features

Relationship to student Professor, instructor Teaching assistant, teaching fellow Academic Advisor Mentor Department administrator Campus Employer Club, organization advisor

“I have had a conversation with the student”

Send a copy of the referral to the student (via e-mail)

Another question…

How would access to alert records be determined on your campus?

Consider academic advisors, student services staff, faculty,

clerical staff, others?

Accessing Early Alert

From the Faculty Center in the Student Information System

To the class roster…

From the class roster…

To the Early Alert form…

After the referral is made…

Review report every morning Real-time e-mail prompt to sender Morning report

Includes following information Demographics Student ID Faculty member’s name Course Reason(s) for referral

Follow-up – Routing alerts

First responders – Routine referrals Residence hall staff Course Achievement Assistants (peers)

More serious issues Academic Readiness Advisors Academic Advisors CARE team Counseling, Health Center

EARS is not designed for urgent situations

More follow-up – The student experience

Caring conversation (no scolding)

Emphasize mattering Resources Self-efficacy Focus on academic success Follow-up2 (we need to get better at this)

Descriptive data from academic year 2010-11

EARS Data from UNT

Alert frequency during the term

A28-S3 S4-10 S11-17 S18-24 S25-O1 O2-8 O9-15 O16-22 O23-29 O30-N5 N6-12 N13-19 N20-26 N27-D3 D4-10 D11-17

14

4953

49

133

59

11

101

30

7

29

4 2 2 1 2

Fall 2011: Alerts by Week (n=546)

Alert frequency during the term

J15-21 J22-28 J29-F4 F5-11 F12-18 F19-25 F26-M3 M4-10 M11-17 M18-24 M25-31 A1-7 A8-14 A15-21 A22-28 A29-M5 M6-12

1711

59

107

31

147

172

82

57

713

2433

13

1 1 1

Spring 2012: Alerts by Week (n=776)

First reasons for alerts

Attendance Issues Academic Issues Behavioral Issues Other Issues

575605

26

116

2011-12: Alerts by Reason

Demographic data

309; 23%

12; 1%

58; 4%

231; 17%

32; 2%11; 1%

669; 51%

Alerts by Ethnicity: 2011-12(n=1322)

Af-Amer Am-Ind

As-Pac Hispanic

Non-Res Other

White

Gender

Female48%

Male52%

Alerts by Gender: 2011-12(n=1322)

Annual Totals

2008-9 2009-10 2010-11 2011-12 2012 (fall only)

0

200

400

600

800

1000

1200

1400

553

882 920

1322

618

Annual Alert Totals(2008-present)

Analysis from Fall 2008 (pilot year)

Outcomes data

Outcomes

Literature suggests early intervention impacts: Student success Student persistence/progression

Fall GPA Spring re-enrollment Use a within-group comparison No useful “control” group

Findings

Success and Persistence

Fall GPA – 1.39 Cumulative GPA –

1.94 Persistence –

70.2%

Course Grade Distribution

A’s – 3.4% B’s – 5.9% C’s – 11.9% D’s – 12.3% F’s – 43.0% I’s – 1.3% Drops – 21.7%

Contact types (frequencies)

Faculty E-mail notice only – 42.0% Personal – 8.2% Both – 3.5% None – 46.3%

Academic Readiness E-mail notice only – 65.9% Personal (phone, response from student, meeting) – 34.1%

Outcomes by contact type

Fall GPAPersistence (% re-enrolling)

Faculty

E-mail only 1.19 62.6

Personal 2.17 85.7

Both 2.07 77.8

None 1.39 73.7

START

E-mail only 1.26 67.9

Personal 1.64 74.7

Some statistics

Personal Contact Mean Term GPA Significance

Faculty

Yes (n=25) 2.15

No (n=213) 1.30 F=11.894, p<.001

START

Yes (n=60) 1.63

No (n=158) 1.26 F= 5.436, p<.021

Outcomes by Contact Type by Reason

(Attendance)

Attendance (n=144)

Fall GPAPersistence (% re-enrolling)

Faculty

E-mail only 0.83 53.1

Personal 1.96 100.0

Both 1.77 80.0

None 1.34 71.2

START

E-mail only 1.06 62.3

Personal 1.48 73.7

Outcomes by Contact Type by Reason

(Performance)

Performance (n=74)

Fall GPA

Persistence (% re-enrolling)

Faculty

E-mail only 1.90 83.3

Personal 1.88 100.0

Both 2.48 100.0

None 1.52 80.0

START

E-mail only 1.58 82.4

Personal 1.88 85.0

EARS 2.0

System Improvements

Making the system better – EARS 2.0

Available to all staff via web portal Immediate e-mail communication

To referrers To service providers To students

Real-time referral based on alert type Improved outcome tracking using

workflow Batch uploads (at-risk students)

From the staff portal…

New responder screen…

Responder notes…

Responders can add an infinite number of “Alert Notes” to track conversations / referrals they have made for each student. Each note will be time / date stamped and include Advisors’ EUID and name.

Assessment data…

Advisor / Responder contacts student

Advisor / Responder creates notes / adds additional notes.

Advisor / Responder “completes” Alert only if student completes prescribed intervention.

COTS Early Alert Offerings

SunGard Course Signals (Purdue) - http://www.sungardhe.com/signals/

Hobson’s Early Alert system - http://www.hobsons.com/products/earlyAlert.php

Starfish Early Alert - http://www.starfishsolutions.com/sf/solutions/earlyalert.html

Datatel Retention Alert - http://www.datatel.com/products/products_a-z/student-retention-software.cfm

EducationDynamics Early Alert - http://www.educationdynamics.com/Retain-Students/Early-Alert-Systems.aspx

EBI MAPWorks - http://www.map-works.com/

Sinclair Community College -http://www.sinclair.edu/support/success/ea/

What we’ve learned

1. Including faculty in the design was critical2. Linking to class roll, self-populating made it easier for

faculty to use3. Faculty generally focus on course-related issues4. Personal faculty contact is the most effective follow-up5. E-mail contact by itself is not effective6. Some positive effect on success and persistence based

on type of contact7. Timing of alert has no apparent effect on success or

persistence8. Tracking confirmed contacts needs improvement9. EARS is not a “large class” solution

Resources

Bowen, E., Price, T., Lloyd, S., & Thomas, S. (2005). Improving the quantity and quality of attendance data to enhance student retention. Journal of Further and Higher Education, Vol. 29 (4), 375-385.

Eimers, M. (2000). Assessing the impact of the early alert program. AIR 2000 Annual Forum Paper. (ERIC Document Reproduction Service No. ED446511) Retrieved February 28, 2009, from ERIC database.

Fischman, J. (2007, October 29). Purdue uses data to identify and help struggling students. Chronicle of Higher Education Online, Retrieved May 15, 2009 from http://chronicle.com/daily/2007/10/530n.htm.

Geltner, P., & Santa Monica Coll., CA. (2001). The characteristics of early alert students, Fall 2000. (ERIC Document Reproduction Service No. ED463013) Retrieved February 28, 2009, from ERIC database.

Hudson, W. (2006). Can an early alert excessive absenteeism warning system be Effective in retaining freshman students? Journal of College Student Retention, Vol. 7(3-4), 217- 226.

More references

Kelly, J. & Anandam, K. (1979). Computer enhanced academic alert and advisement system. (ERIC Document Reproduction Service No. ED216722) Retrieved February 23, 2009, from ERIC database.

Richie, S. & Hargrove, D. (2005). An analysis of the effectiveness of telephone intervention in reducing absences and improving grades of college freshmen. Journal of College Student Retention, Vol. 6(4), 395-412.

Tampke, D. (2013). “Developing, implementing, and assessing an early alert system,” Journal of College Student Retention, 15 (1), in press.

The Hanover Research Council. (May 2008). Intrusive advising and large class intervention strategies: A review of practices. Washington, DC: Author.

Wasley, P. (2007, February 9). A secret support network. Chronicle of Higher Education, 53(23), A27.

Thank you for your participation!

Dale R. Tampke

Dean, Undergraduate Studies

University of North Texas

dale.tampke@unt.edu