Improving Student Success with Analytics (242532298)

8/11/2019 Improving Student Success with Analytics (242532298)

http://slidepdf.com/reader/full/improving-student-success-with-analytics-242532298 1/20

Improving Student Success with Analytics: A report on SUNY Oswego's Starfish Early Alert pilot

Rameen Mohammadi Associate Provost for Undergraduate and Special ProgramsGreg Ketcham Interim Director of Academic Programs,

Division of Extended Learning

2014 EDUCAUSE Annual Conference



Acknowledgement

Oswego Starfish Pilot Implementation Team:Michelle Bandla, Liz Schmitt, Sue Fisk, Jerret LeMay, SaraBlaney, Sue Fisk, Corie Kohlbach, Lori Reed, Mike Pisa, DanRoberts, Cathy Santos, Starr Wheeler.

Slide 2



Agenda

• Choosing Starfish

• Planning, implementation, and deployment overview

• Pilot populations and selection criteria

• Results (to date) achieved

• Next steps

Slide 3



Choosing Starfish

• Nationally SUNY comprehensives are in the 75-80 th percentile of 4-year publics in persistencemeasures

• SUNY Oswego’s retention and persistence rateswere below institution goals

Oswego’s retention and graduation rates are lowerthan most SUNY comprehensives

Slide 4



Choosing Starfish: Early alert sysem

• Metacognition• Dunning & Kruger (1999), Dunning et al (2004)• Early targeted communications offer more effective

feedback to struggling students• Earlier intervention gives us more options• Communication to build connection

• Student satisfaction with their classroom experience(Pascarella 2011)

• Connection to resources• Connection to instructors



Choosing Starfish: Selecting a vendor

• Oswego had been evaluating IPAS vendors forover several years

• Constantly-changing products• Major financial commitment• SUNY adoption?

• The (unsuccessful ) Gates grant opportunitypushed us to focus on a vendor and moveforward with a pilot.

IPAS-Integrated Planning and Advisement Services

Slide 6



Choosing Starfish

•

• Integrates with ANGEL, Blackboard, Banner, Google Calendar,

Tutortrac, Noel Levitz CSI• Easy to use and train progress Surveys provide instructors the

opportunity to raise concerns or offer kudos• Notification to students of concerns via email that include

comments provided by the instructor• Notification to advisors and other professionals to offer

outreach flagged students

Slide 7



Plan, implement, and deploy

• Executed a 10-week initial rollout Plan in 8 weeks• Banner-starfish (SIS Adaptor)• ANGEL-starfish (LMS Adaptor)• Templates for email notifications, setting up roles and

privileges• Google calendar, Tutortrac, and Noel Levitz CSI

Slide 8



Example Survey Announcement

To: {Instructor Name}From: [email protected]: [Starfish] Progress Survey for {Deployment Date}

Dear {Instructor Name},This spring is the second semester of our year-long Starfish Early Alert Pilot. You are receiving this email because you have oneor more of the pilot students in your course(s). You have a critical role in this process as you best know how the student is doingin your course. Starfish provides us with the communication tools to identify the struggling students and intervene earlier in orderto help them succeed.

Approximately 1000 first year and transfer students are in the PILOT cohort from the following majors: Biology/Zoology, PublicJustice, Psychology and those in the School of Business. With over 3,600 performance flags and kudos generated in the fall,

faculty participation exceeded expectations. We are hoping that you will continue to help us through this pilot and utilize Starfishto report on the academic progress of our students.

There will be two Progress Surveys deployed this semester through Starfish prior to mid-term in order to receive earlyperformance feedback about our PILOT students. Starfish system offers many other tools to aid in assisting our students thatsome faculty have already found useful. For example, you have the ability to identify “To Do” items for students, use the Starf ishappointment features that synchronize with Google Calendar, or take advantage of the “Zoom In” option that uses your Angelgradebook to raise flags or generate kudos for students. To learn more about the functionality of Starfish, please follow this link:Oswego.edu/starfish

Here is a short YouTube video explaining how to submit a progress survey:http://www.youtube.com/watch?v=NMMPaZxDMP8&feature=youtu.be Thank you for your participation! We appreciate your dedication to student success.

On behalf of the Starfish Implementation Team,

Rameen Mohammadi Associate Provost

http://oswego.edu/starfish

http://www.youtube.com/watch?v=NMMPaZxDMP8&feature=youtu.be

http://www.youtube.com/watch?v=NMMPaZxDMP8&feature=youtu.be

http://oswego.edu/starfish



Example Student Email Notification

From : [email protected] : [email protected] : [Starfish College Early Alert] Progress Concern: CourseName

Dear StudentName,You have been flagged for FlagName in CourseName.

The professor in this class has indicated that you are not making adequate progress and she/he is concerned about your ability tosuccessfully complete the class.[if comments] Below are your instructor’s comments: These are my co mm ents!

Please contact your instructor to discuss these issues and to make a plan to improve your performance in the course. Refer to the

course syllabus for contact information for your professor.

We also recommend that you contact your advisor to discuss this issue and any other issues you are experiencing this semester.

The College has free tutoring services to help you improve your academic performance. In addition, advisement and personal counselingservices are available to support academic and career planning, as well as to assist you with personal concerns.To seek tutoring, please contact the Office of Learning Services:

Address: 173 Campus CenterEmail: [email protected]: http://www.oswego.edu/academics/support/OLS.html

To seek Major or Career Exploration contact The Compass Address: 145 Campus CenterEmail: [email protected]: http://www.oswego.edu/student/success/compass.html

To seek personal counseling, please contact the Counseling Center: Address: 113 Walker Health CenterPhone: 315-312-441website: http://www.oswego.edu/student/services/counseling_services.html

Sincerely,Student Success Team

http://www.oswego.edu/academics/support/OLS.html

http://www.oswego.edu/student/success/compass.html

http://www.oswego.edu/student/services/counseling_services.html

http://www.oswego.edu/student/services/counseling_services.html

http://www.oswego.edu/student/success/compass.html

http://www.oswego.edu/academics/support/OLS.html



Pilot populations and selection criteria

• Starfish Pilot, 2013-14• 967 students• Freshmen and Transfers (Biology, Psychology, Public

Justice, Business)• Criteria: Department interest, past success rates, good

indicator of campus experience

Slide 11



What are the results?

Fall 2013 Spring 2014*

Students 967 977Courses and labs 711 704

Instructors 360 383

Advisors 100 131

Courses /w Completed Surveys 429 (60%) 431 (61%)

Academic Flags 1440 765

Kudos 1951 768

System Flags (3 flags) 199 55

System Flags (6 flags) 76 6

Slide 12

*Significant changes in the flag structure occurred between these semesters as aresult of faculty feedback



What are the results--Tracking Items Changed Fall toSpring 2014

Fall 2013 Flags Spring 2014 Flags

Attendance ConcernExcessive Absences

In Danger of Failing In Danger of Failing

Low Quiz/Test Scores

Incomplete or Unsatisfactory Graded WorkLow Participation

Failure to complete Assignments

Behavioral ConcernGeneral/Behavioral Concern

General Concern



What are the results--Tracking Items Changed Fall toSpring 2014

Fall 2013 Kudos Spring 2014 Kudos

Keep Up the Good Work

Outstanding Academic Performance Outstanding Academic Performance

Showing Improvement Showing Improvement



What are the results? An assessment strategy

• Performance Differences between Fall Cohorts (2012and 2013)

• Course specific performance Fall 2013: Pilot vs. non-Pilot• Also Compared to Fall 2012 for pilot-eligible cohorts

• Fall to Spring Retention• Fall to Fall Retention, GPA, and Academic Standing• Student, faculty and staff user surveys (in progress)

Slide 15



What are the results--Retention, GPA, Academic Standing ()

Freshman Fall 2012 Fall2013

retention 85% 87%

end of year gpa 2.8 2.8

dismissed 7% 6%

warning 12% 8%

Transfer Fall 2012 Fall2013

retention 85% 86%

end of year gpa 2.7 2.8

dismissed 12% 10%

warning 11% 9%

Note —2012 vs. 2013 percentage differences are not statistically significantNote —data here ONLY includes students who were in the pilot majors forall of 2013-2014 compared to the same group in 2012-2013; 642 studentsfrom Fall 2012 and 780 students from Fall 2013 were included in the datashown.

Slide 16



What are the results –course comparisons Fall 2012vs. 2013

Slide 17

MAT208 N GPA

2012 47 1.88*

2013 42 2.83

*significant at 5% level

ECO101 N GPA

2012 216 2.42*

2013 287 2.65

No difference:BIO 110, BIO 203, CHE 111, CHE 209,HIS 100, HIS 202,HIS 203, MAT 104, MAT 158, MAT 210, PSY 100

Selected Courses, All Sections



What are the results--Eco 101 Fall 2013

• Principles of Microeconomics• 410 students• Mastery learning quiz system•

i<clicker sessions every class (attendance proxy)

Slide 18*significant at 5% level

N Course

Average

Missedi<clickersessions

Missedmodulequizzes

Pilot 249 71.46 3.35* 1.45*

Non-pilot 161 70.30 4.45 2.22



What are the next steps?

• Starfish Pilot 2014-2015: Pilot expansion• ~2,700 students

• All new incoming student (freshmen and transfers)• All At-Risk students: those on AW and DQ

• Upload of data from Tutortrac and Noel Levitz CSI• Implemented System Flag raised as a result of CSI

Survey results.

Slide 19



What are the next steps?

• Two new Academic Success Specialists fortracking progress and closing the loop with at riskstudents

• More training on Starfish Connect: Office Hours,Closing the Loop, etc.

• October 2014: advisors have closed the loop on 118flags (out of 805) flags

• October 2013: advisors had closed 6 out of 759 flags)• Administer a survey of students and faculty

Slide 20

Date post:	02-Jun-2018
Category:	Documents
Upload:	educause
View:	215 times
Download:	0 times

Improving Student Success with Analytics (242532298)

Documents