Analytics in Higher Ed
Learning Analytics Learner Analytics Academic Analytics Enrollment Analytics Marketing Analytics
Data and Analytics
Raw Data Descriptive Statistics (Descriptive Analytics) Predictive Modeling (Predictive Analytics) Explanatory Modeling (Prescriptive Analytics)
Raw Data File
Source: Basketball Reference, http://www.basketball-reference.com/leagues/NBA_2013_advanced.html
Raw Data Questions for Higher Ed
• How is data stored at your institution?
• Is academic data mapped to financial data? Student involvement data?
• This data indicates a sandy beach on a large body of water, located in a warm climate. • There appear to be adequate picnic facilities for 8 groups. • Given the low angle of the sun, it is likely the data was collected in the late evening or early morning.
Descriptive Analytics
Descriptive Questions for Higher Education
• How have second year retention rates changed over the past five years?
• How many of our current students are adult learners?
• What was last year’s retention rate for adult minority students?
• What percent of fully online students are part-time?
Notes on Example Data
• Contains a random sample of 15,000 students from a four year university who first enrolled between 2009 and 2014
– 10,000 past students to explore past data and build a model, and 5,000 ‘current’ students to build predictions for • Retention rates and predictions are based on retention to a second year
• Credits attempted and part-time/full time status are taken from a student’s first fall or spring term
• Fully Online: Students who took 100% of their course attempts in an online setting
• Fully Onground: Students who took 100% of their course attempts on-campus
• All Others: Students who took both online and onground courses, or courses in other delivery modes such as hybrid courses
• 80% chance that sunset occurred within an hour of this data collection point. • 85% chance that none of the grills are in use. • 90% chance of sunburn if the data collector is Scott.
Predictive Analytics
Prediction Questions for Higher Education
• How likely is a student to retain to the next year?
• How likely is a student to pass College Algebra?
• How likely is a student to graduate in four years?
Predicted Retention by Delivery Mode
• Target students whose probability of retention is between .40 and .60 for new intervention
• Note that many of
these students will be online learners
Predictive Analytics Recap
• We used predictive analytics to score students by their probability of second year retention
• Used the probabilities to target students for new interventions
• Identified that many of the targeted students are online learners
• Will likely want to tailor at least one intervention to meet the specific needs of online learners
• Retention probabilities and individual student characteristics can be great information for advisors
Jumping to Causation
Take action by: • Nudging online learners to take courses in person?
• Eliminating online degree offerings? We have identified online students are less likely to be retained. We have not identified a causal mechanism that explains this
• When you visit this location, wear casual attire appropriate for the beach in a warm climate. • Bring your own picnic supplies and food if you plan to eat. • Scott should apply sunscreen liberally.
Prescriptive Analytics
Prescriptive Questions for Higher Education
• How effective is the current tutoring program for entry math courses?
• What students benefit the most from student success courses? • How does online course taking impact a student’s chances of retention once other factors are
accounted for?
Predicted Retention by Credits Attempted
12 or more credits attempted
Less than 12 credits attempted
Predicted Retention for Part-Time Students Only
A majority of part time students are fully online The distribution of retention probabilities is similar for fully onground and fully online students after isolating part-time
Predicted Retention for Full-Time Students Only
Very few fully online students are full time, making it difficult to compare the distributions
Research Question
Are online learners at risk of stopping out due to online course taking itself, or due to other factors that differentiate online learners from on-campus learners (i.e. Total Credit Attempts, Student Age, Financial Need, etc.)?
Research Findings
Variables Controlled For: • GED • Gender • Pell Grant • Race • Student Age • Transfer • Undeclared Major • Developmental Education
Course Attempts • Prior Credits • Course Credits Attempted
Research Findings
Fully Online students had 1.24 times greater odds of stopping out than Fully Onground students. This is a small effect.
Effect of Delivery Mode Estimate 95% Confidence Limits
Statistically Different from 1.0?
Effect Size
All Others vs Fully Ongound
1.12 (.98 , 1.3) No None
Fully Online vs Fully Onground
1.24 (1.07, 1.42) Yes Small
Interpretation and Action
Final Take-Aways
• Fully online students have lower probabilities of second year retention according to the model
• Differences in predicted retention between fully online and fully onground students are mostly explained by the amount of credits students are attempting and demographic factors
Interpretation and Action
Action Items • Meet with fully online students to better understand their unique needs
• Work to make degree plans more flexible and relevant to the needs of fully online learners
• Look for ways to make fully online students feel more connected to the university