8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 1/48
THE ROAD TO
ANALYTICS:LAYING THEFOUNDATION
Educause Annual Conference9/29/2014
Linda Gilbert
Juliana Lancaster
Georgia Gwinnett College
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 2/48
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 3/48
ABOUT OUR INSTITUTION
Georgia Gwinnett CollegeBegan in 2006
Massive growth since inception
Enrollment in 2006 = 118
Enrollment in 2014 = 11,000
“Campus of Tomorrow”
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 4/48
ABOUT YOU….
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 5/48
ROAD TO ANALYTICS
Learning Outcomes
Learn about the current status of analytics in higher education andcontributing factors
Explore general issues, challenges, and potential strategies related to
implementing analytics in higher education
Identify next steps to apply information from this session to yourinstitutional context
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 6/48
SESSION OVERVIEW
Initial Presentation: Analytics Overview
Activity: Resources and background
Activity: Identify productive working areas
Presentation: GGC Case StudyFinal Activity: Next Steps (using working areas)
Wrap-Up
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 7/48
ANALYTICS:
THE HOPE AND THE HYPE
Linda Gilbert
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 8/48
THE TECHNOLOGY HYPE CYCLE
Technology Trigger
Peak of Inflated Expectations
Trough of Disillusionment
Slope of Enlightenment
Plateau of Productivity
Visibility/Expectations
TimeSource: Gartner Researchhttp://www.gartner.com/technology/research/methodologies/hype-cycle.jsp
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 9/48
VISIBILITY/EXPECTATIONS
Horizon Report 2014 (One year out)
Horizon Report 2013 and 2013 (Two to three years out)
“Positioned in the same two-to-three year adoption
timeframe as it was last year, learning analyticscontinues to be an emerging field, one that is growing
quickly, but is still just out of reach for most
educators.”
Horizon Report 2013
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 10/48
VISIBILITY/EXPECTATIONS
Horizon Report 2014 (One year out)
Horizon Report 2013 and 2012 (Two to three years out)
MOOC – Learning Analytics and Knowledge (LAK) 2013
SoLAR – Society for Learning Analytics Research
Educause – Featured Topic in Library
http://www.educause.edu/library/analytics
Educause Review Special Issue July/Aug 2012
Educause Analytic Sprint, July 2012
ECAR Study of Analytics in HE, August 2012
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 11/48
TROUGH OF DISAPPOINTMENT
Google Flu Study
Purdue Course Signals
Next Gen Learning Analytics Reports
Chronicle of Higher Education ArticleRecent Big Data Struggles are ‘Birthing Pangs,’ Researchers Say (March 2014)
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 12/48
THE SLOPE OF ENLIGHTENMENT
Sharing lessons learned
Building on effective strategies
Identifying common issues and challenges
Developing useful modelsCustomizing to context
Building expertise!
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 13/48
INFLUENCES AND DEFINITIONS
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 14/48
(Learning) Analytics
BusinessIntelligence
Big Data
EducationalData Mining
StatisticalmethodsIntelligent
Tutors
Personalization
Adaptivelearning
INFLUENCES
Source: http://www.slideshare.net/gsiemens/iadis-shanghai
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 15/48
INFLUENCES
“Big Data”
Defined by the 3V’s - Volume, Variety, Velocity
Getting to “the unknowns”
How to leverage the benefits of “big data” while“avoiding paralysis by analysis”.
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 16/48
ANALYTICS IN HIGHER ED
Academic Analytics
Learning Analytics
Predictive Analyticshttp://net.educause.edu/ir/library/pdf/ELI3026.pdf
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 17/48
ACADEMIC ANALYTICS
“Academic analytics, although a relatively new concept,refers to the analysis of data to help educational institutions monitor
progress on key institutional goals, such as student retention,faculty productivity, and the impact of outreach and engagement.”
http://www.educause.edu/library/academic-analytics
“A process for providing higher education institutions with the data
necessary to support operational and financial decisionmaking.”
http://net.educause.edu/ir/library/pdf/ELI3026.pdf
Focus: Institutional, regional/national/international
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 18/48
LEARNING ANALYTICS
Learning analytics is the measurement, collection, analysis, and reporting of data
about learners and their contexts, for purposes of understanding
and optimizing learning and the environments in which it occurs.
Society of Learning Analytics Research, cited by
“Field associated with deciphering trends and patterns from educational big data, or
huge sets of student-related data, to further the advancement of apersonalized, supportive system of higher education
Horizon Report 2013
“The use of analytic techniques to help target instructional, curricular, and support
resources to support the achievement of specific learning goals.” http://net.educause.edu/ir/library/pdf/ELI3026.pdf
Focus: Learners, Faculty, (Departments)
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 19/48
PREDICTIVE ANALYTICS
“An area of statistical analysis that deals with extractinginformation using various technologies to uncover relationships and
patterns within large volumes of data that can be used to predictbehavior and events.
http://net.educause.edu/ir/library/pdf/ELI3026.pdf
“Educational Data Mining is an emerging discipline, concerned withdeveloping methods for exploring the unique types of data that comefrom educational settings…, it often has multiple levels of meaningfulhierarchy, which often need to be determined by properties in the dataitself , rather than in advance.”
LAK2013 Course Materials Focus: All levels
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 20/48
MODELFocus
(Unit of Analysis)
Learner (Learning Analytics)
Institution (Academic Analytics)
Forecasting (Predictive Analytics)
Type of Analysis(Level of Inference)
Reporting (Descriptive Analytics)
Gilbert 2013
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 21/48
MODELS
Focus Type of analytics
1 What happened? Standard reports in real time
2 How many, how often, where? Ad hoc reports (real time)
3 Where exactly is the problem? Query/drill down (real time)
4 What actions are needed? Alerts (real time)
5 Why is this happening? Statistical analysis
6 What if these trends continue? Forecasting/extrapolation
7 What will happen next? Predictive modeling
8 What’s the best thing that could happen? Optimization
Adapted from the Davenport/Harris FrameworkSource: http://www.educause.edu/library/resources/building-organizational-capacity-analytics
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 22/48
APPLICATIONS IN HE
1. Extracting and analyzing data from learning management systems
2. Building an analytics matrix that incorporates data from multiple sources (socialmedia, LMS, student information systems, etc).
3. Profile or model development of individual learners (across the analytics matrix)
4. Predictive analytics: determining at-risk learner
5. Automated intervention and adaptive analytics: i.e. the learner model should beupdated rapidly to reflect near real-time learner success and activity so thatdecisions are not made on outdated model
6. Development of "intelligent curriculum" where learning content is semanticallydefined
7. Personalization and adaptation of learning based on intelligent curriculum wherecontent, activities, and social connections can be presented to each learner based
on her profile or existing knowledge8. Advanced assessment: comparing learner profile with architecture of knowledge in
a domain for grading or assessment Source: LAK2013 (MOOC)
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 23/48
APPLICATIONS IN HE
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 24/48
INTRIGUING DIRECTIONS
Trend analysis and prediction Early warning/intervention
Enrollment/Admission decisions
Personalization and adaptive learning
Online class content Mentoring support
Modeling: Learners, knowledge domains, relationships
Development of policies, norms, and technology OLA: Open Learning Analytics Architecture (SoLAR)
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 25/48
CONSIDERATIONS AND CONCERNS
Messy Data
Availability
Data choices
Human Biases
WYSIATI Limits to Prediction
“Black Swans”
Autonomous Agents
Feedback Loops – Self-fulfilling prophecies
“The problem starts when smart people innice suits and lab jackets proclaim that “the
data says…” In truth, the data never saysanything.
We interpret it in one way or another and
there are lots of ways to interpret itincorrectly.” - Greg Satell
http://www.digitaltonto.com/2011/the-pitfalls-of-prediction/
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 26/48
CONSIDERATIONS AND CONCERNS
Data Tools
Many current systems are stand alone, single functionality
Visualization Tools Needed
Data Use/Culture
Good tools =/= Good models =/= Good decisions
Privacy and Ethics:
Just because we can… should we?
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 27/48
MOVING INTO ANALYTICS
10/26/2012
Source: http://www.educause.edu/library/resources/building-organizational-capacity-analytics
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 28/48
Governance
Policies
Culture
MODELS
Tools andTechnology
Culture
Expertise
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 29/48
ACTIVITY:RESOURCES AND BACKGROUND
Form groups of THREE at each table.
Individually read pages 29(bottom)-33 fromBuilding Organizational Capacity for Analytics
Five minutes – we’ll time for you
As you read, identify passages (main one and a couple ofback-ups) that you feel have important implications for yourwork. Be prepared to share it in a few minutes.
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 30/48
ACTIVITY:RESOURCES AND BACKGROUND
“Three Levels of Text”
Rounds (5 minutes total per person) LEVEL 1: Read aloud the passage you selected
LEVEL 2: Say what you think about the passage (interpretation, connectionto past experiences, etc.)
LEVEL 3: Say what you see as the implications for your work.
The group responds to what has been said.
Credit: schoolreforminitiative.org
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 31/48
ACTIVITY:IDENTIFY WORKING AREAS
On your own, take five minutes to review your responses to thematurity index that you did as you came in.
Feel free to change responses based on any shift in your thinking duringthe session.
Identify the category on the index that you see as yourinstitution’s strongest area. Provide a short ‘why’ statement onthe green post it.
Identify the category on the index that you see as yourinstitution’s weakest area. Provide a short ‘why’ statement onthe yellow post it.
Post your notes on the appropriate flipchart. Take a break!
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 32/48
BREAK TIME
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 33/48
ACTIVITY: REVIEW
Review the flip charts and look for interestingpatterns.
Move to the table flagged for the area you want tofocus on for the next activity Ok to sit with colleagues now
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 34/48
GGC: DESIRE, CHALLENGE, AND NEED
Juliana Lancaster
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 35/48
THE OVERALL GOAL
Broad and deep use of data for improvement and decisionmaking at all levels
Straightforward access to data needed for specific purposes
Strategic use of analytics for large-scale and long-termplanning
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 36/48
THEN AND NOW
2006-07
Completely new institution
118 students
3 academic majors
11 faculty/ 30 staff No policies or processes
Technology infrastructureinherited from USG multi-institution Center
2014-15
10,900 students
14 academic majors
500 faculty/ 500staff
Policies and processes in place fordata security, management, andsharing
Technology infrastructuresupports ??????
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 37/48
BIGGEST CHALLENGES
Rapid Growth
10 faculty over 500 (FT and PT)
118 students over 10000 students
30 administrators/staffapprox. 500 (FT and PT)
Budget Limitations “Great Recession” in 2008
Reduced state funding overall
Lag in funding relative to growth
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 38/48
BIGGEST CHALLENGES
“Building the plane while it is in the air”
“Doing the urgent instead of the important”
Establishing processes for the right now while trying to envision what will
be needed later Ad hoc and idiosyncratic uses of various resources by different people
Little time for careful planning
Duplication of requests and effort rampant
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 39/48
THE ROAD WE TRAVELED: GETTINGSTARTED
Identifying the need… and “whose problem is it?”
Managing the massive growth and complexity.
Getting permission from TPTB
Getting the right people in the roomFiguring out where to start!
Locating resources USG
EDUCAUSE
Policy Dimensions of Analytics inHigher Education. Educause
Review July 2012
Data Governance,Data Classification,Roles and
Responsibilities,Data Policies,Legal Requirements
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 40/48
THE ROAD WE TRAVELED:WHAT WE DID AND WHAT WE LEARNED
Data/Reporting/Tools
Home-grown internal system first
Used experience to identify longer-term needs
Then investigated larger solutions
Governance/Infrastructure
Secured commitment of senior administration first
Focused early on policy needs
Managed up
Investment
Extensive discussion (aka time)
Communication!
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 41/48
THE ROAD WE TRAVELED:WHAT WE DID AND WHAT WE LEARNED
Expertise
Gathered the right people to tackle the needs Culture
Changed behavior by changing policy
Insisted on use of new process
Culture
Built on existing commitment to use of data
Kept discussions going until everyone agreed
Changed behavior by changing policy
Insisted on use of new process
Process
Established and regularized a common process Built relationships and communication through regular meeting
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 42/48
THE EARLY WIN…DATA REQUEST PROCESS
Building culture of data awareness
Need to ask
Need to secure/protect
Building expectation of accessibility
Easy to make a request, spawns automatic email notifications
Manage approval and fulfillment process within on-line form
Data provider is easier to identify
Building repository of information
Data requests capture work done
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 43/48
WHERE ARE WE NOW?
MaturityIndex
Data/Reporting/
Tools
3
Governance/Infrastructure
4
Investment
2Expertise
2
Culture
4
Process
4
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 44/48
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 45/48
ACTIVITY: YOUR NEXT STEPS
Take a little time to review resources related to your issuearea.
Together with the group, brainstorm “what can be done” inyour area of focus. Be ready to share with other groups.
List three “next steps” YOU can take.
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 46/48
CLOSING QUESTIONS,
COMMENTS, AND DISCUSSION
10/26/2012
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 47/48
RESOURCES AND CREDITS
Materials will be posted under the session topic at Educause
Presenters:
Linda Gilbert ([email protected])
Juliana Lancaster ( [email protected])
Credits: We would like to acknowledge the other charter members of the Data Managersand Stewards Working Group at GGC:
Nancy Grattan
Lily Hwang
Laura Ledford
Kris Nagel
10/26/2012
8/11/2019 Seminar 08P - The Road to Analytics: Laying the Foundation (separate registration is required) (242333352)
http://slidepdf.com/reader/full/seminar-08p-the-road-to-analytics-laying-the-foundation-separate-registration 48/48