45
Better-Informed Decision Making 28 August 2012 Kent Chen Market Development Director
Better-Informed Decision Making
28 August 2012
Kent Chen Market Development Director
2
3
“The Third Wave”- Malcomm Brown, Director of EDUCAUSE Learning Initiative
4
“The Third Wave”- Malcom Brown, Director of EDUCAUSE Learning Initiative
20052000 2015
Learning Management System
Web 2.0
2010
5
Why Analytics?
How effective are our online tools at producing better student outcomes?
6
Why Analytics?
How effective are our online tools at producing better student outcomes?
Who are our at‐risk students and how can we help them become successful?
7
Why Analytics?
How effective are our online tools at producing better student outcomes?
Who are our at‐risk students and how can we help them become successful?
Who are our most successful educators, and why are they successful?
8
Information Needs Vary
DATA TO HELP ME
On DemandEasy to Access
and Easy to Digest
But There Are Common Themes
9
Getting Information Today
10
Academic Leadership
Top 50 Courses Course Design & Trends
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and tTrends
Assessment Coordinators
Course, program, and gen ed-level goal & rubric-based reporting
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Students By Course: • Student performance against goals
Learning Analytics – Metrics/Reporting
Course Reporting
In‐House Custom Reporting
11
Course Reporting
One course, one semester viewInstructors are primary users
Academic Leadership
Top 50 Courses Course Design & Trends
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and Trends
Assessment Coordinators
Instructors
By Course: • Student performance against goals• Rubric-based reporting• User activity per dayEarly warning system
Students By Course: • Student performance against goals
Learning Analytics – Metrics/Reporting
Course Reporting
In‐House Custom Reporting
13
Academic Leadership
Top 50 Courses Course Design & Trends
At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:
instructor, department, subject, term
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and Trends LMS usage and trends Capacity planning Course open after term end
Assessment Coordinators
Aggregate, student, cross-course, cross-term goal performance (LMS scores)
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Course dashboard - comparison of instructor’s course design to other courses
Correlation of grades to activity
Students By Course: • Student performance against goals
Student dashboard - performance across courses and compared to peers
Learning Analytics – Metrics/Reporting
For LearnCourse Reporting
In‐House Custom Reporting
Information Needs Vary
DATA TO HELP ME
On DemandEasy to Access
and Easy to Digest
But There Are Common Themes
CustomerSuccess Story
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
Impact to Lingnan University
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
Impact to Lingnan University
Thank you!
APPENDIX
George SiemensAssociate Director
Technology Enhanced Knowledge Research Institute
Thank you!
24
Best Practices
Defined Metrics
Business Rules
Derived Information
Transformation Layer
Data Warehouse
OLAP
Star Schema
Pre-Built Reports &
Dashboards
Blackboard AnalyticsA Packaged Analytics Application
Pre-Built Data Integration
SIS
For Leadership and Staff
Flexible Reporting
For Instructors
For Students
29
APPENDIX
30
31
32
33
34
35
36
37
38
Baseline Implementation
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
Rapid Deployment
Homegrown Data Warehouses
Your Data is Loaded
5 Step Approachmonths
Design and Development Implementation
ERP Vendor Toolkits at $250 per hour
committed resources
40
Metrics & Dimensions for Analysis
•Gradebook•Possible Score•Score•Weight•Grade Pct•Weighted•Element Count•Highest Score•Lowest Score•Improvement•Average Score•Attempts•Days To Grade
•Goals•TBD
•Course Design•Item Count•Files Attached•VTBE Objects•VTBE Characters•File Size•Folder Depth•Links•Folders
•Course Section•Subject•Department•College•Class Number•Term•Course Career/Level•Instruction Method
•Students•Birthdate•Gender•Ethnicity•Citizenship•Current Term•Primary Major•Career/Level•Academic Level•Current Academic Standing•Primary College
•Instructors•Gender•Ethnicity•Highest Education Level•Faculty Rank•Tenure Status•Primary Department
•User Activity•Sessions
•Logins•Session Minutes•Courses•Content Items•Tools•Modules•Clicks•Pages
•Course Activity•Courses Accessed•Course Access Minutes•Session Course Sequence•Content Items•Pages
•Content Activity•Content Items Accessed•Content Item Hits•Content Access Minutes•Session Sequence•Folder (or Content) Depth
•Submissions•TBD
Learn MetricsKey SIS
Elements
41
Technical Requirements
• Requires the use of Blackboard LearnTM, Release 9.1, Service Pack 4 or later
• Pre-built SIS integration with:• Oracle (PeopleSoft)• Datatel (Colleague) • SunGard Higher Education • An Integration method is provided for other SISs
• Requires a locally hosted SQL Server environment• Minimum hardware & software requirements available on
request
42
Academic Leadership
At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:
instructor, department, subject, term
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS usage and trends Capacity planning Course open after term end
Assessment Coordinators
Course, program, and gen ed-level goal & rubric-based reporting
Goal “coverage” reporting against curriculum
Artifact repository
Aggregate, student, cross-course, cross-term goal performance (LMS scores)
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Course dashboard - comparison of instructor’s course design to other courses
Correlation of grades to activity
Students By Course: • Student performance against goals
Student dashboard - performance across courses and compared to peers
Learning Analytics – Metrics/Reporting
For LearnCourse Reporting
43
Course Reporting
One course, one semester viewInstructors are primary users
44
Cross-course rubric
Assessment Coordinators are primary users
45
Cross-department, cross-college view
Many benefit – students, instructors, executives
for Learn
