Better-Informed Decision Making

28 August 2012

Kent Chen Market Development Director

2

3

“The Third Wave”- Malcomm Brown, Director of EDUCAUSE Learning Initiative

4

“The Third Wave”- Malcom Brown, Director of EDUCAUSE Learning Initiative

20052000 2015

Learning Management System

Web 2.0

2010

5

Why Analytics?  

How effective are our online tools at producing better student outcomes?

6

Why Analytics?  

How effective are our online tools at producing better student outcomes?

Who are our at‐risk students and how can we help them become successful?

7

Why Analytics?  

How effective are our online tools at producing better student outcomes?

Who are our at‐risk students and how can we help them become successful?

Who are our most successful educators, and why are they successful?

8

Information Needs Vary

DATA TO HELP ME

On DemandEasy to Access

and Easy to Digest

But There Are Common Themes

9

Getting Information Today

10

Academic Leadership

Top 50 Courses Course Design & Trends

Instructional Technology

Overall usage per date range User statistics per month Course statistics per month

LMS Usage and tTrends

Assessment Coordinators

Course, program, and gen ed-level goal & rubric-based reporting

Instructors

By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system

Students By Course: • Student performance against goals

Learning Analytics – Metrics/Reporting

Course Reporting

In‐House Custom Reporting

11

Course Reporting

One course, one semester viewInstructors are primary users

Academic Leadership

Top 50 Courses Course Design & Trends

Instructional Technology

Overall usage per date range User statistics per month Course statistics per month

LMS Usage and Trends

Assessment Coordinators

Instructors

By Course: • Student performance against goals• Rubric-based reporting• User activity per dayEarly warning system

Students By Course: • Student performance against goals

Learning Analytics – Metrics/Reporting

Course Reporting

In‐House Custom Reporting

13

Academic Leadership

Top 50 Courses Course Design & Trends

At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:

instructor, department, subject, term

Instructional Technology

Overall usage per date range User statistics per month Course statistics per month

LMS Usage and Trends LMS usage and trends Capacity planning Course open after term end

Assessment Coordinators

Aggregate, student, cross-course, cross-term goal performance (LMS scores)

Instructors

By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system

Course dashboard - comparison of instructor’s course design to other courses

Correlation of grades to activity

Students By Course: • Student performance against goals

Student dashboard - performance across courses and compared to peers

Learning Analytics – Metrics/Reporting

For LearnCourse Reporting

In‐House Custom Reporting

Information Needs Vary

DATA TO HELP ME

On DemandEasy to Access

and Easy to Digest

But There Are Common Themes

CustomerSuccess Story

Blackboard AnalyticsCustomer

Lingnan University Key Facts

• Public University – 2,800 Students

• Small Class Size

• High International Community

Blackboard AnalyticsCustomer

Lingnan University Key Facts

• Public University – 2,800 Students

• Small Class Size

• High International Community

Why Blackboard Analytics?

Blackboard AnalyticsCustomer

Lingnan University Key Facts

• Public University – 2,800 Students

• Small Class Size

• High International Community

Why Blackboard Analytics?

Impact to Lingnan University

Blackboard AnalyticsCustomer

Lingnan University Key Facts

• Public University – 2,800 Students

• Small Class Size

• High International Community

Why Blackboard Analytics?

Impact to Lingnan University

Thank you!

APPENDIX

George SiemensAssociate Director

Technology Enhanced Knowledge Research Institute

Thank you!

24

Best Practices

Defined Metrics

Business Rules

Derived Information

Transformation Layer

Data Warehouse

OLAP

Star Schema

Pre-Built Reports &

Dashboards

Blackboard AnalyticsA Packaged Analytics Application

Pre-Built Data Integration

SIS

For Leadership and Staff

Flexible Reporting

For Instructors

For Students

29

APPENDIX

30

31

32

33

34

35

36

37

38

Baseline Implementation

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12

Rapid Deployment

Homegrown Data Warehouses

Your Data is Loaded

5 Step Approachmonths

Design and Development Implementation

ERP Vendor Toolkits at $250 per hour

committed resources

40

Metrics & Dimensions for Analysis

•Gradebook•Possible Score•Score•Weight•Grade Pct•Weighted•Element Count•Highest Score•Lowest Score•Improvement•Average Score•Attempts•Days To Grade

•Goals•TBD

•Course Design•Item Count•Files Attached•VTBE Objects•VTBE Characters•File Size•Folder Depth•Links•Folders

•Course Section•Subject•Department•College•Class Number•Term•Course Career/Level•Instruction Method

•Students•Birthdate•Gender•Ethnicity•Citizenship•Current Term•Primary Major•Career/Level•Academic Level•Current Academic Standing•Primary College

•Instructors•Gender•Ethnicity•Highest Education Level•Faculty Rank•Tenure Status•Primary Department

•User Activity•Sessions

•Logins•Session Minutes•Courses•Content Items•Tools•Modules•Clicks•Pages

•Course Activity•Courses Accessed•Course Access Minutes•Session Course Sequence•Content Items•Pages

•Content Activity•Content Items Accessed•Content Item Hits•Content Access Minutes•Session Sequence•Folder (or Content) Depth

•Submissions•TBD

Learn MetricsKey SIS

Elements

41

Technical Requirements

• Requires the use of Blackboard LearnTM, Release 9.1, Service Pack 4 or later

• Pre-built SIS integration with:• Oracle (PeopleSoft)• Datatel (Colleague) • SunGard Higher Education • An Integration method is provided for other SISs

• Requires a locally hosted SQL Server environment• Minimum hardware & software requirements available on

request

42

Academic Leadership

At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:

instructor, department, subject, term

Instructional Technology

Overall usage per date range User statistics per month Course statistics per month

LMS usage and trends Capacity planning Course open after term end

Assessment Coordinators

Course, program, and gen ed-level goal & rubric-based reporting

Goal “coverage” reporting against curriculum

Artifact repository

Aggregate, student, cross-course, cross-term goal performance (LMS scores)

Instructors

By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system

Course dashboard - comparison of instructor’s course design to other courses

Correlation of grades to activity

Students By Course: • Student performance against goals

Student dashboard - performance across courses and compared to peers

Learning Analytics – Metrics/Reporting

For LearnCourse Reporting

43

Course Reporting

One course, one semester viewInstructors are primary users

44

Cross-course rubric

Assessment Coordinators are primary users

45

Cross-department, cross-college view

Many benefit – students, instructors, executives

for Learn