Evaluating Online Learning Programs

Upcoming MCCVLC Webinars

• Apr. 12 – Effective Practices for • New Online Student Orientation• Student Support Services• Professional Development

• Apr. 19 – Accessibility for Online Learning

Comparing Program Reviews to Accreditation

How does the accreditation process

inform our program reviews?

Definitions• Distance Programs (as per HLC)• Distance-delivered programs are those

certificate or degree programs in which 50% or more of the required courses may be taken as distance-delivered courses.

• Source: HLC Substantive Change Application

Definitions• Distance-delivered courses are those in

which all or the vast majority (typically 75% or more) of the instruction and interaction occurs via electronic communication, correspondence, or equivalent mechanisms, with the faculty and students physically separated from each other.

• Source: HLC Substantive Change Application

Definitions• Correspondence Education:• 1) Education provided through one or more

courses by an institution under which the institution provides instructional materials, by mail or electronic transmission, including examinations on the materials, to students who are separated from the instructor.

• Source: HLC Substantive Change Application

Definitions• Correspondence Education: (cont.)• (2) Interaction between the instructor and the

student is limited, is not regular and substantive, and is primarily initiated by the student.

• (3) Correspondence courses are typically self-paced.

• (4) Correspondence education is not distance education.

• Source: HLC Substantive Change Application

Percentage Bracket

Percentage of Distance-Delivered Offerings(calculations are based on degree programs, not certificates)

3 Up to 100% of total degree programs

2 Up to 20% of total degree programs

1 Initiation of distance-delivered offerings up to 5% of total degree programs

0

No degree programs or more than four certificate programs (Commission policy does permit the institution to offer up to four certificate programs as well as a limited number of courses leading to degree programs through distance education or correspondence education without seeking prior approval.)

Seeking Approval from HLC

Initiation with the HLCInitiation occurs and a review is required when the institution:

– plans to initiate its first distance education degree program

– has initiated four distance education certificates and plans to initiate a fifth

• NOTE: Add’l language about correspondence omitted

Expansion of Distance EducationExpansion occurs and a review is required when the institution:

– plans to increase its current activity in distance education degree programs (or correspondence) to a higher Percentage Bracket.

An HLC Review• Comprehensive reviews examine

– curriculum, – staffing, – support services, – access to appropriate laboratory and

library resources, – and all other facets of quality higher

education.

Interregional Guidelines for the Evaluation of Distance Education

(Online Learning)

http://xlents.com/2012/02for links and info

C-RAC (Council of Regional Accrediting Commissions)

The new guidelines have been endorsed by all regional accrediting

organizations in the U.S.

9 Hallmarks of Quality1. Appropriate to mission and purposes2. Part of regular planning and evaluation3. Incorporated into governance/oversight4. Curricula are coherent, cohesive, &

comparable in academic rigor (to trad)5. Evaluates effectiveness of online

offerings, incl. outcomes achievement

9 Hallmarks of Quality

6. Online faculty are appropriately qualified & effectively supported

7. Effective student and academic services for online students

8. Sufficient resources to support and expand online offerings

9. Assures integrity of online offerings

On-CampusProgram Review

What is the purpose?

On-CampusProgram Review

What is your established practice?

Example: Glendale CC

Glendale – Trend Analysis

Glendale – Student Learning Outcomes

Glendale – Parts 3, 4, 5• 3.0 Evaluation of Previous Goals• 4.0 Action Plan

– What is the goal? • Is it reflected in the Academic Master Plan?

– What are the SLOs• Core competencies (college-wide)• Program SLOs• Course SLOs

– How will it improve student learning?• 5.0 Resource Requests

Comparing On-Campus to Online Program Reviews

In what ways should review of online programs be different?

A Starting Point• Gather proof that your courses are distance

education courses and not correspondence courses. Gather evidence of:– Regular and substantive interaction between the

instructor and the student – and not primarily initiated by the student.

– Timeline expectations for online students to indicate that they are not self-paced. (Maybe!)

Comparing Online Program Data• Possible benchmarks:

– Similar on-campus programs– Similar online programs– Trend analysis over time– Internally set goals or targets– State or National data (to the extent

possible)

Possibilities to Consider• Course Design Quality• Online Teaching Evaluations (by admin)• Online Course Evaluations (by students)• Assessments of Learning Outcomes

– College-wide outcomes– Program outcomes

3 Major Components of e-Quality

Learning LevelIs High

Teaching LevelIs High

Course DesignMeets Standards

Learning Assessment

Performance Eval Qua

lityM

atter

s®

Designing for Interaction

Specific Review Standards PointsV.1 The course design provides learning activities to foster instructor-student, content-student, and if appropriate, student-student interaction. 3

V.2 The student requirements for course interaction are clearly articulated. 3

V.3 Clear standards are set for instructor response and availability (turn-around time for email, grades posted, etc.) 2

V. LEARNER INTERACTIONGeneral Review Standard: The effective design of instructor-learner interaction andmeaningful learner cooperation is essential to learner motivation, intellectual commitment, and personal development. (From LSC Course Design Rubric)

More Possibilities to Consider• Student e-Services Audit (if program specific)

• Student Satisfaction Surveys• Course Completion Rate Analyses• Course Success Rate Analyses • Term-to-term Persistence Analyses• Graduation Rate Analyses• Transfer & Placement Rate Analyses

Review of e-Services

CENTSS

More coming on April 12

Student Satisfaction

Option 1 - PSOL

Why PSOL?

PSOL BasicsThere are 72 questions that comprise the PSOL. Average completion time is 15 minutes.

NOTE: questions are answered on a 7-point Likert scale, where 7 is high.

26 Priorities statements (can add 10)

7 Information sources about school/program

11 Factors to enroll in the program

3 Overall satisfaction questions

14 Demographics questions (can add 1)

The Importance of Importance

Data Comparisons

36

A.A. Students are Very Satisfied

Reputation Offerings Library Quality4.6

4.8

5.0

5.2

5.4

5.6

5.8

6.0

6.2

LSC National A.A. Business Other

2006 Data

5-year Trend Data

38

Completion and Success

Grades Earned

A B C D F FN W0%

5%

10%

15%

20%

25%

30% LSC Developmental Courses - 2009-2010

Online On-ground

Passing Grades in These Courses

C or better D or better50%

55%

60%

65%

70%

60.4%

67.5%

62.3%

66.8%

Passing Grades in Developmental Courses

Online On-ground2009-2010

Persistence

Graduation

Placement

Transfer

Q & A

Evaluating Online Learning Programs