Costs of e-learning

in HE - and the benefits

Professor Paul BacsichSheffield Hallam University, UK

2

Methodological foundation

“Suitably interpreted and broadened, and leavened with pedagogy,

the rather dry methods of business costing and planning can provide a sound basis for

analysing e-learning developments and their impact.”

3

Summary CNL Phase 1 [JISC JCALT] CNL Phase 2 [JISC JCALT] Effectiveness: impact of other work

including analysis of the training sector [EU Telelearn etc] and the NLN Evaluation work [FEDA]

Some impact from related work including RCIT (CNL methods applied to admin IT systems) [JCALT] and soon perhaps from “Managing Change” (formerly TLTR) on change management in HEIs [JCIEL]

4

Main issues

What learners want - must be first!Costs Effectiveness

5

What learners want...

From our own work on learners (CNL): students believe that e-learning increases

costs to them but they behave as though it saves costs

and saves time!

6

What learners want

From our own work on learners (CNL): students believe that e-learning increases

costs to them but they behave as though it saves costs

and saves time!

From other work (including USA): they want flexibility (but not too much) some want overclocking (ie faster

throughput)

7

Hidden Costs - examples

Increased home telephone call bills for students due to Internet usage

Entertainment expenses incurred by lecturers but not reimbursed by the HEI

The usage of teaching or research budgets to bolster IT budgets - or vice versa

CNL1

9

In order to accurately record the Costs of

Networked Learning you must have a universally accepted method of what costs to record and how.

Much of the literature formed interesting background reading but failed to travel far enough towards operational conclusions to be taken (individually) as a basis.

The method must include a lifecycle model, stakeholders, activity based costing and a “cost-aware” approach to planning.

Conclusions - CNL1

10

Course Lifecycle Model

Planning and Development

Production and Delivery

Maintenance and Evaluation

Three-phase model of course development

11

Breakdown of three-phase model

Planning andDevelopment

collecting materials coming up with - or being told -

the idea writing user guides or course

publicity

Production andDelivery

curriculum delivery duplication of materials tutorial guidance

Evaluation andMaintenance

quality assurance exercises replacement and updating of

materials evaluation against course aims

12

The main Stakeholders The HEI and the staff of that HEI

The learners

13

Stakeholders - two more special cases The HEI and the staff of that HEI

The learners and their organisation

and in some cases their parents/partners

14

The Financial Schema- cf Transparency Review

The traditional financial model underpinning work in most HEIs is defective in four ways:

No Stakeholders

Crude Overhead Allocation

No agreed set of activities

No time recording

15

Planning Document and Financial Schema

Multi-level

Activity-Based Costing

Flexible Overheads

Three-phase Course Lifecycle

Model

Based on previous works

Multi-stakeholder

Time DivisionTime recording

Activity-Based Costing

Activity-based costing (ABC)

ABC was developed as an alternative costing methodology

by Robin Cooper and Robert Kaplan of the Harvard Business School (1988) in their research

into product costing in the manufacturing industry.

18

Cooper and Kaplan (1988)

They argued that traditional costing distorted product costs (“peanut butter spread approach”)

There is a cost to all activities carried out within an organisation

Activity costs should be distributed to products in relation to their use

ACTIVITY DRIVERS

Hold Tutorials

No. Students

Business Intelligence

Resources

Cost for each Customer/product/channel group

method Activities

Staff time

Cost driver

I.T. & Management

I.T. & Management

FT

FT

DL

Activity A series of related tasks carried out repeatedly

Chase a student for late payment

Process A series of activities required to achieve an outcome

Invoicing Process

Task One action by one person at one point in time

Telephone student

Definitions

21

Advantages of ABC

Gives more than just financial information

Fairer system of overhead allocation Accountability of central services Highlights cross-subsidisation Recognises the changing cost behaviour

of different activities as they grow and mature

Can provide data for other initiatives

22

Disadvantages of ABC

Data collection is costly and time consuming

Initially it can be difficult to collect the data you want

Determining appropriate cost drivers More complex system

23

Implementing ABC takes time

“Its important in a service organisation to get started with ABM and not worry about total accuracy from the start. Accuracy will improve with time”

Antos (1992)

24

CNL2

This was a trial of ABC in one faculty of Sheffield Hallam University, using industrial-strength ABC software and consultancy (Armstrong-Laing)

The trial was funded by JISC JCALT The report of the trial (plus workbook)

will be available in August 2001 The full set of costs are not reported!

Tiptoeing into effectiveness issues

26

History

Relatively early in the history of CNL1, effectiveness issues were looked at (Ash, November 1999)

Main conclusion was that a uniformly accepted methodology was needed there also; and some suggestions were given, based to a certain extent on the US Flashlight work.

This work led into the NLN evaluation studies (English FE sector)

27

Effectiveness: a big “border” between E & T The NSD effect in academia

(No Significant Difference)

versus learning gains and large Return on Investment in training world: 700% ROI at Royal Bank of Scotland

(Shepherd) routine 10-25-40% learning gains and situations with 60% learning gains

28

NSD in academia... It is commonly believed, and apparently

validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students

The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work.

29

NSD in academia It is commonly believed, and apparently

validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students.

The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work.

But we contend that the NSD work has several major flaws - in particular many key variables were not controlled.

30

Self-incrimination? “There were no significant pre- and

post-course differences on the items related to students’ expectations and perceived benefits...

There were no significant differences between students in the [online] compared to students in the [face to face] courses for course retention or course grade...

but there were increases in the amount of time and the purposes for which students used computers." (Green et al 2001)

31

So is a key difference to do with time? And what is time?

32

What is time?

“Time is the new distance”(Mason, inaugural lecture 2001)

33

Can e-learning save time? travel time

34

Can e-learning save time?

travel time reduce “time on task”

by “better” training (25-40%)

by individualised training...

35

Can e-learning save time?

travel time reduce “time on task”

by “better” training (25-40%)

by individualised training... by using “time of the 3rd kind” by consolidating time fragments

36

Time of the 3rd kind

Time1 : on duty - used to be called “at work”

Time2 : off duty (not at work)

37

Time of the 3rd kind

Time1 : on duty - used to be called “at work”

Time2 : off duty (not at work) Time3 : in-between

on duty but less productive eg travelling eg slots in between other tasks

off-duty but somewhat productive eg in bar with colleagues at dinner with customers

38

One key conclusion

To understand effectiveness issues we need a much better understanding of time and its monetary and other value to

students Only when one understands time should

one move on to other measures of effectiveness

Details about our work at:

http://www.shu.ac.uk/cnl/

Paul BacsichProfessor of Telematics

School of Computing and Management SciencesSheffield Hallam University