U21 Educational Innovation ConferenceUniversity College Dublin31 October – 01 November 2013

Exploring the Cost & Benefits of Online Innovation

Diana LaurillardLondon Knowledge Lab

Institute of Education

Outline of the argument

• Global demand for education

• Why we need to understand costs and benefits• What it takes to teach with technology

• Teaching on the large scale

• Tools for teachers as designers

• Modelling costs and benefits

Exploring the costs and benefits of online innovation in education

The global demand for education

The new UNESCO goals for education:• Every child completes a full 9 years of

free basic education … • Post-basic education expanded to meet

needs for knowledge and skills … (Draft for UNESCO post 2015 goals)

By 2025, the global demand for higher education will double to ~200m per year, mostly from emerging economies (NAFSA 2010)

Implying significant graduate and teacher training growth for this level of schooling, FE and HE

1:25 staff:students??

The global demand for HE is a demand for new pedagogies capable

of educating millions – far from the current model

Teaching has to adapt continually to rapid changes in opportunities

from digital technology innovation, changing student needs,

capabilities and expectations

teachers have to discover the digital pedagogic forms that

scale up for large-scale high quality teaching

The iterative and adaptive nature of design must be at the core of

teaching innovation in HE in this rapidly changing world

Why we need to understand costs and benefits

• Adaptive feedback (sim/models/games)

• Expositions (lecture videos)

• Automated grading (MCQs, quizzes)

• Readings (pdfs)

• Collaboration activities (wiki)

• Peer group discussion (forums)

• Peer grading against criteria (tests)

• Tutored discussion (forums)

• Tutor feedback (e-portfolio)

• Adaptive feedback (sim/models/games)

• Expositions (lecture videos)

• Automated grading (MCQs, quizzes)

• Readings (pdfs)

• Collaboration activities (wiki)

• Peer group discussion (forums)

• Peer grading against criteria (tests)

• Tutored discussion (forums)

• Tutor feedback (e-portfolio)

• Adaptive feedback (sim/models/games)

• Expositions (lecture videos)

• Automated grading (MCQs, quizzes)

• Readings (pdfs)

• Collaboration activities (wiki)

• Peer group discussion (forums)

• Peer grading against criteria (tests)

• Tutored discussion (forums)

• Tutor feedback (e-portfolio)

Understanding high quality TEL

MOOC Preparation time (fixed costs)

Support time (variable costs)

MOOC vs standard online course

What it takes to teach with technology

The teaching workload is increasing in terms of Planning for how students will learn in the mix of the physical, digital and social learning spaces designed for themCurating and adapting existing content resourcesDesigning the activities, tools and resources that afford all types of active learning Personalised and adaptive teaching that improves on traditional methodsProviding flexibility in blended learning optionsGuiding and nurturing large cohorts of studentsUsing learning technologies to improve scale AND outcomes

BUT: Institutions and teachers do not typically plan for the teaching workload implied by these learning benefitsnor for the need to collaborate to innovate with technology

The MOOC as ‘large-scale’ pedagogy

MOOCs are not large scale – Duke University

Completed = 2% of enrolment, 25% of ‘engaged’Duke University Report 2012

The MOOC as ‘large-scale’ pedagogy

Average student numbers per course - Edinburgh

Statement of Accomplishment

Week 5 asst's

Engaged Week 1

Accessed Week 1

Enrolled

0 10000 20000 30000 40000 50000 60000

5500

6000

15000

20500

51500

Completed = 10% of enrolment, 37% of ‘engaged’

MOOCs @ Edinburgh 2013 – Report #1

The MOOC as undergraduate education

Not for undergraduates

Enrolled students

Duke University Report 2012

72% have degrees

The MOOC as undergraduate education

Not for undergraduates

Enrolled students

Less than high school

School

College

Degree

PG degree

0% 5% 10%15%20%25%30%35%40%45%

40%

30%

17%

10%

3%

MOOCs @ Edinburgh 2013 – Report #1

70% have degrees

What it takes to teach a basic MOOC vs the Duke MOOC

Teaching time 50 500 5000Duke MOOC 20 hrs 200 hrs 2000 hrsBasic MOOC 0.00 0.00 0.00

50 500 50000

500

1000

1500

2000

2500

3000

Duke MOOCBasic MOOC

50 500 50000

500

1000

1500

2000

2500

3000

Duke MOOCBasic MOOC

Total teaching time

Preparation time = 420 hrs

The variable cost of high quality teaching does not achieve economies of scale if you maintain the same pedagogyPrep

time = 420

Duke University Report 2012

Modelling the benefits and costs

It’s important to understand the link between the pedagogical benefits and teaching time costs of online learning – especially for the large-scale

What are the new digital pedagogies that will address the 1:25 student guidance conundrum? – How to shift variable cost support to fixed cost support?

Can we develop a viable business model that will make HE affordable for novice learners?

Concealed MCQs

The (virtual) Keller Plan

The vicarious master class

Pyramid discussion groups

Pedagogies for supporting large classes

Tutorial for 5 representative studentsQuestions and guidance represent all students’ needs

Conceal answers to questionAsk for user-constructed input Reveal multiple answersAsk user to select nearest fit

240 individual students produce response to open questionPairs compare and produce joint responseGroups of 4 compare and produce joint response and post as one of 10 responses...6 groups of 40 students vote on best responseTeacher receives 6 responses to comment on

Introduce contentSelf-paced practiceTutor-marked testStudent becomes tutor for creditUntil half class is tutoring the rest

Pedagogies for supporting large classes

Concealed MCQs

The (virtual) Keller Plan

The vicarious master class

Pyramid discussion groups

Laurillard, 2002

Keller, 1974

Mayes et al, 2001

Gibbs et al, 1992

The traditional pedagogies for large classes could be redesigned as digital formats

1. Library of learning designs indexed by learning outcome & topic2. Three different subject instances for each to promote migration of good

pedagogy across domains3. Academic ‘adopts’ a design and adapts it as needed using edit tools &

links to other resources – creating a computationally interpretable design

4. Library of OERs for academics to link to from learning designs5. Feedback on ‘learning experience’ created, total learning time, and

teacher preparation and contact time6. Prompt to include ‘production’ activity for collecting learning analytics on

outcomes7. Design sent to Moodle to test with students – collects data8. Students can annotate design for detailed evaluation9. Academic redesigns as needed – tests again – publishes to Library

Tools for academics as learning designers

1. Library of learning designs indexed by learning outcome & topic2. Three different subject instances for each to promote migration of good

pedagogy across domains3. Academic ‘adopts’ a design and adapts it as needed using edit tools &

links to other resources – creating a computationally interpretable design

4. Library of OERs for academics to link to from learning designs5. Feedback on ‘learning experience’ created, total learning time, and

teacher preparation and contact time6. Prompt to include ‘production’ activity for collecting learning analytics on

outcomes7. Design sent to Moodle to test with students – collects data8. Students can annotate design for detailed evaluation9. Academic redesigns as needed – tests again – publishes to Library

Teachers as design scientists need the tools for innovation

Tools for teachers as learning designers

To find or create new ideasAdoptAdaptTest

To collect learning analyticsRedesignAnalysePublish

Creating knowledge about effective blended and online pedagogies

http://tinyurl.com/ppcollector

Tools for academics as learning designers

http://tinyurl.com/ppcollector

1. Library of learning designs indexed by learning outcome & topic

2. Three different subject instances for each design to promote migration of good pedagogy across domains

3. Academic adopts’ a design and adapts it as needed using edit tools & links to other resources – creating a computationally interpretable design

4. Library of OERs for academics to link to from learning designs

5. Feedback on ‘learning experience’ created, total learning time, and teacher preparation and contact time

6. Prompt to include ‘production’ activity for collecting learning analytics on outcome

7. Design sent to Moodle to test with students – collects data

8. Students can annotate design for detailed evaluation

9. Academic redesigns as needed – tests again – publishes to Library

Tools for academics as learning designers

Select Adopt

Adapt

Test

Redesign

Test

Publish

The design cycle for teaching

Feedback - learner time

Redesign

Feedback - teacher time

Redesign

Feedback- learning

types

Redesign

Building teaching community knowledge

Make links to existing content

resources

Redesign existing content

resources?

Build on others’ tested designs

Select Adopt

Adapt

Test

Redesign

Test

Publish

Similar to the design cycle for science

Building scientific knowledge

What is the teaching design

equivalent of the journal paper?

A learning design for Ed students

Check the feedback on the overall distribution of learning activity

Add link to an OER, e.g. a digital tool for practice

Make links to existing content

resources

The learning design as a ‘content shell’ ready to receive content products?

Export to Word or LMS/VLE

Export to Moodle for Ed students

• Interprets metadata to assign activity types in Moodle (or other LMS)

• Attaches resource links• Inserts study guidance from text

in the pattern• Collects data on student

performance on TEL-based activities

Reversioned for Med students

• Same pedagogical pattern• Same study guidance except

for subject content terms and resources

• Different resources attached• Same type of evidence data (?)

AcquisitionInquiryDiscussionPracticeProduction

Acquisition

Inquiry

Discussion

Practice

Production

Conventional

Blended

Categorised learning activities

Analysis shows more active learning

A computational representation can analyse how much of each learning activity has been designed in

Modelling the pedagogic benefits

The Course Resource Appraisal Model …

Run No. of studentsRun 1 15Run 2 20Run 3 20

Run 1 Run 2 Run 3Students 15 20 20Profit -£27k £4k £11k

Run No. of studentsRun 1 15Run 2 30Run 3 60

Run 1 Run 2 Run 3Students 15 30 60Profit -£27k £11k £38k

Modelling the teaching time costs

An interactive learning design tool can analyse how much design and teaching time is needed

Design hrs

Teaching hrs

Yr1 Yr2 Yr3 Yr1 Yr2 Yr3

40 40 40 40 80 160

Design hrs

Teaching hrs

Students Students

• Create and test professional content

• Curate existing professional content

• Adapt and customise existing content

• Select and organise community content

• Create reusable learning designs as ‘content shells’

• Adapt and customise existing learning designs

Managing the fixed costs of teaching

✓✓✓

✓✓✗

✓✗

✗

✗

✗

What academics need Do publishers help?

Could the learning design interface to the VLE, for creating ‘content shells’ suggest a new kind of content product?

Teaching as a Design Science: Building pedagogical patterns for learning and technology (Routledge, 2012)

d.laurillard@ioe.ac.uk

tinyurl.com/ppcollector

http://buildingcommunityknowledge.wordpress.com

Further details…

Teaching as a design science: Summary

The global demand for education requires investment in pedagogic innovation for high quality large scale teaching

We need to design, test, use, and reuse high quality open educational resources that amortise high fixed costs over large student cohorts

We need to invent variable cost pedagogic innovations that supports students at a better than 1:25 staff-student ratio

Teachers need the tools to design, test, gather the evidence of what works, and model benefits and costs

Teachers are the engine of innovation – discovering the means by which we fulfill our social responsibility of doing more for less to meet that global demand

Break-out questions

1. Should our universities play any role, or take any responsibility for meeting the global demand for HE? [Slides 2, 3]

2. Can universities and teachers plan for the teaching workload implied by the learning benefits that technology can confer? [Slide 4]

3. Can academic teachers play a part in discovering the digital pedagogic methods that will scale up to provide large-scale high quality teaching in order to reduce the costs to students/government of meeting the national demand for HE and lifelong learning? If not – who? [Slide 5, 6]

4. Can we use pedagogy-driven learning analytics to understand better the relationships between teaching and learning? [Slide 6]

5. Can university teachers collaborate to innovate with technology? [Slide 7]6. Do we know the real costs of current teaching as a business model with a

related return? How can we understand the new cost models for moving to online courses if we do not have a clear activity-based cost model for current teaching? [Slides 8, 9]