Franklin Consulting

Towards an effective framework for the evaluation of e-learning

Tom Franklintom@franklin-consulting.co.uk

Jim Petchjim.petch@man.ac.uk

Jill Armstrongjill.armstrong@HEAcademy.ac.uk

Franklin Consulting

Types of evaluation

Formative Summative Illuminative Integrative Auditive

After Oliver 2000

Franklin Consulting

Purposes of evaluation

Control Understanding Action

Evaluation for Values are Self evaluation

External evaluation

Empirical evaluation

Action & understanding

Local and personal

Toolkits Peer review Formative evaluation

Action and control

External and fixed

Checklists QA review Outcomes based league tables

Franklin Consulting

Stakeholders

Institution Department / school / faculty Course team Students External authorities

Franklin Consulting

Proposed model

ReviewP

lanningC

ou

rse desig

n

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Franklin Consulting

Example models

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Focus of model

Included in model

Institute for Higher Educatrion Policy

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Focus of model

Included in model

American Council for Education

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Focus of model

Included in model

Open and Distance Learning Quality Council

ReviewP

lanningC

ou

rse desig

n

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Focus of model

Included in model

Quality Assurance Agency

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

Drivers

Processes

Start

En

d

Focus of model

Included in model

Western Cooperative for Educational Telecommunications

Franklin Consulting

Review and planning Curriculum creation Teaching and Learning Professional body review

Institutional mission / strategic plan

Professional requirements

Professional accreditation

Professional accreditation

Student motivation

Student evaluation Institutional capacity

Design for QA External examiners External examiners Institutional audit

External evaluation

External collaboration

Health and safety QA validation processes

Formal QA course review

Regional agenda Ethics Institutional course procedures

Infrastructure

Drivers and controls

Professional bodies

Stage Review Planning Course design Development Teaching Learning Resource

allocation Resource allocation Resource allocation Resource allocation Learning

strategies Course team evaluation

Develop workplan

Staff development Recruitment Entry requirements Ubiquitous computing

Formative assessments

Admissions Course review against strategy and operational plan

Market research

Curriculum design Course aims Learning Outcomes Assessment strategies Teaching and learning

strategies

Learning / e-environment development Induction

Undertaking summative assessments

Teaching processes Institutional capacity

Course content development / acquisition Setting and marking

assessment

Feedback to students

identify course development team

Integration of course procedures

Pastoral support Achievement and progression

Decision to proceed with course

Promotion and marketing

Technical support Peer mentoring

Library support Cost analysis Learner support Student tracking

Development Retention

Processes

Peer mentoring

Franklin Consulting

Major dependencies in the e-learning life Cycle

ReviewP

lanningC

ou

rse desig

n

development

Tea

chin

gL

earn

ing

Start

En

d

Franklin Consulting

Jim Petch - Uniform

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Jill Armstrong – business school

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

John Davey – Edge Hill

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Tony Toole - Glamorgan

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Gayle Calverly - Manchester

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Computer aided assessment

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Open University

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Review

Planning

Co

urse d

esign

development

Tea

chin

gL

earn

ing

StartE

nd

Franklin Consulting

Conclusion

Evaluation must be designed for action if it is to have any impact

A coherent framework is needed to understand the processes

We offer a model of the full life-cycle which can Support greater understanding of the processesHelp identify critical issues

Franklin Consulting

Towards an effective framework for the evaluation of e-learning

Tom Franklintom@franklin-consulting.co.uk

Jim Petchjim.petch@man.ac.uk

Jill Armstrongjill.armstrong@HEAcademy.ac.uk

www.elrc.ac.uk