www.efquel.org

Online Peer-Review... of e-learning programmes

Anthony F. Camilleri

ECB-Check TrainingITC/ILO Turin – 28.09.2012

www.efquel.org

What‘s in a Name?

Audit Expert-Review

Objective Subjective

Precise standards Broad guidelines

Verifies account of client Interprets account of client

Requires detailed knowledge of administrative procedures

Requires detailed knowledge of the subject area concerned

standards-basedreview

www.efquel.org

Who is a peer?

• an expert in the field• an expert in what field?• e-learning!• what is e-learning?

www.efquel.org

Features of an Expert-Peer

experience in course design and delivery

familiar with best-practice

comparative perspective

sense of missionnatural communicator

www.efquel.org

Inter-subjectivity

• different subjective perceptions, taken from different viewpoints, give an objective view of reality

• the quality basis of an external review• requires consensus

consensus is not a diplomatic nicetybut an essential pre-requisite of quality

www.efquel.org

Communicating with the institution

Communication starts

• This is not a one-way exercise!– Has the institution given enough

information to allow a successful review?–What don‘t you understand, and who can

answer your questions?• Be active in schedule-design

with self-assessment!

www.efquel.org

Conducting a Review

Familiarise yourself with the standards

1. Understand which standard covers which area (sometimes they can overlap).

2. Understand their relation to each other.3. Make sure you have a vision of successful and non-

successful completion of each standard – based on comparative experience

www.efquel.org

Inspect the Course

„Most Quality Managers are Magicians“

Look Here

don‘t look here!

www.efquel.org

Inspect the Course

• Log onto the LMS• Look through every part of the

course:– Lessons– Help-pages– Tests– Forums / support pages– Interactive material–What else?

www.efquel.org

Fill in your Forms

• Your forms are a letter to the agency

COMMUNICATE!

www.efquel.org

Communicating with the agency

„I don't know anything about art, but I know

what I like“

Gelett Burgess

www.efquel.org

Communicating with the agency

„I don't know anything about the standards, but I know quality when I see

it“Unnamed reviewer

www.efquel.org

Communicating with the agencyStandards of Proof

• Some credible evidence• Preponderance of evidence• Clear and convincing evidence• Beyond reasonable doubt

www.efquel.org

Communicating with the agencyIn your Report

• Be yourself

Bad: The institution showed....

Good: The review team saw / found / observed....

www.efquel.org

Communicating with the agencyIn your Report

• Be specific

Good: The institution showed....

Better: The review team found multiple and consistent examples of

www.efquel.org

Communicating with the agencyIn your Report• Say what you know

Bad: The institution lied....

Good: The review team found inconsistencies between evidence (x) and interview (y)

www.efquel.org

Communicating with the agencyIn your Report• Give your opinion (where relevant)

Bad: The institution is... / or NOTHING

Good: We suspect, It seems likely that, Given the evidence available, etc...

www.efquel.org

Communicating with the agencyIn your Report

• Give your reasoning

Bad: There is no quality.

Good: When we consider (X), (Y) and (Z), we find it impossible to say there is quality

www.efquel.org

Communicating with the agencyIn your Report• Be CLEAR

Bad: The structure of the LMS is in need of improvement.

Good: The LMS barely functions – it is littered with wrong links, the help-function is unusable and most sections still need to be populated

www.efquel.org

Communicating with the agencyIn your Report• Link Effect with Cause

Bad: Quality systems are in place, but there is no evidence of iterative improvement.

Good: Quality systems are not effective, due to lack of iterative improvement procedures.

www.efquel.org

One last thing

Remember that

Quality Labels should act as Quality-Enhancers

not only transparency tools

offer a path towards quality improvement

www.efquel.org

Hvala!Thank-you for your attention

Questions?

Anthony F. Camilleri (anthony@kic-malta.com)Presentation available from: http://www.slideshare.net/anthonycamilleri/

Released under a Creative Commons Attribution ShareAlike 3.0 Slovenia License

You are free:• to Share — to copy, distribute and

transmit the work• to Remix — to adapt the work

Under the following conditions:Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).Share Alike — If you alter, transform, or build upon this work, you may distribute the resulting work only under the same or similar license to this one.