2 Aug 2013PMG Meeting, Singapore

Measuring the effectiveness of training – myth or reality?

What is

EFFECTIVENESSof training?

What are some key

CHALLENGESin organization training?

Program design is not always linked to business objectives

Training may not be addressing the key needs of the organisation

Implication

Lack of effective post-training follow up, reinforcement and measurement

Training effectiveness

cannot be measured

Implications

Learning design cannot be

adjusted in real time

Performance gap not closed

in time

Participants are not clear about the business goals and strategic initiatives of

the company

Transfer of learning to the job may not happen

Implication

You cannot manage what you cannot measure!

”“

The

KIRPATRICKmodel for evaluation

The Kirkpatrick Model for training evaluation

Reaction

The Kirkpatrick Model for training evaluation

Smile sheetFeedback form

To what degree participants

react favourably to the learning

event

Level 1

LearningReaction

The Kirkpatrick Model for training evaluation

Pre/Post testPerformance

test

Smile sheetFeedback form

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 2Level 1

BehaviourLearningReaction

The Kirkpatrick Model for training evaluation

Superior f/backBehaviour obs.

Pre/Post testPerformance

test

Smile sheetFeedback form

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 3Level 2Level 1

ResultsBehaviourLearningReaction

The Kirkpatrick Model for training evaluation

% increase in sales, profits

etc.

Superior f/backBehaviour obs.

Pre/Post testPerformance

test

Smile sheetFeedback form

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

ResultsBehaviourLearningReaction

The Kirkpatrick Model for training evaluation

% increase in sales, profits

etc.

Superior f/backBehaviour obs.

Pre/Post testPerformance

test

Smile sheetFeedback form

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

Easy to implementUsually done

ResultsBehaviourLearningReaction

The Kirkpatrick Model for training evaluation

% increase in sales, profits

etc.

Superior f/backBehaviour obs.

Pre/Post testPerformance

test

Smile sheetFeedback form

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

Difficult to implementNot usually done

Easy to implementUsually done

How can this be

IMPLEMENTED

How can this be

IMPLEMENTED(easily)?

Start TNA with the end in mind – what are the organisation’s business

goals

1

Translate business goals into indicators or metrics for departments

and individuals

2

Identify the specific actions and behaviours that can positively impact

the business goals

3

Actions and behaviours

must be measurable

Write learning objectives that can be drivers for these actions and

behaviours

4

Build post-training activities during program design

5

Only 15% of what is learned during training will be applied on the job if it is not reinforced and monitored.

- Research by Robert Brinkerhoff

ResultsBehaviourLearningReaction

Starting with the end in mind

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

ResultsBehaviourLearningReaction

Starting with the end in mind

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

“Start with the end in mind”

“Identify indicators and behaviours”

“Design post training activities with program design and evaluation”

ResultsBehaviourLearningReaction

Starting with the end in mind

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

1

“Start with the end in mind”

“Identify indicators and behaviours”

“Design post training activities with program design and evaluation”

ResultsBehaviourLearningReaction

Starting with the end in mind

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

12 3

“Start with the end in mind”

“Identify indicators and behaviours”

“Design post training activities with program design and evaluation”

ResultsBehaviourLearningReaction

Starting with the end in mind

To what degree targeted outcomes occur, as a

result of learning

event(s) and subsequent

reinforcement

To what degree participants apply what

they learned during training when they are back on the job

To what degree participants acquire the intended

knowledge, skills and attitudes based on their participation in

the learning event

To what degree participants

react favourably to the learning

event

Level 4Level 3Level 2Level 1

12 34 5

“Start with the end in mind”

“Identify indicators and behaviours”

“Design post training activities with program design and evaluation”

What will this

ACHIEVE?

Framework for measurement of

effectiveness of training

Training aligned to business goals

Ability to make real time changes to help

meet the business goals

Measurable and Effective Training

What is needed for the

SUCCESSof this model?

Agreement on business measures is required even before the learning

intervention is designed

Other teams need to get involved in the entire learning process

Implication

Participants (and their supervisors) need to be aware of the business goals, the objectives of the program and how it links to their job and

KPIs

Communication between

HR/Training and other teams is

critical

ImplicationsParticipants and supervisors must understand what will be measured

after training

Supervisors may need to be included in the learning and evaluation design

process

Appropriate tools and processes for post training follow up and measurement need to be

put into place

Indicators and measures of behaviours and actions

need to be defined clearly and realistically

Implications

Process of measurement should be defined

Monitoring mechanism / tools need to be available to check in real time if performance gaps are being closed and business goals are

being met

Business dashboard for training / learning & development is required for monitoring and tracking

Implication

But the mostCRITICAL

NEEDis this…

HR has to STOP playing a support role in the organisation!

HR needs to start playing the role of a

strategic business partner in the organization

Accord Business Solutions Pte. Ltd.146 Robinson Road, #07-01, Singapore 068909

Phone: +65 9298 9590Singapore | India | UAE

www.accordsingapore.com.sg | saurabh@accordsingapore.com.sg

A presentation by Saurabh Anand