Kirkpatrick

The Four Levels

Reaction Learning Behavior Results

All about Kirkpatrick

In 1959, Kirkpatrick wrote four articles describing the four levels for evaluating training programs. He was working on his dissertation for a Ph.D. when he came up with the idea of defining evaluation.

Evaluation, as according to Kirkpatrick, seems to have multiple meanings to training and developmental professionals. Some think evaluation is a change in behavior, or the determination of the final results.

All about Kirkpatrick (continued) Kirkpatrick says they are all right, and

yet all wrong. All four levels are important in understanding the basic concepts in training. There are exceptions, however.

Kirkpatrick: Evaluating Training Programs

“What is quality training?”

“How do you measure it?”

“How do you improve it?”

Evaluating

“The reason for evaluating is to determine the effectiveness of a training program.” (Kirkpatrick, 1994, pg. 3)

The Ten Factors of Developing a Training Program

1. Determine needs

2. Set objectives

3. Determine subject content

4. Select qualified applicants

5. Determine the best schedule

The Ten Factors of Developing a Training Program

6. Select appropriate facilities

7. Select qualified instructors

8. Select and prepare audiovisual aids

9. Co-ordinate the program

10. Evaluate the program

Reasons for Evaluating

Kirkpatrick gives three reasons ‘why’ there is a need to evaluate training:

1.“To justify the existence of the training department by showing how

it contributes to the organizations’ objectives and goals.”

Reasons for Evaluating

2. “To decide whether to continue or discontinue training programs.”

3. “To gain information on how to improve future training programs.”

(Kirkpatrick, 1994, pg. 18)

The Four Levels

Reaction Learning Behavior Results

“The Four Levels represent a sequence of ways to evaluate (training) programs….As you move from one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information.” (Kirkpatrick, 1994, pg. 21)

Reaction:

is the measuring of the reaction of the participants in the training program.

is “a measure of customer satisfaction.” (Kirkpatrick, 1994, pg. 21)

Learning:

is the change in the participants’ attitudes, or an increase in knowledge, or greater skills received, as a result of the participation of the program.

Learning

The measuring of learning in any training program is the determination of at least one of these measuring parameters:

Did the attitudes change positively? Is the knowledge acquired related and

helpful to the task? Is the skill acquired related and helpful to

the task?

Behavior

Level 3 attempts to evaluate how much transfer of knowledge, skills, and attitude occurs after the training.

The four conditions Kirkpatrick identifies for changes to occur:

Desire to change

Knowledge of what to do and how to do it

Work in the right climate

Reward for (positive) change

When all conditions are met, the employee must:

Realize an opportunity to use the behavioral changes.

Make the decision to use the behavioral changes.

Decide whether or not to continue using the behavioral changes.

When evaluating change in behavior, decide:

When to evaluate

How often to evaluate

How to evaluate

Guidelines for evaluating behavior: Use a control group Allow time for change to occur Evaluate before and after Survey/interview observers Get 100% response or sampling Repeat evaluation, as appropriate Consider cost versus benefits

Results

Level 4 is the most important and difficult of all - determining final results after training.

Evaluation Questions:

Increased production? Improved quality? Decreased costs? Improved safety numbers? Increased sales? Reduced turnover? Higher profits?

Guidelines for evaluating results: Use a control group. Allow time for results to be achieved. Measure before and after the program. Repeat the measurements, as needed. Consider cost versus benefits. Be satisfied with evidence if proof is not

possible.

Case Study #1

INTEL CORPORATION

Intel’s Compromise of the Kirkpatrick Model

Intel uses the four-level model as an analysis instrument to determine the initial training needs and design of its training program; as well as using the model for evaluations.

Intel’s Compromise of the Kirkpatrick Model

Their uniqueness of using the model is in the fact that the designers of the training program worked backwards in the analysis of the training, starting with Level Four.

The Model

This implementation of the Kirkpatrick Model stands as vivid testimony to the versatility of the model as a training tool, and in developing fledgling training programs.

The Model

It also reflects the open-mindedness of the senior executives at Intel for their infinite use of the model and the use of the genius and visions of Kirkpatrick.

How Intel applies the analysis to their training program

Level Four …”Determine the organizations’ structure and future needs.”

Level Three. Change the environmental conditions and employee conditions to improve business indicators.

How Intel applies the analysis to their training program

Level Two. “Design a training program that would ensure a transfer of deficient skills and knowledge.”

Level One. Use a questionnaire, according to their skill level, that would instruct and inspire training participants.

How Intel applies evaluationto their training program Level One - Questionnaire.

Level Two - Demonstrate competency, create action plans through group simulations.

Level Three - Follow-up to determine if action plans were met (specific steps to implement concepts of what was learned).

Level Four - Ongoing process of tracking business indicators.

Case Study #2

ST. LUKE’S HOSPITAL

St. Luke’s is unique -

Evaluation of outdoor-based training program, not classroom.

Results analyzed statistically to determine the significance of any change.

Evaluation led to recommendations for future programs.

The New Questionnaire

Used before attendance in the program. Used 3 months after completion of the

program. Used again 6 months after completion of

the program. (Communication showed statistically significant

improvement, and Group Effectiveness showed

statistically significant change.)

Kirkpatrick’s 4 Levels of Evaluation are:

Level 1 - Reaction: how participants reacted to the program.

Level 2 - Learning: what participants learned from the program.

Level 3 - Behavior: whether what was learned is being applied on the job.

Level 4 - Results: whether that application is achieving results.

Post-test Questions

(1) Name three ways evaluation results can be measured.

(2) Do all 4 Levels have to be used?

(3) Do they have to be used in 1,2,3,4 order?

(4) Is Kirkpatrick’s method of evaluation summative or formative?

(5) Which developmental “view” does

Kirkpatrick use? (discrepancy, democratic, analytical, diagnostic)

“IF YOU THINK TRAINING IS EXPENSIVE, TRY IGNORANCE.”

and, remember, the definition of ignorance is

repeating the same behavior, over and over, and expecting different results!