The purpose for Kirkpatrick’s evaluation is to determine the effectiveness of a training program. According to this model, evaluation should always begin with level one, and then, as time and budget allows, should move sequentially through levels two, three, and four. Information from each prior level serves as a base for the next level\'s evaluation. The purpose of Brinkerhoff’s SCM to prove and to improve impact. It is a cost effective way in determining which components of initiative are working and which are not, and reporting result in a way that organizational leaders can easily understand and believe.
11
Kirkpatrick’s Levels of Kirkpatrick’s Levels of Evaluation: Evaluation: Larry D Weas, Ed. D. Program Larry D Weas, Ed. D. Program ETR 531 Program Evaluation ETR 531 Program Evaluation Education, Technology & Research Education, Technology & Research Northern Illinois University Northern Illinois University Model for Evaluating Training Model for Evaluating Training Programs Programs Presented by: Presented by: … … Brinkerhoff’s Brinkerhoff’s Success Case Success Case Method (SCM) Method (SCM)
Transcript
Kirkpatrick’s Levels of Kirkpatrick’s Levels of Evaluation:Evaluation:
Larry D Weas, Ed. D. ProgramLarry D Weas, Ed. D. Program
ETR 531 Program EvaluationETR 531 Program Evaluation
Education, Technology & ResearchEducation, Technology & Research
Northern Illinois UniversityNorthern Illinois University
Model for Evaluating Training ProgramsModel for Evaluating Training Programs
Presented by:Presented by:
……Brinkerhoff’s Brinkerhoff’s Success Case Success Case Method (SCM)Method (SCM)
An Evaluation Is An Evaluation Is WorthWorth a Thousand Words a Thousand Words
The four-level model was developed by Donald The four-level model was developed by Donald Kirkpatrick (1998). In Kirkpatrick's four-level model, Kirkpatrick (1998). In Kirkpatrick's four-level model, the focus is four levels (the focus is four levels (Reaction, Learning, Reaction, Learning, Behavior, Behavior, andand Results Results), which represent a sequence ), which represent a sequence of ways to evaluate training. Each successive of ways to evaluate training. Each successive evaluation level is built on information provided by evaluation level is built on information provided by the lower level. the lower level.
The Success Case Method (SCM) approach The Success Case Method (SCM) approach developed by Robert Brinkerhoff (2003) is a quick developed by Robert Brinkerhoff (2003) is a quick and simple process to evaluate training and and simple process to evaluate training and organizational change. organizational change.
Time & Looking at the Business NeedsTime & Looking at the Business Needs
Purpose of EvaluationPurpose of Evaluation
The purpose for evaluation is to determine the effectiveness of a training program. According to this model, evaluation should always begin with level one, and then, as time and budget allows, should move sequentially through levels two, three, and four. Information from each prior level serves as a base for the next level's evaluation
Situation Driven of Different PerspectivesSituation Driven of Different Perspectives
Level One Level One
Level TwoLevel Two
Level ThreeLevel Three
Level FourLevel Four
Reaction
Learning
Behavior
Results
Kirkpatrick (1998)
Kirkpatrick’s Kirkpatrick’s LevelsLevels of of EvaluationEvaluation
ReactionReaction what they thought and felt about the training
How well did participants like the training program? In other words, the participants' opinions about the training, processes, and results.
LearningLearning
What knowledge (principles, facts, and techniques) did the participants gain from the training program? That is, the degree to which learning actually took place.
the resulting increase in knowledge or capability
BehaviorBehavior
What positive changes in the participants' job performance could be attributed to the training program? This is the link between learning and workplace performance.
extent of behavior and capability improvement and implementation/application
ResultsResults
What was the impact (ROI) of the training program on the performance of the organization?
the effects on the business or environment resulting from the trainee's performance
Brinkerhoff's Success Case Method (SCM)Brinkerhoff's Success Case Method (SCM)
Brinkerhoff’s SCM is based on five steps: (1) focusing and Brinkerhoff’s SCM is based on five steps: (1) focusing and planning the study; (2) planning the study; (2) clearly defining what outcomes will clearly defining what outcomes will be considered "success"; (3) identifying success cases; be considered "success"; (3) identifying success cases; (4) conducting interviews to learn exactly how success (4) conducting interviews to learn exactly how success was achieved;was achieved; and (5) and (5) communicating results throughout communicating results throughout the organization.the organization.
Brinkerhoff used four key questions to gather information:Brinkerhoff used four key questions to gather information:
• What is happening?What is happening?
• What results are being achieved?What results are being achieved?
• What is the value of the results?What is the value of the results?
• How can it be improved?How can it be improved?
Brinkerhoff's Success Case Method (SCM)Brinkerhoff's Success Case Method (SCM)
Analysis of Case Study & Story Telling Analysis of Case Study & Story Telling
Brinkerhoff (2003)
Brinkerhoff's Success Case Method (SCM)Brinkerhoff's Success Case Method (SCM)
The Process & OutcomeThe Process & Outcome
Brinkerhoff (2003)
• Impact of Training Impact of Training
• ROIROI
• Unrealized ValueUnrealized Value
• Performance factors that align with ImpactPerformance factors that align with Impact
ReferencesReferences
Kirkpatrick, D. L. (1998). The four levels: An overview. Ch. 3 in Evaluating Training Programs: The Four Levels, 3nd ed. San Francisco: Berrett-Koehler.
Kirkpatrick, Donald L. (1998). Another Look at Evaluating Training Programs. American Society for Training & Development.
Kaufman, R., Keller, J., & Watkins, R. (1995). What works and what doesn't: Evaluation beyond Kirkpatrick. Performance & Instruction, 35, (2). 205-209.
Morrison, Gary R., Kemp, Jerrold E., & Ross, Steven M. (2001). Chapter 10, The Many Faces of Evaluation. In Designing Effective Instruction (3rd edition). New York: John Wiley & Sons.
Worthen, Blaine R, James R. Sanders, Jody L. Fitzpatrick (1997). Program Evaluation: Alternative Approaches and Practical Guidelines (Second Edition). Addison, Wesley, Longman, Inc.
Phillips, J. (1997). Handbook of Training Evaluation and Measurement Methods (Improving Human Performance Series) 3rd Edition. Butterworth-Heinemann. Chapters 1 - 3.
Sieloff, Debra A. (1999). The Bridge Evaluation Model. International Society for Performance Improvement.
Brinkerhoff, R. O. (2003). The Success Case Method. Berrett Koeler: San Francisco.
Brinkerhoff, R. O. & Dressler, D. (2002). Using evaluation to build organizational performance and learning capability: A strategy and a method. Performance Improvement.
Van Tiem, Darlene M., Moseley, James L., Dessinger, Joan Conway (2004). Chapter 7, Evaluation. In Fundamentals of Performance Technology: A Guide to Improving People, Process, and Performance, 2nd Edition. Washington, DC: International Society for Performance Improvement.
