+ All Categories
Home > Business > Training-How to Evaluate

Training-How to Evaluate

Date post: 15-Jan-2015
Category:
Upload: humantalents
View: 720 times
Download: 1 times
Share this document with a friend
Description:
 
Popular Tags:
19
Training-How to Evaluate? by Jayadeva de Silva M.Sc, FIPM, FITD Training funds are shrinking in the organizations as the top management disapprove more often and more stridently that they have not seen any payback from having their staff away on training, not worth parting employees outside office for one or two days. Many training programs today fail to deliver the expected organizational benefits. The conventional training evaluating structure in 1
Transcript
Page 1: Training-How to Evaluate

Training-How to Evaluate? by

Jayadeva de Silva M.Sc, FIPM, FITD

Training funds are shrinking in the organizations as the top management disapprove more often and more stridently that they have not seen any payback from having their staff away on training, not worth parting employees outside office for one or two days.

Many training programs today fail to deliver the expected organizational benefits. The conventional training evaluating structure in place helped determine where the trouble lies. On an optimistic note, being able to display a genuine and significant benefit to your business from the training you provide can help you gain more resources from important decision-makers. The business environment is not at a standstill. Your competitors,

1

Page 2: Training-How to Evaluate

technology, Government legislation and set of laws are constantly changing. External pressures may be leading you to think about improving your current programs. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.

Measuring the efficacy of training programs and modules, however, consumes valuable time and resources – time and resources that are already in short supply. One needs to consider cautiously about how and to what level you will appraise the results of training. The standard method is of Donald Kirkpatrick’s four-level evaluation model that remains as the most renowned and used model these days.

Kirkpatrick developed this model in the late 1950s and the model has since been modified and made to order by various researchers. However, the model’s basic constitution is fighting fit the test of time and many HR Guru’s recommend it.

Before we move on to discuss that model we should remember that there are two types of Evaluations or Assessments as follows

Formative EvaluationFormative evaluation (also known as internal) is a method of judging the worth of a program while the program activities are forming (in progress). This part of the evaluation focuses on the process.

2

Page 3: Training-How to Evaluate

SummativeThe summative evaluation (also know as external) is a method of judging the worth of a program at the end of the program activities (summation). The focus is on the outcome.

The Ten Factors of Developing a Training Program

1. Determine needs2. Set objectives3. Determine subject content4. Select qualified applicants5. Determine the best schedule6. Select appropriate facilities7. Select qualified instructors8. Select and prepare audiovisual aids9. Co-ordinate the program10. Evaluate the programme

Evaluating

“The reason for evaluating is to determine the effectiveness of a training program.” (Kirkpatrick, 1994,)

Reasons for EvaluatingKirkpatrick gives three reasons ‘why’ there is a need to evaluate training:1. “To justify the existence of the training department by showing how it contributes to the organizations’ objectives and goals.”

3

Page 4: Training-How to Evaluate

2. “To decide whether to continue or discontinue training programs.”

3. “To gain information on how to improve future training programs.” (Kirkpatrick, 1994,)

Kirkpatrick: Why Evaluate Training Programs

“What is quality training?”

“How do you measure it?”

“How do you improve it?”

Who is involved in 'Training Evaluation?

-Senior management -The trainers -Line management -The training manager -The trainee

Levels of Training evaluation

Tier-1(Level one): how closely does Training meet the expectations of trainees?

-Feedback score collected from each trainee at end of the class-Record general impression about the Training on a numerical scale of 1-5 or similar-Average score reflects how closely training met expectations of field and how close it is aligned with

4

Page 5: Training-How to Evaluate

business needs identified earlier which drives the training.

Level 1 is called Reactions Level

Reaction may best be defined as how well the trainees liked a particular training program." Reactions are typically measured at the end of training.

When Verifying of Reactions You must remember- These are typically 'happy sheets' -Feedback forms are based on subjective personal reaction to the training experience -Verbal reactions also can be noted and analyzed -Post-training surveys or questionnaires can be used-Subsequent verbal or written reports given by delegates to managers back at their jobs is also possible

Tier-2(Level 2): how well Training improved skills of trainees?

5

Page 6: Training-How to Evaluate

-Collect Pre-training and post-training data on trainee’s skills before and after the training. A well drafted skill based survey can be used. -Compare improvements in exposure or expertise gained by the trainee as seen on post-training survey by comparing it with pre-training survey. Conduct in-training skill tests as part of the qualification criteria to measure the improvement in the learned skills.

Level 2 is also called Learning.

-What principles, facts, and techniques were understood and absorbed by the participants?" -What the trainees know or can do can be measured during and at the end of training

When verifying Learning remember to look at the following

WHAT?

-What knowledge was acquired? -What skills were developed or enhanced? -What attitudes were changed?

HOW is this done?

-Tests before and after the training -Interview or observation can be used before and after training.-Measurement and analysis is possible and easy on a group scale

6

Page 7: Training-How to Evaluate

-Reliable, clear scoring and measurements need to be established

Tier-3(Level 3): How effective does the training prove in changing On-job-behavior and skills of trainee?

-Collect quarterly feedback from trainee’s manager on his observations on improvement of trainee’s on-job performance/ behavior or skills after attending training. -Analyze the trends in key on-job-performance parameters or indicators as seen on survey with respect to previous quarters.-Convert delta into normalized scores to indicate the value created by training in the work efficiency of the trainee-Ensure continual measurement of effectiveness after regular intervals to assess long term value of the training.

7

Page 8: Training-How to Evaluate

Level 3 is also called -Behavior.

-Changes in on-the-job behavior, Behavior changes are acquired in training and they then transfer (or don't transfer) to the work place. -What skills did the learner develop, that is, what new information is the learner using on the job?

When evaluating change in behavior, decide:

-When to evaluate-How often to evaluate-How to evaluate

When Verifying the Behaviour, remember to assess

WHAT?

-Whether the trainee is able to transfer the learning to the work environment-New learning is demonstrated- Whether the trainee is motivated

HOW is this done?

-self-assessment can be useful, using carefully designed criteria and measurements -cooperation and skill of observers, typically line-managers, are important factors, and difficult to control-Use of focus groups

8

Page 9: Training-How to Evaluate

Tier-4(Level 4): To what extent training does impact in improving business / revenue or service parameters specific to trainees or group of trainees?

-Measure quarterly the business indicators of the trainee’s job or service parameters based on nature of job. Business indicators could be collected based on individual job or group responsible for the said function.-Ideally business indicator data before the training should be used as baseline. -Record the business parameters or governing service parameters on quarterly basis-Calculate qualitative or quantitative impact factor based on normalized delta.

Level 4 is also called -Results.

-Reduction of costs; -Reduction of turnover and absenteeism; -Reduction of grievances;- Increase in quality and quantity or production;

9

Page 10: Training-How to Evaluate

Or Improved morale which, it is hoped, will lead to some of the previously stated results.-These factors are also measurable in the workplace

Thus the Four Levels of Training Evaluation are

Reaction Learning Behavior Results

“The Four Levels represent a sequence of ways to evaluate (training) programs….As you move from

10

Page 11: Training-How to Evaluate

one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information.” (Kirkpatrick, 1994,)

When all conditions are met, the employee must:

-Realize an opportunity to use the behavioral changes.-Make the decision to use the behavioral changes.-Decide whether or not to continue using the behavioral changes.

Kirkpatrick identifies the following four conditions for changes to occur:

-Desire to change-Knowledge of what to do and how to do it-Work in the right climate-Reward for (positive) change

11

Page 12: Training-How to Evaluate

Summary of Evaluation methodologies

Evaluation Level

Data Collection Procedure

Criteria and Analysis

Reaction ‘Smile’ sheets 75% of students satisfied 93% average positive ratings

Learning Acquisition

Tests Observation checklists Work sample analysis

Before-after tests 80% mean test scores 4.0 improvement pre- to post-test

Behavioural Intention

Action plans Plans formulated

Work Behaviour

Observation checklists Work sample analysis Superior - peer - subordinate reports

Achievement of training objectives 80% reported improvement Comparison of trainees and non-trainees Cohort analysis of trainees entering the program at different times Trainees’ performance at different locations Longitudinal analysis of trainees and non-participants’ performance, career path, etc

Changes in Others

Superior - peer - subordinate reports - interviews - surveys

80% reported changes

Organizational change

Organizational reviews - interviews - surveys - focus groups

80% reported changes

Performance

Monitoring – information systems: Outputs - workload Productivity - efficiency Outcomes - effectiveness Cost of outcomes Effectiveness surveys

Benchmarks - standards - targets Comparisons Quality analysis Utility analysis Customer satisfaction 80% Telephone response in 3 minutes 12

Page 13: Training-How to Evaluate

An individual /Group exercise Group 1- As an HR Manager, how will you evaluate:- The output effectiveness of a supervisory training programme?

Group 2- The Outputs to be evaluated after a Sales Training Programme

Group 3- The Outputs to be evaluated after a Customer Relations Training Programme

13

Page 14: Training-How to Evaluate

Jayadeva de Silva

Jayadeva de Silva obtained Master’s degree in Science from Russian Friendship University Moscow and Diploma in Personnel Management from National Institute of Business Management( SriLanka). He is a fellow of both the Institute of Personnel Management IPM) and Institute of Training & development.(SLITAD) He is also professionally qualified in training systems & curriculum design with an ILO fellowship. A strong advocate of Human Talents Development, Jayadeva is the Principal Consultant/Director of Humantalents Unlimited, a professional practice that provides training & consultancy in Management. Jayadeva has carried out several consultancy projects and conducted numerous management development programs/strategic planning workshops, and has been trained in many modern management and leadership concepts. He has contributed articles (over 50) and authored the trend setting book ‘Human Talents Management’. He founded humantalents International and HRSriLanka virtual learning Groups. Jayadeva de Silva functioned as Group Manager (Human Resources Development) of Hayleys Group of Companies and Group Director–Human Resources of Brown & Co. He serves as a resource person for professional and post graduate courses primarily in the areas of HRM & HRD at Post graduate Institute- University of Peradeniya, University of Ruhuna, ICFAI University (India), University of Ballarat (Australia) SriLanka Foundation Institute IPM & SLITAD.He is a past president of HRDGateway, an International organization of over 45,000 HR Professionals worldwide. He is featured in the millennium registry of SriLankan personalities

He can be contacted as follows

E mail [email protected] 011 2562449 077 7272295 Web [email protected]

Some of his publications are available for fee download from www.slideshare.net/Jayadeva

14


Recommended