Evaluating of Training – HR Training and Development Strategy

Welingkar’s Distance Learning Division

CHAPTER-06

Evaluating of Training

We Learn – A Continuous Learning Forum


Introduction

Training requires time , energy and money. Therefore an organization needs to know whether the investment made in training is being effectively and is worth the effort. Top management is concerned with evaluation as a process by which the effectiveness of the organizations programs and operating procedures are demonstrated. Supervisors are concerned more with the “results” of training as compared by changes in workers on the job performance.

Contd…We Learn – A Continuous Learning Forum


Evaluation is a planned process which provides specific information about a selected session, program for the purpose of determining value and or decision making. Related to training evaluation is concerned with providing information on the effectiveness of the training activity to decision-makers who will make decisions based on the information. There are various models that have been developed to describe the role of evaluation in the training process. It is important that evaluation be a planned or systematic effort that is built-in from the start of the training process.



Ascertaining Reactions Of the Participants

As the programme gets over, it is important to ascertain the participants views and observations about its various components both in terms of the contents and the training process. The evaluation can either focus on a session-by-session review or on the programme as a whole. The formal format that the trainers choose to adopt for eliciting end-term appraisal of the participants’ ranges from the relatively simple “highlight the best and worst aspects” to a detailed response through a properly devised questionnaire or instrument.



Areas for Ascertaining Reactions of the ParticipantsAn important decision that the training team takes prior to the programme is about the specific elements it would like to include in the evaluation exercise.

Course Objective• How far the programme objectives are realistic.• Clarity• Individual understanding and response• How far the training objectives are realized.

Contd.We Learn – A Continuous Learning Forum


Trainers Performance• Effectiveness of the presentations.• Skill of the trainers to use the training methods• Trainers expertise in responding to the questions of the

participants.• Whether the trainers attitude was supportive and

helpful.• Their relationship with the training group.

Training Methods• Their appropriateness tot the programme and the

modules.• The extent to which they facilitated learning. Contd.



Training Group• Its size and composition• Selection procedure• Level of the group and quality of its participation.

Time Schedule• Duration of the programme• Allocation of time to each module• Was the daily schedule hectic.• Sequencing the topics• General flow and momentum of the programme



Training Facilities• Suitability of the sitting arrangement • General ambience• Appropriateness and quality of the equipment used• General learning environment at the venue.

Physical Arrangements• Appropriateness of the accommodation• Arrangement for food etc.

Training Support Materials• Usefulness and quality of the materials• Their timely distribution. Contd.



Sources for Ascertaining Participants’ Reactions

• Structured Questionnaire• Tests• Open Forum• Personal interviews• Program committee meetings for ongoing evaluation

of the contents and process of the programme.• Observations of the trainers• Comments of the participation



Purposes and Uses of Evaluation

The definition of evaluation implies 2 purposes:

1. Making decisions about improvements to be made in the training program itself.

2. Making decisions about the value of the training program in terms of whether to continue to conduct the program.



Specifically, evaluation can be used:

a) To determine whether the training program is accomplishing its assigned objectives and if they were the “right” objectives.

b) To Identify the strengths and weaknesses of training activities.

c) To determine the cost/Benefit Ratio of the Training Program.d) To Establish a Data base which Organization Leaders can use

to Demonstrate the Productivity and Efficiency of their operational Procedures.

e) To Establish a Data Base which can assist Organization Managers in Making Decision.



Evaluation of Training

After the training program has been completed and the current training record filled in, the performance of trainees can be evaluated and the attention of the training staff drawn towards specific action for improvement. Thus evaluation of training should help to:• Isolate areas of difficulty and suggest strategies for

overcoming them.• Modify unrealistic training targets in the light of

actual group performance Contd.We Learn – A Continuous Learning Forum


• Determine whether motivation of trainees is of the required level e.g. financial incentives for greater output during training.

• Compare initial selection to actual performance rating

• Highlight causes of absenteeism and labour turnover during training

• Evaluate and modify instruction for future training programme

• Formalized management training programme have a vital role to play in the development of management skills. Contd.



Evaluation can be four-pronged attempt.1. Evaluation of reaction 2. Evaluation learning3. Evaluation behavior 4. Evaluation results

Implementation of Kirkpatrick-Phillips ModelLevel 1 in which evaluation is used by most organizations through questionnairesLevel 2 evaluations can be accomplished by written or oral testsLevel 3 is ApplicationLevel 4 is tying training to organizational impactLevel 5 A new approach, measuring returns on investment



Measuring Training Effectiveness and Impact

I. Prior to training• The number of people that say they need it during the needs

assessment process.• The number of people that sign up for it.

II. At the end of training• The number of people that attend the session.• The number of people that paid and attended.• Customer satisfaction at the end of training• A measurable change in knowledge• Ability to solve a “mock” problem at the end of training• Willingness to try to use the skill at the end of training. Contd.



III. Delayed Impact (non-job)• Customer satisfaction at X weeks after the end of

training• Customer satisfaction at X weeks after the training when

customers know the actual costs of training• Retention of knowledge at X weeks after the end of

training.• Ability to solve a “mock” problem at X weeks after the

end of training.• Willingness to try the skill at X weeks after the training.• The three systems followed are: 1. 360 degree 2. Performance DNA system 3. Janus performance

Management SystemWe Learn – A Continuous Learning Forum


IV. On the job behavior change• Trained individuals that self-report that they

changed their behavior , used the skill on the job after the training.

• Trained individuals who’s managers report that they changed their behavior or used the skill after training.

• Trained individuals that actually are observed to change their behavior or use the skill after the training.



V. On the job performance change• Trained individuals who self-report that their actual

job performance changed as a result of their changed behaviour, skill.

• Trained individuals who’s manager’s report that their actual job performance changed as a result of their changed behaviour/ skill.

• Trained individuals whose manager’s report that their job performance changed either through improved performance appraisal scores or specific notations about the training on the performance appraisal forms. Contd.



• Trained individuals that have observable / measurable improvement in their actual job performance as a result of their changed behaviour.

• The performance of employees that are managed by individuals that went through the training.

• Departmental performance in departments with X % of employees that went through training ROI of return on training dollar spent.

Contd.



(a) Other measures• CEO / top management knowledge of/ approval of/

or satisfaction with the training program.• Rank of training seminar in forced ranking by

managers of what factors contributed most to productivity/ profitability improvement.

• Number of referrals to the training by those who have previously attended the training

• Additional number of people who were trained by those who have previously attended the training.

• Popularity of the program compared to others.Contd.



(b) What to evaluateOne way to approach the issue of what to evaluate is to identify what kind of information is needed. Dr. Kirtpatrick developed four levels of evaluation. Reaction1. Learning2. Behaviour3. Results

4. ReactionReaction is what a trainee thought of a particular program. There are some guidelines for determining participant reaction. Contd.



1. Design a questionnaire based on information obtained during the need assessment phase.

2. Design the instrument so that the response can be tabulated and quantified.

3. To obtain more honest opinions provide for the anonymity of the participants.

4. Provide space for opinions about items that are not covered in the questionnaire.

5. Pretest the questionnaire n a sample of participants to determine its completeness.



2. LearningThe trainer is concerned with measuring the learning of principles, facts, techniques and attitudes that were specified as training objectives. There are many different measures of learning performance including paper-and-pencil test, learning curves etc. The objectives determine the choice of the most appropriate measure. There are several guideposts used in establishing a procedure for measuring the amount of learning that takes place. Contd.



1. The learning of each participant should be measured so that quantitative results can be determined.

2. A before-and-after approach should be use so that learning can be related to the program.

3. Learning should be measured in an objective basis.4. A group not receiving training should be compared

with the group receiving training.5. Where possible the evaluation results should be

analyzed statistically. Where principles and facts are taught rather than skills, it is more difficult to evaluate learning. Contd.



3. BehaviourThe term is used in reference to the measurement of job performance. There are several guideposts in evaluating training programs in terms of behaviour changes.1. A systematic appraisal should be made on-the-job

performance on a before-and-after basis.2. The appraisal of performance should be made by

one or more of the following groups: a. The person receiving the training. b. The persons supervisor. c. The persons subordinates. d. The persons peers. Contd.



3. A statistical analysis should be made to compare the performance before and after and to relate changes to the training program.

4. The post training appraisal should be made three months or more after the training so that trainees have an opportunity to put into practice what they have learnt.

5. A group not receiving the training should be used for comparison.

Contd.



4. ResultsEvaluations in this level are used to relate the results of the training program to organizational objectives. Some of the results that could be examined are costs, turnover, grievances and morale. Where the objectives of the training program are closely tied to specific organizational objectives effort should be made to show a link between the training and the changes called for in the organization objective.



The process of Evaluation---Measuring Change

One way of visualizing a process of evaluation is to set the four levels of evaluation against a backdrop of a training program. Evaluation should start before the training program is designed. The next step is to plan the evaluation, how is the data to be gathered. Who will perform the evaluation? What techniques will be used? When will the evaluation be conducted? The evaluation design needs to be used on the objectives of the program and the criteria to be measured. Once the plan is completed and the training program begins, the data collection phase is initiated.



Validity and Reliability

Validity is a concept that means the degree to which an evaluation technique or instrument measures what it was intended to measure. Its a measure of accuracy.

Reliability is the degree to which evaluation techniques and instruments measure a given characteristic consistently. In order to get a long term view of a training programs effectiveness, an instrument must measure reliably for each session of the training program.



Types of Evaluation Techniques and Instruments

The process of Training EvaluationA training programme, like design, development, and manufacture of a product passes through several stages.1. Pre-Training EvaluationThis is prior to the course and should cover an analyses of the expectations of the trainee and his superior officer. The existing level of knowledge and skills are reassessed. It is carried out through discussions, workshops etc. Contd.



2. Input and Delivery EvaluationThis is done concurrent to the training. It involves long term management programme. It is done individually or in group. Each topic, module is evaluated in terms of its content, presentation, relevance and applicability. Using a questionnaire, soon after the course, invariably on the last day the participants reaction is obtained which generally elicits information about the course inputs and the impressions about the course.



3. Post Training EvaluationThe objectives of training is to enhance individual effectiveness which helps in improving organizational performance. Thus the process of post-evaluation must have reference to job improvement plan. This cannot be done when the person is training , he needs to go back to his job and demonstrate his newly acquired knowledge and skills. After a lapse of 6-12 months the results obtained are used for evaluating the applicability of training.



Evaluation Of The Training Institutes

It is believed that organization climate is known to contribute or hinder the transfers of learning to the job, the training centre itself is expected to contribute towards acquiring the required knowledge and skills by the trainees. Its climate, facilities faculty resources, library are some factors which would differentiate one centre from the other.The StudyTo study the effectiveness of the training of the in-house training Institutes in public enterprises detailed investigations using structured, protested questionnaires are undertaken. Contd.



Three sets of questionnaires:1. For the trainees2. To the superior officers of trainees and3. To the training centers are prepared and

administered.These questionnaires contain information such as relevance of the course inputs, extent of its fulfillment, perceptions of organizations climate for effective transfers of learning to the job, possible hindrances, positive as well as negative factors for training in the organization. The variables identified as relevant are 1. organizational variable. 2. Behavioral variables



Transfer of Learning

Individual GainsTo evaluate the effect of training on the individual after 6-12 months, the training persons stated the significant areas where the individual development had taken place. They were: 1. State of art knowledge on the subject2. Development of the analytical skills. 3. Refined interpersonal skills. 4. Innovation/ creativity. 5. Management of stress.



Transfer of learning to the organization

To evaluate whether in-house training has resulted in enhanced organization performance, trainees were requested to indicate the resultant benefits to public enterprises. The non-tangible benefits are: 1. Better management practices. 2. improved problem solving. 3. job improvement plans. 4. better systems.There is general evidence that systematic in-house training has contributed to individual growth in public enterprises, while the same level of organization improvement has not taken place because of organization climate, PE systems and Lack of motivation in PEs



Alternate approach in post-course evaluations

New approaches to post-course evaluation are being explored. One such recent development is monitoring the progress of the participant through post-course project work. In this approach participants at the time of nomination to the course in consultation with their departmental heads identify a proble, of practical value to the organization. During the program participants discuss, reformulate and identify approaches to its solution in the light of the knowledge and skill inputs in the training sessions. This way they related the learning to work situations and made innovating solutions.



Evaluation of in-house training institutes

In order to make training a continuous activity addressing itself to specific training needs of the organization, public enterprises have established large number of in-house training institutes/ Centers.There are 5 factors considered important for evaluating the institute and methods of assigning point rating to each.a) Faculty Resources: The number of qualified faculty members on full time basis at the centre, years of experience in training are considered. Contd.



b) Infrastructure: This covers all infrastructure facilities other than library, such as class rooms, syndicate rooms, hostel, recreation facilities etc.c) Library: This is the “brain” of any training institution/ Centre. Relevant books, journal, cases, audio, video etc.d) In-house Research / Consultancy: The in-house centers suffer from under exposure to research development and consultancy work. Such research/consultancy may be both internal or external.e) Financial Support: The annual budget allocation, unit cost per training are items under this factor.


Date post:	20-May-2015
Category:	Education
Upload:	we-learn-a-continuous-learning-forum-from-welingkars-distance-learning-program
View:	3,859 times
Download:	0 times

Evaluating of Training – HR Training and Development Strategy

Education