EVALUATION OF EFFECTIVENESS OF TRAINING

HUMAN RESOURCE MANAGEMENT

Evaluation of effectiveness of training a continuous cycle consisting of defining

training objectives, carrying out training needs analysis, delivering training, assessing reactions to training, and measuring the bottom-line effects of training

Why evaluation?

To measure training is one of the key components on how a business will succeed

If a training program is proven to be effective, it will definitely yield to positive results

Training in itself is expensive and adding more components to it may not be a good idea in terms of financial capacity

to reduces cost and to improve the methodologies of the company or organization

How to measure?

Metrics should be used to see how effective the training was. Moreover, metrics should be checked regularly to see if see if they are at par with the expectations of the company

an in-depth analysis should be done performance alignment program have a weekly or monthly product

knowledge check balance Using Kirkpatrick model

The Kirkpatrick Model

Developed by Donald Kirkpatrick in the late 1950s

The basic structure of Kirkpatrick’s four-level model is:

Level 4: result

Level 3: behavior

Level 2: learning

Level 1: reaction

An evaluation at each level answers whether a fundamental requirement of the training program was met

Each level provides a diagnostic checkpoint for problems at the succeeding level.

The difficulty and cost of conducting an evaluation increases as you move up the levels. So, you will need to consider carefully what levels of evaluation you will conduct for which programs

Using Kirkpatrick model

Level 1 (Reaction)• completed participant feedback

questionnaire• informal comments from participants• focus group sessions with participants Level 2 (Learning)• pre- and post-test scores• on-the-job assessments• supervisor reports

Level 3 (Behavior)• completed self-assessment questionnaire• on-the-job observation• reports from customers, peers and

participant’s manager Level 4 (Results)• financial reports• quality inspections• interview with sales manager

TATA Power Company Recognized as India’s largest private sector

power utility the founder of the house, Jamshedji Nusserwanji

Tata started this venture with the establishment of a textile mill in Central India over a century ago and then consolidated by J.R.D. Tata

The core company is to generate, transmit and distribute electricity.

The company operates in two business segments: power and other.

Tata Power has a first of its kind joint venture with Power Grid Corporation of India for the 1200 km Tata Transmission Project.

The Strategic Electronics Division of Tata Power has been in operation for over 30 years

Leveraging upon its engineering skills and understanding of the power business, Tata Power has carried out several overseas projects

Tata Power is committed to setting high standards in its pursuit of social responsibility and remaining sensitive to the issues of resource conservation, environment protection and enrichment and development of local communities in its areas of operations

Company Profile

ChairmanR.N.Tata

Broad industryPublic industry

Managing DirectorF.A.Vandervala

Sub IndustryIntegrated Electric Utility

Executive Director Finance

S.R.Ramakrishnan

Previous NameTata Electreic

Companies

SecretaryB.J.Shroff

Installed Capacity2300MW

Employees3.400

SWOT analysis of the Company Strengths :• Harmonious management employee

relationships• Low employee turnover• Strong sense of TATA values at all levels• Internal trade union for decades• Competence in hydro/thermal/CC

generation and HT transmission and Distribution

Weaknesses:• Lack of performance based culture• Higher age of employees • Gaps in succession planning• Accountability:• Need to build leadership, customer

orientation, commercial and marketing competencies.

• Surplus manpower• Resistance to change

Opportunity:• Can expand in power fields with relative

ease due to availability of skilled technical manpower

• Opportunity for top performers to move across group companies

• Can adopt to change in competitive environment when the going is good

Threats:• Retention of young talent when

competition picks up• Competitor has an edge in retail

business talent• Low cost power generators

Work system at TATA Power:1. Recruitment and selection2. Performance management system3. Succession planning4. Leadership development plan5. Career progression path6. Employee training and development

process7. Star process

9. Employee retention practices

10. Rewards and recognition

11. Employee engagement process

12. Organizational transformation

13. Welfare programmes

Training @ TATA Power

Employee Education, Training and Development:

• Conducts training workshops to help its employees achieve still higher levels of competence and come out with the best in them

• Conducts programmes on safety, health and environment

• Training needs are revisited during quarterly/half yearly reviews to keep current with changing business requirements e.g. contract labor management

TATA management Training Centre:• TMTC has been positioned primarily as

in-house training centre for TATA group.• Its training programmes are designed to

develop leadership traits• It conducts programmes not only during

their induction and at the end of their probation period, but also during their early years with the service.

Training Process

TPC lays stress on building competencies. Learning initiatives are designed to improve technical competence, build new skills for growth areas and inculcate behavioral pattern that are required for business expansion.

Process: 1. Need identification 2. Development of the training program3. Deliver them program4. Measuring effectiveness of the program

ETD Process

TNI

Survey Feedbac

k

Division needs

Base Level

Needs

Individual Needs

Strategic Needs

ETD Design

Long Term

Short Term

Annual Plan

Quarterly Plan

Half Yearly Review

Training Delivery

QuarterlyReview

Monitoring Training

Effectiveness

ETD Process Review

New requirements(

org)Employe

e Skill Record

Employee feedback/sur

vey

Training process

Individual Training needs identification

Analysis and compilation of individual needs and prepare group

training need form

Compile training needs from all the departments and Prepare training

plan

Deliver / conduct the training as per the plan

Evaluate and record training effectiveness

Is training Effective

?

end

Initial suitable action

Select alternatives

Procedure

Sr focus

1. 40% focus on strategic objectives, low power cost, cost reduction, long term

distribution2. 40% focus on divisional/individual

performance

3. 10% on base level requirements

4. 5% on consistent high performers

5. 5% on Sr. Management

Evaluation of Training EffectivenessThe Kirkpatrick Model of Training

Evaluation: Level 1(Reaction):• Level 1 evaluation should not just include

reaction towards the overall programme, it should also include measurement of participant’s reaction towards the specific component of the program

• Evaluation relies on the measurement of the attitudes, usually through the set of questionnaires. as in:

1. In your view, what were the 3 most important weaknesses of the program?

2. In your view, what are the 3 most important strengths of the program?

Advantages:• It will help us to know the participants feel about

the training event.• It may point out content areas that trainees felt

were missing from the training event.• It can provide information about the overall

participants reaction as well as participants feedback and evaluation of specific aspects of training event

• Detailed level 1 can provide information that can be used to improve future versions of the training program

Level 2(Learning):• The goal is to determine what the training

program participants learned during the training event

• This should be done immediately after the training event to determine if participants gained knowledge, skills, or attitude. A couple of issues are here:

1. How shall one measure knowledge, skills and attitude?

2. What research design should be used to demonstrate important in level 2 outcomes?

Level 2 evaluation is done by conducting test and questionnaires.

Advantages:• Demonstrating participants learning helps

trainers in promoting their training program

• It helps in interpreting the result of level 3 evaluation

• Detailed level 1 can provide information that can be used to improve future versions of the training program

Level 3(Behaviour):• The goal is to find out if training participants

change their on-the-job-behaviour (OBJ) as a result of their having attended and participated in the training program.

• It specifically involves measuring the transfer of knowledge, skills, and attitudes from the training context to the workplace

• This level is often harder to evaluate because behavioral changes at workplace are often harder to measure than reaction and learning

Process adopted for level 3 at TATA is:1. Level 3 evaluation is done 3 months after the

training is imparted2. The program are short listed based on the

criteria3. Program specific questionnaires are then

prepared for short listed (keeping in mind the objectives and course content)

4. The questionnaires consists of 10 questions based on the initiatives taken by the participant to use the knowledge gained, the conceptual clarity and support provided by superiors and peers.

5. Questionnaires are made on a four scale, 1 being the least and 4 being the highest.

6. These questionnaires are sent to the participants and asked to respond

7. The effectiveness index for each program is calculated from the responses received

8. The effectiveness index for the month is the average of the indexes of all the programs considered for the month.

Level 4(Result):• The goal is to find out if the training program

led to final results, especially business results that contributing to the “bottom line”

• Level 4 outcomes can include other major results that contribute to the well functioning of an organization

• These outcomes are either changes in variables that should have a relatively direct effect on the financial outcomes at some point of time in future

The outcomes of level four are:• Improved quality of work• Higher productivity• Reduction in turnover• Improved quality of work life• Improved human relations (e.g. improved

vertical and horizontal communication)• Increased sales• Fewer grievances

• Lower absenteeism• Higher work morale• Fewer accidents• Greater job

satisfaction• Increased profit

Research Methodology

Primary data:It is collected mainly through questionnaires

which is sent to trainees Secondary data:Secondary data is collected mainly through

existing data available in the company record

Questionnaire type:An open-ended questionnaire is sent to

managing staff through e-mail. For every question there are 4 options-strongly agree, agree, disagree and strongly disagree

Sampling size:

A sample size of 25 employees are selected from each training programme conducted

Limitations• Difficulty in obtaining relevant materials from

the company• Reluctance on the part of the company to

give away their closely guarded policies of evaluating the training effectiveness

• Difficulty in getting back the response to the questionnaire from the trained employees regarding the program

Training programs conducted in July’05

SR NO.

PROGRAMMES

1. Problem Solving

2. Building Winning Teams

3. SAP MM Module

4. Financial Management for Electrical Distribution

5. Team Building

6. 7 Q C Tools

8. Sharpening Customer Focus

9. Enhancing Managerial Effectiveness

10. Team Building

11. VAT-SED Bangalore

12 Reduction in T & D Losses

13. Reactive Power Management

14. Develop yore own Potential

15. Excellence in Executing Change

16. Lead Auditors Course

Data Analysis & Effectiveness The training programmes, which were conducted during the months

of July, august and September 2005, are taken for evaluation. TRAINING EFFECTIVE QUESTIONNAIRE (DATE:

…………)

Program Name: SAP MM Module

Name: ……………….. Division:…………………….. Grade:…………….

Dear Respondent,

Please mark your responses to the following questions as given below:

Attribute Rating

Strongly disagree 1

Disagree 2

Agree 3

Strongly agree 4

ITEM RATING

The program helped me understand the concept of SAP MM Module

The program increased ma awareness about the SAP MM Module

I shared the knowledge I gained with my other colleagues

The program helped me perform better at work

The program taught me the ways & means through which the work can be done

My superiors and colleagues have acknowledged my improved knowledge in the field

I believe I have been able to take more effective decisions after undergoing the course

I could apply all the knowledge I gained

Conclusion

Training is conducted very effectively at TATA power

Training effectiveness index up to September ‘05 82%.

Training is successfully selected and ensures that it meets the needs of TATA power

PRESENTED BY:DEEPALI CHOUDHARY

DFT-IIINIFT GANDHINAGAR

THANK YOU

Systemanalyse programmentwicklungGerman for system analysis and program development benefit is It standardizes the process and support across a company