Does learning solve business performance problems?

By Javier Amaro

What is value?

• Think about a training program that you recently implemented.

• How would you define the value delivered by the program?

What is the value of training?

New definition of value

Be balanced, with qualitative and quantitative data

Contain financial and nonfinancial perspectives

Reflect strategic and tactical issues

Satisfy all key stakeholders

Be consistent in collection and analysis

Be grounded in conservative standards

Come from credible sources

Do we have a place in the table?

Do we have a place in the table?

• Do training and development professionals participate in business decisions?

• Do training and development professionals participate in government strategic decisions?

Shifting Paradigms

Activity Based Results Based No business need for the program Program linked to specific business needs

No assessment of performance issues Assessment of performance effectiveness

No specific measurable objectives Specific objectives for behavior and business impact

No effort to prepare program participants to achieve results

Results expectations communicated to participants

No effort to prepare the work environment to support transfer

Environment prepared to support transfer

No efforts to build partnerships with key managers

Partnerships established with key managers and clients

No measurement of results or benefit-cost analysis

Measurement of results and benefit-cost analysis

Planning and reporting is input focused Planning and reporting is outcome focused

Training Solutions Provider?

Become a business partner

Levels of Evaluation

10

Levels of Evaluation Measurement Focus

0. Inputs and Indicators The input into the project in terms of

scope, volume, efficiencies, costsParticipants, Hours, Costs, Timing

1. Reaction & Perceived

Value

Reaction to the project or program,

including the perceived value

Relevance, Importance, Usefulness,

Appropriateness, Intent to use,

Motivation to take action

2. Learning

Learning to use the content and

materials, including the confidence to

use what was learned

Skills, Knowledge, Capacity,

Competencies, Confidence, Contacts

3. Application &

Implementation

Use of content and materials in the work

environment, including progress with

actual items and implementation

Extent of use, Task completion,

Frequency of use, Actions completed,

Success with use, Barriers to use,

Enablers to use

4. Business Impact

The consequences of the use of the

content and materials expressed as

business impact measures

Productivity, Revenue, Quality, Time,

Efficiency, Customer Satisfaction,

Employee Engagement

Typical Measures

5. ROIComparison of monetary benefits from

program to program costs

Benefit-Cost Ratio (BCR), ROI%,

Payback Period

Executive view – the report

MeasureWe currently measure this

We should measure this in

the future

My ranking of the importance of this

measure

1. Inputs: “Last year, 78,000 employees received formal learning.” 94%

2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.”

78%

3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.”

53%

4. Learning: “92% of participants increased knowledge and skills” 32%

5. Application: “At least 78% of employees are using the skills on the job.”

11%

6. Impact: “Our programs are driving our top 5 business measures in the organizations.”

8%

7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.”

4%

8. Awards: “Our learning and development program won an award from American Society of Training and Development.”

40%

Executive view – the need

MeasureWe currently measure this

We should measure this in

the future

My ranking of the importance of this

measure

1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 85%

2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.”

78% 82%

3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.”

53% 22%

4. Learning: “92% of participants increased knowledge and skills” 32% 28%

5. Application: “At least 78% of employees are using the skills on the job.”

11% 61%

6. Impact: “Our programs are driving our top 5 business measures in the organizations.”

8% 96%

7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.”

4% 74%

8. Awards: “Our learning and development program won an award from American Society of Training and Development.”

40% 44%

Executive view – the gap

MeasureWe currently measure this

We should measure this in

the future

My ranking of the importance of this

measure

1. Inputs: “Last year, 78,000 employees received formal learning.” 94% 86% 6

2. Efficiency: “Formal learning costs $2.15 per hour of learning consumed.”

78% 82% 7

3. Reaction: “Employees rated our training very high, averaging 4.2 out of 5.”

53% 22% 8

4. Learning: “92% of participants increased knowledge and skills” 32% 28% 5

5. Application: “At least 78% of employees are using the skills on the job.”

11% 61% 4

6. Impact: “Our programs are driving our top 5 business measures in the organizations.”

8% 96% 1

7. ROI: “Five ROI studies were conducted on major programs yielding an average of 68% ROI.”

4% 74% 2

8. Awards: “Our learning and development program won an award from American Society of Training and Development.”

40% 44% 3

Fear of negative ROI

Chain of Impact

Reaction & Planned Action

Learning

Application & Implementation

Isolate the Effects of the Program

Intangible Benefits

Impact

Program Alignment: The V Model

Matching evaluation levels

Improve work group productivity by 20% in 6 months 4

Initiate at least three cost reduction projects in 15 days 3

Achieve an average cost reduction of $20,000 per project 4

Use counseling skills in 90% of situations where work habits are unacceptable 3

Be able to describe the four elements of the employee assistance program 2

Provide a 4 out of 5 rating on relevance of course content 1

Achieve a 2:1 benefit-to-cost ratio one year after the new leadership program is implemented

5

For each objective listed below, indicate the level of evaluation at which the objective is aimed.1. Reaction and planned action2. Learning3. Application4. Impact5. Return on Investment (ROI)

Program Alignment V Model

Decrease absenteeism by 5%

Key Alignment Questions

Potential ROI ROIPayoffs Objectives5 5

Is this a problem worth solving?

Is there a potential pay off?

What is the actual ROI?

What is the BCR?

Needs ProgramAssessment Objectives Evaluation

Needs ProgramAssessment Objectives Evaluation

Business Impact BusinessNeeds Objectives Impact

Key Alignment Questions

4 4

What is the specific measure?

What happens if we do nothing?

Which business measure improved?

How much is related to the program?

Needs ProgramAssessment Objectives Evaluation

Job Performance Application ApplicationNeeds Objectives

Key Alignment Questions

3 3

What is occurring or not occurring on the job that influences the business measure?

What has changed?

Which skills/knowledge has been applied?

Skills/Knowledge Learning LearningNeeds Objectives

Needs ProgramAssessment Objectives Evaluation

Key Alignment Questions

2 2

What skills or knowledge is needed to support the job performance need?

What did they learn?

What are the takeaways?

Preferences Reaction Reaction &Objectives Planned Action

Needs ProgramAssessment Objectives Evaluation

Key Alignment Questions

1 1

How should the solution be structured?

What was the reaction to the program?

How will the program be implemented?

Business Alignment process

1. Determine payoff needs

2. Identify business needs

3. Identify performance needs

4. Identify learning needs

5. Set Objectives

6. Design for results

7. Measure the business impact

8. Isolate the effects of the program

Training with the end in mind

• What is the value of your training?

– ROI

– Impact

– Application

– Learning

– Reaction

– Input

• Report the whole picture

Build up Capabilities in ROI

• ROI Certification program

– Sydney from 7th to 11th December 2015

– http://insources.com.au/index.php/roi/roi-certification

Feedback Counts!

Your feedback helps ATD 2015 APC continue to provide top-notch educational programs that help you stay on top of a changing profession.

Evaluation forms for this session are available via the mobile app and at the following link: https://zh.surveymonkey.com/r/ATD2015APC