Date post: | 26-Dec-2015 |
Category: |
Documents |
Upload: | opal-chambers |
View: | 213 times |
Download: | 0 times |
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation
Learning Effectiveness Measurements
Lauge SorensenFaye WanklingJune 17, 2004
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation2
Agenda
IBM Learning Learning Life Cycle Performance Value Chain Relationship Discussion
How do you make the linkage between learning and business results?
What gets in the way of doing effective business impact measurements?
How do you use Learning Effectiveness Measures in your organization?
Our Continued Challenges & Lessons Learned Learning Measurements: The Positive Way Questions
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation3
IBM Learning Process
People Level Requirements: Audience segmentation: current/desired behaviorDrivers (Measurements, skills, values, tools/info)
Business Level Requirements: Business problem statement
Detail level solution design: Offerings within cluster
Deployment, Delivery and Execution
Development of Offerings within cluster
Deployment & Delivery
Requirements
Design & Development
AgreementCollaboration
AgreementCollaboration
High level solution design: Total cost of learning, cluster design
Business AnalysisBusiness Impact Objectives
Gap AnalysisRoot Cause Analysis
Solution IdentificationSolution DevelopmentTotal Cost of LearningMeasurement Design
Solution ImplementationSolution Measurement
Continuous ImprovementNew Requirements, Enhanced Design
Measurem
ents
Measurements
Measurem
ents
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation4
Learning Development Cycle & Performance Value Chain
Business Impact
Behavioral Impact
Learning Gain
Readiness
Phase 6:
MEASUREMENTS &REPORTING
Phase 5: DELIVERY
Phase 4: DEVELOPMENT
Phase 1: BUSINESS PROBLEM TARGETING BUSINESS RESULTS
Phase 2: REQUIREMENTS
Phase 3: DESIGN
EVALUATE
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation5
Performance Value Chain Relationship
Learners’ intention, readiness, expectations, career aspirations, etc.
Design DeliveryManagement
Support
LearningSatisfaction
LearningGain
LearningApplied
Job Performance
Business Impact/ROI(e.g. Revenue, Employee Satisfaction, etc.)
LearningEffectiveness
BusinessImpact
LearningSupport
Readiness &Participation
L1/2/3+
L4+
L4/5+
L1+
L1+
Business Impact
Behavioral Impact
Learning Gain
Readiness
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation6
In your organization how do you make the linkage between learning and business results?
Honestly - We don’t… “Not very well” “We just know if we do some education or training, it’s bound to have results” “We try but we haven’t really been able to connect business results to learning –
at least not ONLY to learning – so many other factors contribute”We know what we need to do – but aren’t doing it… “Behavior is often viewed as results so a deep understanding of employees
environment (processes, tools, etc.) and skills/gaps helps us get there” “We link learning with talent management (job placement, etc.)” “We must tell the full story at each level – we must know that some learning is
being applied before we can then attribute to business results” “It is challenging and just because it is difficult to measure the impact it doesn’t
mean there isn’t a real linkage to driving lasting behavior change and ultimate business results”
“Performance = Results therefore we need to pursue gaps in results; every training request should be tied to a measurable gap in results”
“We need to move up from only job level performance because seldom will this alone impact organizational results”
We try – but we are not sure what method to use….. “We use Performance Value Chains” “We use Impact Maps” “We do Causal Analysis”
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation7
What Gets in the Way of Doing Effective Business Impact Measurements
It’s complicated… “ROI seems hard/complex” “Reaction data makes us feel good (“this was a great class”) – why get more” “Data collection is difficult, data is inconsistent, systems don’t support data
collection in a consistent way; Internal systems don’t support” “Difficult to effectively isolate the impact variable” “We measure what is easily measured instead of what is really important to
measure”We don’t have a process… “Failure to collect baseline data so nothing to compare to” “We don’t start off solving a business problem so measuring a business result
may be impossible” “Lack of standards – each developer thinks their offering is unique” “We often fall short of defining true business impact measurements and instead
look for evidence of some application of skills – but forget to ask ourselves “so what””
“We prefer to focus on achievement and forget about the power of creating an environment of continuous improvement”
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation8
What Gets in the Way of Doing Effective Business Impact Measurements
There’s Enterprise/Organizational barriers… “There are geography infrastructure/process inconsistencies making
overall global consistency difficult” “Deployment of business impact measurements is a major
organizational change” “Privacy issues; legal inhibitors both real and imagined”Unclear priorities…. “Business Impact Measurements drive increased accountability for
Learning organizations and business stakeholders – who wants that?!”
“Pressure to show results prevents us from taking time up front and at the back end of an intervention (front end to get started, back end to implement and get on with the race)”
“Unrealistic expectations of what can or can’t be measured” “Inability to uncover the root causes of the gap in results so we don’t
have a concrete business result to measure” “Lack of commitment from learning teams – especially to take time
at the front end”
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation9
How do you use Learning Effectiveness Measures in your organization?
?????....... “Use ‘em? For what?” “To look good to our CEO” “To feel good about what we’ve delivered” “Whatever we need to provide on the company scorecard”We know what we should do…. “We demonstrate our value/ROI for every dollar we spend on learning” “To justify continued/more investment in learning” “To make learning investment decisions in terms of allocating scarce
resources (invest in high impact learning only)” “We don’t really want to measure the business impact of all initiatives but we
sure want to use those ones that are a significant investment and make sure we report to our leadership team”
“We need to ensure more emphasis on incentives, reinforcement guidelines, etc. to ensure the measures keep driving the change”
“Specific to an individual organization but with standards and an informed statistical basis”
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation10
Our Continued Challenges
Shifting the perception Earn a seat at the table when problems are discussed and strategies
developed
Move from delivering training to solving business problems
Traditional thinking – some “get it” and some don’t
Data collection; privacy issues, global infrastructure
On demand learning Work embedded, makes separation of learning from performance
impossible; how will we measure
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation11
Our Lessons Learned
Just Do It Solve business problems, find opportunities to link learning to business results and speak in
business terms
Call out business and operational barriers to performance even if you can’t solve them
Learning is never in a vacuum; establishing a correlation or contribution to a result may be the best we’ll ever do
Reduce complexity of measurements and “just do it” using practical methods
Do it Enterprise Wide Tell the whole story; Participation and Reaction measures matter; but the full picture must be
told with business results
Anticipate major resistance; focus on strategic change management acknowledging that this is a major cultural change
Establish standards and methodology and watch for data distorted to tell only a happy story
Report results in all the right places Drive shared accountability for business results; share data with business stakeholders
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation12
A Positive Approach to Learning Measurement
The Guiding Principles: Continuous improvement; Collaboration; Consistency; Confidentiality; Objective; Ease of Use; Flexibility
IBM Learning Measurement is different from traditional learning measurement, as shown below:
Traditional Learning MeasurementTraditional Learning Measurement IBM Learning MeasurementIBM Learning Measurement
Fault-finding, judgmental Business Impact-Driven
Audit/Control Continuous Improvement
Imposed Collaborative
Add-on, after the fact Established in Requirements; Built into the design
Sporadic Continuous
Subjective Objective
Difficult to understand Easy to understand
Focus on individual solution Focus on all levels
IBM … creating the future of learning
IBM Learning Solutions © 2004 IBM Corporation13
Questions?Gartner, Inc., has recognized IBM in the "Leaders" quadrant in the "2004 E-Learning Suite Magic Quadrant”, the “2004 Learning Management System Magic Quadrant” and the "2003 E-Learning Content Magic Quadrant."
IBM #1: TRAINING magazine conducted extensive research on 606 companies to determine the top providers of employee training and development. IBM was honored as #1 in the 2004 TOP 100 listing. In addition, IBM's "Basic Blue for Managers" blended program was highlighted among only 8 "Best Practices" -- formal learning programs deemed most innovative or successful.
IBM received 2003 ASTD BEST Award, and was ranked highest among information technology companies.
IBM’s Shades of Blue program Recognized by ASTD as a 2003 Excellence in Practice
Managing@IBM was awarded two 2002 ASTD Citations for Organizational Learning and Workplace Learning and Development
IBM received a 2003 Corporate University Xchange Excellence award for Leadership Development sponsored by Fortune Magazine.
IBM Lotus Learning Management System awarded a Top Training Product of 2003 award from HR Executive Magazine
IBM LearningSpace Everyplace wins 2003 Silver Award
Ibm.com Americas – “Blended e-Learning in Call Centers wins 2002 Bronze Award
IBM's Edvisor wins 2001 Gold Award