11
DRIVING LEARNING IMPACT WITH HARVARD MANAGEMENTOR® { perspectives } LEADERSHIP DEVELOPMENT
DRIVING LEARNING IMPACT WITH HARVARD MANAGEMENTOR®
{ perspectives } LEADERSHIP DEVELOPMENT
© 2015 Harvard Business School Publishing. All rights reserved. Harvard Business Publishing is an affiliate of Harvard Business School.
ABOUT CORPORATE LEARNING
Harvard Business Publishing Corporate Learning
partners with clients to create world-class leadership
development solutions for managers at all levels. Our
team leverages the management insight, thought
leadership, and expertise of Harvard Business School
faculty and authors from Harvard Business Review
to create tailored leadership development solutions.
With more than 20 years of practical experience,
our innovative, technology-enabled solutions drive
meaningful and lasting business results. Corporate
Learning is a market group within Harvard Business
Publishing.
ABOUT HARVARD BUSINESS PUBLISHING
Harvard Business Publishing was founded in 1994 as
a not-for-profit, wholly owned subsidiary of Harvard
University. Its mission is to improve the practice of
management and its impact in a changing world. The
company achieves its mission through its relationships
with customers in three market groups: Higher
Education, Corporate Learning, and Harvard Business
Review Group. Through these platforms, Harvard
Business Publishing is able to influence real-world
change by maximizing the reach and impact of its
essential offering—ideas.
H A R VA R D B U S I N E S S P U B L I S H I N G C O R P O R AT E L E A R N I N G { 1 }
DRIVING LEARNING IMPACT WITH HARVARD MANAGEMENTOR®
Reaching for ImpactOrganizations know that for leadership development interventions to be successful, learning needs to be integrated with work and linked to the achievement of goals and business results. Leadership develop-ment measurement approaches, however, have traditionally focused on knowledge gain and are too often disconnected from actual work.
The disconnect is largely a result of poor learning design. If the learning design fails to provide learners with adequate opportunity to practice and apply what they learn on the job, it becomes challenging to reli-ably measure learning impact. Development should be integrated with learners’ work to move from learning how to be a leader to actually becoming a leader. Without that mind-set shift among current manag-ers, organizations will not effectively prepare their managers to drive business impact.
With these challenges in mind, how can an organization augment the traditional model of measuring knowledge gain by also measuring learning impact? In order for leadership development to create mea-surable learning impact, it must have three fundamental elements: an optimized learning design, valid and reliable metrics and measurement, and appropriate reporting and analytics.
Harvard ManageMentor is an on-demand learning and performance support resource designed to drive learning transfer through an inno-vative work-based approach and focus on measurable learning impact that brings these fundamental elements together.
For leadership development to have lasting effect, programs require the right learning design, metrics, and reporting and analytics. Harvard ManageMentor’s on-demand, work-based approach and new On-the-Job component provide organizations with new ways to drive measurable learning impact.BY DIANE BELCHER, DENNIS GULLOTTI, AND MICHAEL WARECH
HOW CAN AN ORGANIZATION AUGMENT THE TRADITIONAL MODEL OF MEASURING KNOWLEDGE GAIN BY ALSO MEASURING LEARNING IMPACT?
driving learning impact with harvard managementor{ 2 }
Three Keys to Success
Learning DesignStrong leadership development programs start with a design that is oriented toward learning impact. The most impactful learning programs are focused not just on increasing knowledge, but also on helping learners improve by applying that knowledge consistently in their day-to-day activities.
For example, learners can master a skill through practicing, making mistakes, and trying things out. Skill development and, ultimately, mastery do not come from passively receiving information. Instead, they come from cycles of learning, reflecting, and solving problems. That’s why impactful leadership development programs incorporate concepts of doing and reflecting directly into their learning design.
Harvard Business School professor David Garvin points out in Rethinking the MBA, “Innovative thinking and creativity, much like swimming and biking, is a skill best learned through practice, reflection, and repeated application. Reflection plays a key part in learning from experience—in fact, it is just as important as practice.”
Technology can be a great enabler of practice and reflection as an integral part of learning. For example, reflection can be built right into an on-demand platform making it easy for the learner to capture reflections and reference them later.
Since managers are learning on the job, it is critical that the learning design focuses them on the elements they need to practice and apply the most. Programs can enhance learning by providing tools at the moment when learners need them. These resources include focused pieces of content, job aids, and activities. A flexible, modular, and on-demand approach to learning provides busy managers with exactly what they need to know, in a format that is relevant and only a tap or click away.
And we all know that learning is more impactful when it’s part of an ongoing process. Learners need to be engaged in a learning experience process versus a one-time event. When given repeated opportunities to explore concepts via a wide variety of experiences, learners absorb more.
REFLECTION IMPROVES PERFORMANCE
A 2014 study by Giada Di Stefano (HEC Paris), Francesca Gino (Harvard Business School), Gary Pisano (Harvard Business School), and Bradley Staats (University of North Carolina Kenan-Flagler Business School) provides evidence that routine reflection by adults improves their performance. Specifically, the research team conducted a series of studies that compared groups focused on learning by doing with groups focused on learning by doing and reflecting. Their results showed that the groups whose experience was deliberately coupled with reflection consistently outperformed the other groups. They concluded that reflecting on what we’ve done teaches us to do it more effectively the next time around. Also, “When we stop, reflect, and think about learning, we feel a greater sense of self-efficacy,” Gino says. “We’re more motivated and we perform better afterward.”
measurable impact = design + metrics + reporting/analytics
H A R VA R D B U S I N E S S P U B L I S H I N G C O R P O R AT E L E A R N I N G { 3 }
Work-based Learning Approach Focuses on Application of Skills on the JobHarvard ManageMentor focuses on integrating learning and work, with an emphasis on application throughout. Harvard ManageMentor is oriented toward learning impact starting with the design of each management topic. Each online topic is broken into concise, bite-size learning bursts called lessons. Since every minute is precious to time-starved managers, these short learning bursts were designed so that each piece of content is important and useful, and to allow for maximum impact through practice and reflection.
Each lesson has a built-in cycle of learn-practice-reflect, in which learners explore the core content, practice what they have learned in scenario-based activities, and then reflect on their learning and how they can apply it to their everyday work. Since Harvard ManageMentor is available on demand, and practice and reflection are part of the learning design, these important steps become routine for learners. The benefit of creating that routine is learners are already thinking about application when starting a lesson, which helps increase learning impact. Also, their reflections are automatically stored and referenced in other components of the learning design.
Learning MetricsThe well-known Kirkpatrick model of training evalu-ation provides the key metrics that any professional needs to consider.
Historically, learning and development (L&D) organizations focus their measurements on the first two levels. While level two, knowledge gain, is important and necessary, that data alone isn’t enough to proclaim that successful behavior change and learning transfer have taken place. It certainly won’t allow you to measure true learning impact. While it is important to maintain these two levels as foundational metrics, we encourage companies to strive for level three and four measurement.
Having valid and reliable learning metrics is a good start, but what hap-pens when some learners, or a segment of learners, meet the desired levels and others do not? How do you determine if outside factors may have either positively or negatively impacted the results?
THE KIRKPATRICK MODEL
LEVEL 1: REACTION The users’ thoughts and feelings about the training.
LEVEL 2: LEARNING
An increase in knowledge or skills.
LEVEL 3: BEHAVIOR The transfer of knowledge or skills to the job.
LEVEL 4: RESULTS
The final results of the training, such as business impact or results.
DEVELOPMENT SHOULD BE INTEGRATED WITH LEARNERS’ WORK TO MOVE FROM LEARNING HOW TO BE A LEADER TO ACTUALLY BECOMING A LEADER
driving learning impact with harvard managementor{ 4 }
Along with learning metrics, additional data needs to be captured throughout the learning experience in order to assess the level of impact certain program features may have played in the outcome of the developmental experience. For example:
oo Were the learners provided with adequate support and resources for applying new skills and concepts?
oo Did the topics and supporting materials help the learners in achiev-ing their development goals?
oo Did the learners discuss their action plans and performance results with their managers?
It stands to reason a learner with higher levels of support from her manager will have increased learning impact. Capturing appropriate underlying metrics allows program managers and organizations to monitor, adjust, manage, and communicate the impact of their leader-ship development programs more accurately, efficiently, and effectively.
Harvard ManageMentor’s Measurement Emphasis on Learning ImpactBecause Harvard ManageMentor is an on-demand resource, it automati-cally captures these vital level three statistics through the On-the-Job component within each topic. Each lesson within a topic has a related performance goal. Upon completion of the topic, learners rate them-selves against these goals and select personal performance goals. When selecting a performance goal, learners see their reflections from each lesson, further helping make the connection from learning a new skill to applying the new skill on the job. After selecting performance goals, learners then create an action plan to apply what they have learned to their everyday work.
Before starting the action plan, learners take a post-topic Impact Survey, which captures their personal insights related to the impact of the learning and their overall experience.
Learners then work on their action plan over a period of time and are “mentored” with emailed content, tips, and reminders to keep them motivated and on track. The Harvard ManageMentor On-the-Job com-ponent helps managers translate their ideas into practice and helps them learn from experience.
After completing the On-the-Job component, learners take another Impact Survey. Comparing responses between the two surveys provides key measurements for determining the magnitude of learning impact.
The two brief On-the-Job surveys assess learners’ performance across several impact drivers, such as:
“The Learn-Practice-Reflect model [in Harvard ManageMentor] helps our learners retain what they are learning. The On-the-Job action plans are a very effective way to apply the learning and develop the correct mind-set within the individual.”
Mutheer Al Kalbani, Senior Manager of HR Development, Bank Sohar S.A.O.G.
H A R VA R D B U S I N E S S P U B L I S H I N G C O R P O R AT E L E A R N I N G { 5 }
SUPPORT FOR APPLICATION ON THE JOB
“I was provided with adequate support and resources (e.g., time, money) for applying new skills and concepts to my job.”
MANAGER REINFORCEMENT
“My manager and I have discussed my action plan or my results from applying new skills and concepts from this topic.”
COURSE RELEVANCE
“Have the topic and supporting materials (videos, handouts, etc.) helped me achieve my development goals?”
BUSINESS ALIGNMENT
“Is this topic aligned to my business priorities and development goals (only asked in the post-topic survey)?”
Three additional questions combine to create a learning impact measure. This measurement is the net improvement across these three areas:
1. Knowledge impact: I used new skills and knowledge from this topic.
2. Job impact: I have applied skills and concepts from this topic to my job.
3. Results: The new learning I have applied has improved my actual performance on the job.
This approach to learning impact delivers a new, single measure of applied learning and answers the question: What is the impact of apply-ing skills learned on their daily work and, by extension, the business?
Reporting and AnalyticsLearning metrics should tell a story, one that provides information to a variety of stakeholders in order to help them make informed business decisions:
oo Instructional designers focused on the effectiveness of the program and how the program aligns to the learner’s job.
oo Organizational leaders focused on who has completed the program and if they are applying the learning in a consistent manner.
oo L&D teams focused on how the investment in learning is being used and if it is having an impact on business results.
Dashboards and reports should support the different levels and types of information required by these stakeholders. Managers also need quick access to deeper analytics and customized reporting so they can diagnose potential issues faster.
Learning analytics allow for the combination and analysis of mul-tiple types of metrics. Simple correlational work all the way through
THE ON-THE-JOB COMPONENT HELPS MANAGERS TRANSLATE THEIR IDEAS INTO PRACTICE AND HELPS THEM LEARN FROM EXPERIENCE
WHO LEARNS FROM REPORTING?
INSTRUCTIONAL DESIGNERS Effectiveness and skills alignment
ORGANIZATIONAL LEADERS
Completion and skills application
L&D TEAMS How the investment is creating business results
driving learning impact with harvard managementor{ 6 }
predictive modeling may be possible, all in the service of identifying relationships, patterns, and trends to determine the effectiveness of an overall program and its various component parts. As noted above, being able to see how a certain factor like a manager’s support impacts learning is critical to making smart program adjustments.
Harvard ManageMentor MetricsAdministrators of Harvard ManageMentor have access to a powerful and easy-to-use analytics and reporting tool that gives them the opportunity to do that kind of detailed analysis. Called the One View dashboard, it includes a series of tiles that provide a snapshot of various aspects of the program, including how effective learners are in applying skills on the job. Learning impact is a single measurement shown in the One View dashboard, for all learners and topics.
Drilling in to the Learning Impact report shows the learning impact percentage over time with filters available for all topics or a specific topic, and over a specific time period. The report also includes the aver-age score for each impact driver compared to Harvard ManageMentor’s benchmarking data.
Having a benchmark helps target if and where adjustments to a program should be made. For example, if management support scores are low relative to the benchmark, then program managers can take extra steps to ensure that leaders are involved in the learning by using Harvard ManageMentor’s Develop Others materials to facilitate a discussion with their teams to reinforce the targeted concepts.
Data-based decision making is crucial to successful talent development and management and overall business success. Harvard ManageMentor’s reports do not simply present information or results; instead, they
BEING ABLE TO SEE HOW A MANAGER’S SUPPORT IMPACTS LEARNING IS CRITICAL TO MAKING SMART PROGRAM ADJUSTMENTS
Learning impact is the net improvement across three areas:
• Knowledge impact• Job impact• Results
HARVARD MANAGEMENTOR ONE VIEW DASHBOARD
H A R VA R D B U S I N E S S P U B L I S H I N G C O R P O R AT E L E A R N I N G { 7 }
ABOUT HARVARD MANAGEMENTOR
Harvard ManageMentor is the premier on-demand learning and performance support resource for leadership and management skill development. Harvard ManageMentor’s high-quality content—regularly updated topics, videos, tools, and on-the-job activities—engage and motivate learners. Harvard ManageMentor is fueled by the latest in thinking and proven practices from Harvard Business Publishing’s world-class experts and designed to drive learning transfer and business impact for organizations.
help drive decision making and identify possible actions related to the developmental programming and the organization’s business.
SUMMARY
Achieving ImpactIn today’s global, connected, and fast-paced environment, competitive-ness requires leadership development that shows its impact on perfor-mance. Traditional approaches that look at just knowledge gain are not sufficient. We need a new approach that considers:
oo A learning design that is fully oriented to application, integrat-ing learning with work and requiring learners to think about how learning new skills can improve their performance.
oo The right learning metrics that don’t just measure knowledge gain, but also the impact new skills are having on individual and organi-zational performance.
oo Reporting and analytics that provide views for different roles and levels, and that can also help determine which aspects of learn-ing are working or not working for which audiences so that smart adjustments can be made.
Harvard ManageMentor’s on-demand design facilitates rapid application via concise, goal-oriented content; built-in cycles of learn-practice-reflect; and an On-the-Job component that helps learners apply new skills and reinforce critical business concepts. In addition, Harvard ManageMentor’s intuitive dashboards, reporting, and analytics give L&D teams the tools to begin to assess the level of learning impact taking place and to con-tinually improve their leadership development programs.
While the learning impact reporting in Harvard ManageMentor is a powerful way to measure the success of a program, organizations should explore additional measures to help assess the level of impact their programs are having on their businesses’ bottom lines.
INTUITIVE DASHBOARDS, REPORTING, AND ANALYTICS GIVE L&D TEAMS THE TOOLS TO BEGIN TO ASSESS THE LEVEL OF LEARNING IMPACT
driving learning impact with harvard managementor{ 8 }
DIANE BELCHER is director of continuous learning for
Corporate Learning. She is responsible for the product
strategy for the Harvard ManageMentor offering.
Prior to joining the Harvard Business Publishing team,
Diane worked for Fidelity Investments, where she held
various roles in human resources and led the customer
experience strategy for the Workplace Investing business.
DENNIS GULLOTTI is associate director, solutions
marketing for Harvard Business Publishing. He is
responsible for the go-to-market strategy and execution
for Harvard ManageMentor, the premier on-demand
learning resource for leadership and management
development. He has over 10 years of experience
in talent management and corporate learning, with
previous work experience that includes positions at
Plateau Systems and Thomson NETg.
MICHAEL A. WARECH, PH.D., is a senior learning
solutions manager with Harvard Business Publishing
Corporate Learning. Michael has over 25 years of
experience designing, developing, and implementing
data-driven human capital solutions that demonstrably
impact an organization’s bottom line. He has led
numerous projects for a variety of Fortune 1000
companies and organizations. He has worked with
companies to identify and articulate their learning
strategies and has deep and direct experience designing
leadership development solutions as well as more broad
and robust learning and development curricula.
ABOUT THE AUTHORS
UNITED STATESHarvard Business Publishing 60 Harvard Way Boston, MA 02163
phone: 800-795-5200 (Outside U.S. and Canada call 617-783-7888) email: [email protected]
EUROPE Harvard Business School Publishing Europe Pvt. Ltd. Vernon House 23 Sicilian Avenue London WC1A 2QS
phone: +0203 463 2350 email: [email protected]
INDIAHarvard Business School Publishing India Pvt. Ltd. Piramal Tower, 7th floorPeninsula Corporate ParkGanpatrao Kadam MargLower Parel (West), Mumbai—400 013
phone: +91 22 6108 4043email: [email protected]
SINGAPOREHarvard Business School Publishing Asia/Pacific Pvt. Ltd. 80 Raffles Place, #25-01 UOB Plaza 1 Singapore 048624
email: [email protected]
© 2015 Harvard Business School Publishing. All rights reserved.
Harvard Business Publishing is an affiliate of Harvard Business School. MC192370515
LEARN MOREwww.harvardbusiness.org
