Date post: | 24-Jan-2018 |
Category: |
Education |
Upload: | kristi-casey-sanders-cmp-cmm-des-hmcc |
View: | 20 times |
Download: | 0 times |
ROI of Learning ToolkitPrepared by Kristi Casey Sanders, CMP, CMM, DES, HMCC,
Director, Thought Leadership, Meeting Professionals International
@KristiCasey #MPI [email protected]
3 Steps in Measurement of Learning
Learning Environment
Learning Process
Learning into Action
2© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Environment Components
If you can’t evaluate all, at least pick two.
DesignSetting
StructureLearner Readiness
Follow-Up
3© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Design Score: ___ out of 90Organizers: Score 1-10 (10 being most true)
Learning offers appropriate level of challenge to participantsThere are activities to suit a wide
variety of learner stylesTechnology is used to aid interaction
with expertsThere is a mix of new and repeat
participantsThe content challenges the status quo
Participants are likely to encounter different points of viewThere will be opportunities to benefit
from collective as well as expert knowledgeParticipants will be able to put and test
learning in their own contextThe meeting encourages mixing
between more industry experienced and less experienced participants
4© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Setting Score: ____ out of 70Organizers: Score 1-10 (10 being most true)
There are varied room layouts (theater-style, cabaret, boardroom, informal, etc.)
There is natural light
Refreshments are available during sessions
There will be spaces for participants to reflect on their own
There will be spaces for pairs to discuss learning
There are small group areas
The meeting provides a safe environment for people to participate
5© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Structure Score: ____ out of 30Organizers: Score 1-10 (10 being most true)
There are frequent breaks
There is time for reflection at the end of sessions
Session timing is right for activities
6© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learner Readiness Score: ____ out of 60Organizers: Score 1-10 (10 being most true)
Speakers will show interest in the audience
Participant needs have been significantly used to inform the selection of content
The content will be very relevant to participants
Participants will receive learning materials in advance of the meeting
Participants will use an app for pre-scheduling and gaining information in advance of sessions
Participants come ready to learn
7© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Follow-Up Score: ____ out of 70Organizers: Score 1-10 (10 being most true)
The learning will provide new insights
Social media will help to engage participants in the content
Sessions will be recorded for participants to use at the event and after the event on any device
Participants will be able to act upon their learning
Participants will be encouraged to contact other participants after the event
Participants are likely to access materials, handouts, slides after the event
This meeting is part of a year-round engagement strategy
8© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Design Score: ____ out of 90Participants: Score 1-10 (10 being most true)
Learning offered the appropriate level of challengeThere were activities to suit my learner
preferenceI used technology to interact with
expertsI met a mix of new and repeat
participantsThe content challenged the status quo
I encountered different points of view
The meeting encouraged mixing between more industry experienced and less experienced participants
I felt I benefitted from collective as well as expert knowledge
I could put and test learning in my own context
9© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Setting Score: ___ out of 70Participants: Score 1-10 (10 being most true)
There were varied room layouts (theater-style, cabaret, boardroom, informal, etc.)
There was natural light
Refreshments were available during sessions
There were spaces for me to reflect on my own
There were spaces for pairs to discuss learning
There were small group areas
This event provided a safe environment for me to participate
10© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Structure Score: ____ out of 30Participants: Score 1-10 (10 being most true)
There were frequent breaks
There was time for reflection at the end of sessions
Session timing was right for activities
11© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learner Readiness Score: ____ out of 60Participants: Score 1-10 (10 being most true)
Speakers showed interest in the audience
I felt participant needs have been significantly used to inform the selection of content
The content was very relevant to me
I received learning materials in advance of the meeting
I used the event app for pre-scheduling and gaining information in advance of sessions
I came ready to learn
12© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Follow-Up Score: ____ out of 70Participants: Score 1-10 (10 being most true)
The learning will provide new insights
I felt engaged by social media communications before, during and after the event
I intend to watch recordings of sessions filmed during the event
I will be able to use what I learned
I will stay in touch with people I met during the event
I intend to refer to session materials, handouts, slides after I return to work
I feel connected to this event’s community all year long
13© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Assessment of Learning Environment Score Card
OrganizerElement Score
Design ____ /90 = _____
Setting ____ /70 = _____
Structure ____ /30 = _____
Learner Readiness ____ /60 = _____
Follow-Up ____ /70 = _____
TOTAL ____ /320 = ____
ParticipantElement Score
Design ____ /90 = _____
Setting ____ /70 = _____
Structure ____ /30 = _____
Learner Readiness ____ /60 = _____
Follow-Up ____ /70 = _____
TOTAL ____ /320 = ____
Average Learning Environment Score (organizer & participant) = _____
14© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Environment Observation Sheet To determine effectiveness of live learning environment, select a random sample of 10% of your sessions and have reviewers audit and rank using the following spectrum. Average
all session scores to determine average onsite score of the live learning environment.
10 123456789
What would your overall score for this session be? Rank the following along the above continuum of 1 to 10.
StructuredWell-TimedInteractiveHands-OnSpeaker Interest in AudienceOpenness to QuestionsAudience-Focused
UnstructuredRushedPassiveTheory
No InterestScripted
Distracted
SESSION SCORE: _____ /70 = _____
Also include reviewer name & notes; date, time & name of session; # of participants & description; session type; and room layout.
15© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Process Pre-Event Survey(feel free to customize/edit to meet your needs)
Meeting my peersMaking new contactsLearning new thingsPromoting my businessMeeting expertsLearning how to do my job betterGaining professional education creditsHaving a break from work
Seeing the destination
Feeling part of this community
Looking for new business
Looking for a new job
Strengthening my network
No specific objectives
Other (please specify) _____________
Participant instructions: Please select your top 3 objectives for attending
High-learning focused people will have 2-3 education-related goals; mid-learning focused will have 1; low-learning will have none. Use data to customize experience & pre-event communications.
16© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Process & Learning to Action Onsite Questionnaire
What learning has taken place?
What questions arise as a result of attending this session/activity?
What did you want to achieve at this meeting/training?
How would you describe your learning experience?
What did you do in advance of the meeting to prepare?
What do you think will happen to the learning after the conference?
Have staff or volunteers stop 1 in every 5 or 10 people at various points exiting sessions or sitting and meeting in specific spaces and ask if they could spare 5 minutes to answer a few questions. Interviewers should introduce themselves as a person acting on your behalf to find out about the experience. Interviews can be verbal, written or facilitated by technology in social, virtual,mobile or blended environments.
Questions for Participants
17© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Process & Learning to Action Onsite Questionnaire Scoring Key
Learning Process (score 0 to 10)• If no responses collected (sheet handed in blank,
nothing to say about learning) = 0• Acknowledgement (repeating info, describing
content) = 2.5• Contemplation (raising questions, using words like
I think, I believe) = 5• Reaction (showing usually emotional responses to
content) = 7.5• Leap to a new idea (showing connection, we can
use this where I work, or what about this) = 10
Learning to Action (score 0 to 10)• No response (no mention of action) = 0
• Application (I will use this, I am going to implement it) = 10
• Adaptation (I want to share this with colleagues, could use this) = 5
Average Learning Process Score = ____
Average Learning to Action Score = ____
18© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Learning Environment, Learning Process and Learning to Action Data
Collection Method Average Score (out of 10)
Learning Environment Evaluations by Organizers and Participants
Learning Environment Observations Onsite
Learning Process Pre-Event/Post-Event Survey
Learning Process & Learning to Action Onsite Questionnaire
Technology (app polls, website & social media behavior)
Traditional Survey Data
TOTAL AVERAGE LEARNING SCORE
19© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Calculating the ROI of Learning
20© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Determine Your Average Learning Score From All Evaluations, Data, Surveys &
Observations• Average Learning Environment Score = ____ out of 10
• Average Learning Process Score = ____ out of 10
• Average Learning into Action Score = ____ out of 10
The average of all three = Average Learning Score = _______ out of 10
Learning Environment
Learning Process
Learning into Action
21© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Determine the Average Learning Cost Per Hour
Total $ spent on average could be:• How much this training costs your company to
produce. • Fully loaded costs could include lost productivity
while learners are out of the office (multiply average per-person hourly salary by hours spent out of office) & how much attendees spend on travel/hotel
• Or, what the competition charges for similar conferences
• Or, what other trainings like this cost in your region
• For free trainings, what is the cost of lost productivity per hour and attendee travel expenses?
• A = Total $ spent, on avg. = $______• B = Total # of learning hours = ___• A / B = Average cost per training hour• Average cost per training hour = $______• Total # of educational hours you’re offering
= ___________• Average per-person registration rate
= $____________
Now calculate the per hour cost:
22© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Calculate the ROI of learning
What was the average learning score? (ex., 7.5)
What is that divided by 10? (ex., .75 or 75%)
What is the avg. cost per training hour times the number of
educational hours you’re offering?
What is #2 multiplied by #3?
What is the average per-person registration fee for your event?
Divide #4 by #5 and multiply by 100 to get ROI of learning%.
23© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula for Benefit to Cost Ratio
Average Learning Score
10
Average Cost Per Training Hour
Total Number of Training or Education
Hours in Your Program
Average Registration Cost Per Person
24© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula
6.5
10$125 21
$1,200
$2,625.65
$1,200
1.4 : 1Benefit to Cost Ratio
25© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula
________
10
$__________ ___________
$ _____________
= _____ Benefit to Cost Ratio26© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula
6.5
10$125 21
$1,200
$2,625.65
$1,200
1.4 : 1Benefit to Cost Ratio
27© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula
________
10
$__________ ___________
$ _____________
= _____ Benefit to Cost Ratio28© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula for ROI percentage
Average Learning Score
10
Average Cost Per Training Hour
Total Number of Training or
Education Hours in Your Program
Total Costs
29
Total Benefits
Total Costs
Total Benefits
100 ROI %
© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula for ROI percentage
6.5
10
$125 21
$525.33/per person cost
30
$1,706.25/per person benefit
$525.33/per person cost
$1,706.25/per person benefit
100 ROI %2.247 224.7
© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
The ROI of Learning Formula
Total benefits($ value of business
impacts)
Fully loadedcosts of program
Fully loaded costsof program
100
= _____ ROI %31© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.
Visit mpiweb.org for more education & inspiration
ROI of Learning Toolkit was prepared by Kristi Casey Sanders, CMP, CMM, DES, HMCC, Director, MPI Academy from Meeting Professionals
International’s research on Measuring Meeting Value and its Measuring Meeting Value Guidebook. More education relating to proving the value of meetings, unique learning formats, event production and experience
design may be found at academy.mpiweb.org.
© 2018, Meeting Professionals International, www.mpiweb.org/join. All rights reserved.