45
1 ©2003 KnowledgeAdvisors. All rights reserved. ©2004 KnowledgeAdvisors. All rights reserved. Examples of How IT Training Companies Measure Learning ROI and Why
1
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.©
20
04
Kn
ow
led
ge
Ad
vis
ors
. A
ll rig
hts
re
serv
ed
.
Exa
mpl
es o
f How
IT
Tra
inin
g C
ompa
nies
M
easu
re L
earn
ing
RO
I an
d W
hy
2
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Dem
onst
ratin
g th
e V
alue
of L
earn
ing
‘We are very interested in
measuring the ROI of training and
certification but I don’t know of a
feasible methodof doing this.’
‘Our annual IT training budget
depends on 2 things. Business
profitability and the success I have in
making the case
that training is
essential for the business.’
3
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Dem
onst
ratin
g th
e V
alue
of L
earn
ing
““ Hav
ing
som
eone
aw
ay
from
offi
ce in
H
avin
g so
meo
ne a
wa
y fr
om o
ffice
in
trai
ning
for
5 da
ys is
a h
uge
cos
t to
us;
I tr
aini
ng fo
r 5
days
is a
hu
ge c
ost t
o us
; I
can
can ’’
t do
that
unl
ess
I kno
w th
at 9
5% o
f t d
o th
at u
nles
s I k
now
that
95%
of
that
is jo
b re
leva
nt.
that
is jo
b re
leva
nt. ””
[[ Tar
get]
Tar
get]
‘‘ [P
rovi
ding
mea
sure
s of
the
impa
ct o
f tra
inin
g on
[P
rovi
ding
mea
sure
s of
the
impa
ct o
f tra
inin
g on
bu
sine
ss p
erfo
rman
ce]
shou
ld b
ecom
e a
rout
ine
busi
ness
per
form
ance
] sh
ould
bec
ome
a ro
utin
e of
ferin
g in
the
arse
nal
of
trai
ning
tool
s em
plo
yed
offe
ring
in th
e ar
sen
al o
f tr
aini
ng to
ols
empl
oye
d b
y ve
ndo
rs.’
by
vend
ors
.’[I
DC
Jul
[ID
C J
ul-- 0
3]03]
4
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Wha
t is
Lear
ning
Ana
lytic
s
Lear
ning
Ana
lytic
s is
the
term
use
d to
desc
ribe
the
met
rics
that
hel
por
gani
zatio
ns u
nder
stan
d ho
wto
bet
ter
trai
n &
dev
elop
em
ploy
ees,
part
ners
and
cus
tom
ers.
5
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
The
sta
te o
f lea
rnin
g an
alyt
ics
Cur
rent
•N
o co
mm
on w
ay to
mea
sure
le
arni
ng p
erfo
rman
ce
•P
oor
proc
esse
s on
exi
stin
g m
easu
rem
ent i
nitia
tives
•Li
mite
d ac
cess
to m
eani
ngfu
l an
d tim
ely
data
•D
isco
mfo
rt in
bei
ng u
ninf
orm
ed
Des
ired
•A
pro
ven
wa
y to
con
sist
ently
m
easu
re
•A
turn
-key
sol
utio
n th
at c
ould
be
impl
emen
ted
quic
kly
and
easi
ly
•A
nee
d fo
r co
mpr
ehen
sive
and
m
eani
ngfu
l per
form
ance
re
port
s
•Lo
w in
itial
inve
stm
ent t
o be
gin
6
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Res
ults
of B
est P
ract
ices
Res
earc
h•
Onl
y 22
% o
f org
aniz
atio
ns b
ench
mar
k ex
tern
ally
•85
% o
f tim
e is
spe
nt o
n ad
min
istr
ativ
e ta
sks
inst
ead
of
anal
yzin
g th
e da
ta•
76%
feel
ana
lyzi
ng jo
b an
d bu
sine
ss im
pact
is v
ery
impo
rtan
t but
onl
y 14
% d
o a
good
job
at it
•77
% fe
el ‘r
easo
nabl
e in
dica
tors
’ are
ade
quat
e fo
r de
cisi
on-m
akin
g•
100%
can
not e
xpen
d si
gnifi
cant
res
ourc
es fo
r ‘p
reci
se’
data
7
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Bal
ance
d S
core
card
for
Mea
sure
men
tLe
vel
1
Leve
l 2
Leve
l 3
Leve
l 4
Leve
l 5
Did
the
y lik
e it?
Did
the
y le
arn?
Do
the
y us
e it?
Wha
t wer
e th
e re
sults
?
Wha
t’s th
e R
OI?
©20
04 K
now
ledg
eAdv
isor
s. A
ll rig
hts
rese
rved
.
Nee
ded
by
clie
nts
>>
>>
provided to clients >>>>
8
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
The
Mea
sure
men
t Pro
cess
Dat
a C
olle
ctio
nD
ata
Sto
rage
Dat
a R
epor
ting
•D
efin
e m
etri
cs (
KP
I’s)
•D
esig
n in
stru
men
ts•
Det
erm
ine
popu
latio
n•
Dis
sem
inat
e in
stru
men
ts•
Col
lect
dat
a
•Dat
abas
e cr
eatio
n•C
entr
aliz
e da
ta
•Dat
a en
try
•Dat
a se
curit
y•R
aw d
ata
acce
ss
•S
tand
ard
repo
rts
(can
ned)
•A
d-ho
c qu
eryi
ng•
Tre
nd-li
ne a
naly
sis
•S
tatis
tical
ana
lysi
s•
Ben
chm
ark
com
pari
sons
•T
rans
actio
nal d
etai
ls•
Man
ager
rep
orts
•E
xecu
tive
sum
mar
ies
Dat
a P
roce
ssin
g
•A
ggre
gatio
n of
dat
a•
Filt
erin
g of
dat
a•
Con
vert
into
met
rics
Be
ca
use
da
ta c
olle
ctio
n,
pro
ce
ssin
g a
nd
re
po
rtin
g is
au
tom
ate
d,
yo
u h
ave
mo
re t
ime
to
an
aly
ze t
he
da
ta a
nd
turn
it
into
ac
tio
na
ble
bu
sin
ess
in
telli
ge
nc
e t
o:
1)
imp
rov
e y
ou
r le
arn
ing
pro
gra
ms
2)
de
mo
nst
rate
va
lue
to
yo
ur
sta
ke
ho
lde
rs
9
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Com
parin
g th
e R
OI M
odel
s •A fo
rmal
ized
pro
cess
for
cond
uctin
g in
dep
th R
OI i
mpa
ct
stud
ies
•Rig
orou
s ap
proa
ch•T
echn
olog
y fa
cilit
ates
task
s•R
equi
res
serio
us in
vest
men
ts o
f pa
rtic
ipan
ts ti
me
•Aut
hore
d by
Drs
. Jac
k an
d P
atti
Phi
llips
leve
ragi
ng a
ll co
mpo
nent
s of
thei
r R
OI p
roce
ss•S
ugge
sted
to b
e us
ed 5
to 1
0% o
f th
e tim
e
•A
n R
OI r
elat
ive
to a
sp
ecifi
c bu
sine
ss r
esul
t•
Pra
ctic
al if
leve
rage
te
chno
logy
•M
ore
com
plex
for
part
icip
ants
to
com
plet
e•
Leve
rage
s P
hilli
ps
prin
cipl
es o
f es
timat
ion,
isol
atio
n,
and
adju
stm
ent
•C
ould
be
used
100
%
of th
e tim
e or
on
a ca
se b
y ca
se b
asis
•A jo
b im
pact
RO
I•V
ery
scal
eabl
e an
d re
plic
able
•Eas
y fo
r pa
rtic
ipan
ts to
com
plet
e •
Sol
id in
dica
tor
of R
OI o
n pe
rfor
man
ce
rela
tive
to th
e in
divi
dual
s H
uman
Cap
ital
(i.e
. sal
ary)
•Le
vera
ges
Phi
llips
prin
cipl
es o
f es
timat
ion,
isol
atio
n, a
nd a
djus
tmen
t•U
se 1
00%
of t
he ti
me
Imp
act
Stu
dy
Pro
cess
Bu
sin
ess
Res
ult
RO
IH
um
an C
apit
al R
OI
10
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Pra
ctic
al
Indi
cato
rs
‘Rea
ctio
n’Le
vel 1
‘Lea
rnin
g’Le
vel 2
‘Job
Impa
ct’
Leve
l 3
‘Res
ults
Leve
l 4&
E
stim
atio
nIs
olat
ion
Adj
ustm
ent
‘RO
I/RO
E’
Leve
l 5
11
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Met
rics
that
Mat
ter
in th
e IT
Indu
stry
**
*
* C
urre
ntly
in p
ilot o
r im
plem
enta
tion
phas
e.
12
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Cas
e S
tudy
: M
icro
soft
•M
easu
res
all t
rain
ing
in th
eir
educ
atio
n ch
anne
l aro
und
the
wor
ld (
colle
ct a
bout
5,0
00 e
vals
per
wee
k)•
Col
lect
rea
ctio
n da
ta a
nd p
redi
cted
impa
ct d
ata
at e
nd o
f cl
ass
via
Inte
rnet
•60
day
s la
ter
colle
ct fo
llow
up
data
for
bette
r im
pact
re
sults
•C
alcu
late
a H
uman
Cap
ital R
OI S
core
card
that
can
be
run
by c
ours
e, c
urric
ula,
clie
nt, v
endo
r, o
r de
liver
y•
Pro
duce
mon
thly
rep
orts
that
go
to s
ever
al s
take
hold
ers
at M
icro
soft
Lear
ning
(ex
. Top
cou
rses
, Top
Ven
dors
, C
lient
Sat
isfa
ctio
n, R
OI o
f Top
25
Cou
rses
)
13
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mic
roso
ft S
ampl
e R
epor
ts
Sa
mpl
e D
ata.
For
Illu
stra
tion
Pur
pose
s O
nly.
14
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mic
roso
ft S
ampl
e R
epor
ts
Sa
mpl
e D
ata.
For
Illu
stra
tion
Pur
pose
s O
nly.
15
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Cas
e S
tudy
: P
eopl
eSof
t Edu
catio
n•
Col
lect
ed o
ver
20,0
00 e
valu
atio
ns s
ince
Sep
tem
ber
2003
•R
equi
re p
artic
ipan
ts to
pre
dict
and
est
imat
e tr
aini
ng
linka
ge to
the
follo
win
g bu
sine
ss r
esul
ts:
–In
crea
sed
Sal
es
–D
ecre
ased
Cos
ts
–In
crea
sed
Pro
duct
ivity
–In
crea
sed
Qua
lity
–D
ecre
ased
Cyc
le T
ime
–In
crea
sed
Cus
tom
er S
atis
fact
ion
–In
crea
sed
Em
plo
yee
Sat
isfa
ctio
n
•C
an r
un a
Bus
ines
s R
esul
t Sco
re C
ard
by C
lient
, cou
rse,
pr
ogra
m e
tc. t
o lo
ok a
t lin
kage
to b
usin
ess
resu
lts
16
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Peo
pleS
oft E
duca
tion
Hig
hlig
hts
•P
R Is
sued
Mar
ch 2
2, 2
004
•20
% im
prov
emen
t in
user
pro
duct
ivity
afte
r co
mpl
etin
g P
eopl
eSof
t tra
inin
g cl
ass
•24
% im
prov
emen
t in
cycl
e tim
e •
22%
impr
ovem
ent i
n qu
ality
•>
90%
feel
the
trai
ning
was
a w
orth
whi
le in
vest
men
t for
th
eir
empl
oyer
s“W
e se
e gr
eat v
alue
in b
eing
abl
e to
link
Peo
pleS
oft’
educ
atio
n co
urse
s to
an
incr
ease
in p
rodu
ctiv
ity.”
--J
osep
h C
umm
ings
, IS
Dire
ctor
, DeP
aul (
clie
nt o
f Peo
pleS
oft)
17
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Peo
pleS
oft E
duca
tion
Hig
hlig
hts
•Gen
erat
es e
nd o
f tr
aini
ng a
nd fo
llow
up
impr
ovem
ent
perc
enta
ges
in k
ey
resu
lts•O
ver
13,0
00
eval
uatio
ns c
olle
cted
in
1st
six
mon
ths
by
leve
ragi
ng g
loba
l te
chno
logy
to c
olle
ct,
stor
e, p
roce
ssan
d re
port
the
data
18
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Ana
lyst
W
orks
heet
A to
ol e
nabl
ing
the
lear
ning
team
to
con
duct
mor
e co
mpr
ehen
sive
bu
sine
ss im
pact
an
d R
OI
anal
ysis
. T
he
tool
allo
ws
you
to
stor
e ac
tual
bu
sine
ss r
esul
ts
tied
to m
ultip
le
prog
ram
s an
d sh
ow tr
aini
ngs
impa
ct a
nd R
OI.
Ana
lyst
W
orks
heet
19
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Cas
e S
tudy
: N
ew H
oriz
ons
•La
rges
t IT
Tra
inin
g C
ompa
ny in
the
wor
ld•
Tra
in in
mul
tiple
cer
tifie
d IT
cha
nnel
s•
Use
d by
NH
cen
ters
aro
und
the
wor
ld•
Ove
r 1.
5 m
illio
n su
rvey
s co
llect
ed•
Col
lect
dat
a us
ing
mul
tiple
IT c
hann
el fo
rms
(Mic
roso
ft,
Citr
ix, C
isco
)•
Use
the
data
to m
anag
e th
eir
oper
atio
ns q
ualit
y an
d cu
stom
er s
ervi
ce•
Leve
rage
the
data
for
stra
tegi
c m
arke
ting
purp
oses
(se
e P
R)
20
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
New
Hor
izon
s H
ighl
ight
s•
38%
of p
erfo
rman
ce im
prov
emen
ts w
ere
dire
ctly
at
trib
utab
le a
nd is
olat
ed to
trai
ning
•
70%
of p
artic
ipan
ts a
pply
trai
ning
to jo
b w
ithin
8 w
eeks
•97
% u
tiliz
e/pl
an to
util
ize
the
trai
ning
on
the
job
•60
% in
crea
se in
ski
ll/kn
owle
dge
from
trai
ning
•A
4:1
pre
dict
ed b
enef
it to
cos
t rat
io w
hen
leav
e tr
aini
ng“P
rior
to o
ur r
elat
ions
hip
with
New
Hor
izon
s w
e w
ere
not
effe
ctiv
ely
prov
idin
g a
prod
uctiv
ity b
asel
ine
for
our
know
ledg
e w
orke
rs.”
--A
ntho
ny G
arre
ffi, M
anag
er E
-Lea
rnin
g/T
echn
ical
Tra
inin
g, Ir
on M
ount
ain
21
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
New
Hor
izon
s H
ighl
ight
s
Sou
rce:
Kno
wle
dgeA
dvis
ors
Met
rics
that
Mat
ter
Dat
abas
e
Thi
s sh
ows
the
pred
icte
d R
OI r
esul
ts fo
r Ir
on M
ount
ain.
It i
s ca
lcul
ated
bas
ed o
n th
e im
prov
emen
t in
hum
an c
apita
l is
olat
ed to
the
trai
ning
, adj
uste
d fo
r bi
as.
4/1
to 4
/30
Iron
Mou
ntai
n R
OI D
ata
22
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
New
Hor
izon
s H
ighl
ight
s
Sou
rce:
Kno
wle
dgeA
dvis
ors
Met
rics
that
Mat
ter
Dat
abas
e
Thi
s sc
orec
ard
coul
d be
ru
n vs
. a v
arie
ty o
f be
nchm
arks
. Y
ou c
ould
al
so s
et g
oals
with
the
clie
nt u
p fr
ont a
nd r
un it
ag
ains
t the
goa
ls to
o.
Her
e Ir
on M
ount
ain
rate
d th
eir
trai
ning
exp
erie
nce
supe
rior
to th
e be
nchm
ark
and
with
be
tter
impa
ct th
an th
e be
nchm
ark.Ir
on
Mo
un
tain
vs. b
ench
mar
k. I
ron
Mou
ntai
n ra
tes
thei
rex
perie
nce
high
er a
cros
s al
l lea
rnin
g le
vels
.
23
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
IT T
rain
ing
Pro
vide
r B
ench
mar
k G
roup
Sa
mpl
e D
ata.
For
Illu
stra
tion
Pur
pose
s O
nly.
24
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Bes
t Pra
ctic
es in
Lea
rnin
g M
easu
rem
ent
•P
lan
your
met
rics
befo
re w
ritin
g su
rvey
que
stio
ns
•E
nsur
e yo
ur m
easu
rem
ent p
roce
ss is
rep
licab
le a
nd s
cale
able
•E
nsur
e m
etric
s ar
e in
tern
ally
and
ext
erna
lly c
ompa
rabl
e•
Use
indu
stry
-acc
epte
d m
easu
rem
ent a
ppro
ache
s
•D
efin
e va
lue
in th
e e
yes
of y
our
stak
ehol
ders
•M
anag
e th
e ch
ange
ass
ocia
ted
with
mea
sure
men
t
•E
nsur
e th
e m
etric
s ar
e w
ell-b
alan
ced
•Le
vera
ge a
utom
atio
n an
d te
chno
logy
•C
raw
l, w
alk,
run
•E
nsur
e yo
ur m
etric
s ha
ve fl
exib
ility
Sou
rce:
CLO
Mag
azin
e, O
ctob
er 2
003
25
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Con
tact
Info
rmat
ion
ww
w.k
now
ledg
eadv
isor
s.co
m
For
furt
her
info
rmat
ion
on th
e in
form
atio
n co
ntac
t:Je
ffrey
Ber
kV
ice
Pre
side
nt, P
rodu
cts
and
Str
ateg
y+
1 31
2 42
3 85
99 (
phon
e)+
1 31
2 37
2 00
44 (
fax)
jber
k@kn
owle
dgea
dvis
ors.
com
26
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
App
endi
x: D
efin
e V
alue
Ask
you
r st
akeh
olde
rs…
.
If I e
xcee
d yo
ur e
xpec
tatio
ns
with
this
lear
ning
pro
gram
w
hat o
utco
mes
sho
uld
I ha
ve d
emon
stra
ted?
27
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Def
ine
You
r S
take
hold
ers
�E
mp
loye
es�
Cu
sto
mer
s �
Su
pp
liers
�S
har
eho
lder
s�
Tax
pay
ers
�P
artn
ers
28
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
3 S
tep
Pro
cess
to V
alue
Opt
imiz
atio
n
1.Id
enti
fy q
uan
tifi
able
org
aniz
atio
nal
ob
ject
ives
2.E
nsu
re p
rog
ram
s si
gn
ific
antl
y im
pac
t o
rgan
izat
ion
al o
bje
ctiv
es
3.Im
ple
men
t co
nti
nu
ou
s im
pro
vem
ent
pro
cess
29
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Rea
sons
to In
vest
in L
earn
ing
�In
crea
se r
even
ue
�R
edu
ce c
ost
�R
edu
ce c
ycle
-tim
e�
Incr
ease
pro
du
ctiv
ity
�Im
pro
ve q
ual
ity
�E
nh
ance
em
plo
yee
loya
lty
�E
nh
ance
cu
sto
mer
loya
lty
�M
itig
ate
risk
30
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Ens
ure
Pro
gram
s A
ligne
d w
ith
Org
aniz
atio
nal O
bjec
tives
Me
asu
re e
ac
h c
ou
rse
ag
ain
st b
usi
ne
ss a
nd
str
ate
gic
ob
jec
tiv
es
toe
nsu
re a
lign
me
nt to
th
ose
o
bje
ctiv
es.
31
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Lear
ning
Effe
ctiv
enes
s
Me
tric
s th
at
Ma
tte
r in
teg
rate
s w
ith
Q
ue
stio
nm
ark
to
en
ha
nc
e
test
re
po
rtin
g
an
d a
na
lytic
s.
32
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Tim
e to
Job
Impa
ct In
dica
tors
This
is
the
tim
e t
o jo
b im
pa
ct
pe
rfo
rma
nc
e in
dic
ato
r. It
le
ts y
ou
kn
ow
if a
nd
wh
en
tra
inin
g w
as
ap
plie
d t
o t
he
jo
b.
It
is c
olle
cte
d m
on
ths
aft
er
the
tra
inin
g,
au
tom
atic
ally
by
Me
tric
s th
at
Ma
tte
r.
It h
elp
s th
e o
rga
niz
atio
n u
nd
ers
tan
d w
he
n t
rain
ing
effe
cts
th
e jo
b, if a
t a
ll.
33
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
App
licat
ion
of L
earn
ing
to J
ob
This
in
dic
ato
r p
rov
ide
s in
sig
ht
into
th
e a
mo
un
t o
f n
ew
kn
ow
led
ge
/sk
ills
pic
ke
d u
p in
th
e t
rain
ing
fo
r e
ac
h c
ou
rse
(o
r c
urr
icu
lum
or
pro
gra
m)
tha
t th
e p
art
icip
an
t d
ire
ctly
ap
plie
d w
he
n t
he
y w
en
t b
ac
k o
n t
he
jo
b a
s th
e d
ata
is
co
llec
ted
se
ve
ral m
on
ths
aft
er
the
tra
inin
g w
he
n t
he
pa
rtic
ipa
nt
is
ba
ck
on
th
e jo
b.
34
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Per
form
ance
Impr
ovem
ent
Indi
cato
rs
Yo
u c
an
tra
ck
sp
ec
ific
bu
sin
ess
re
sults,
su
ch
as
pro
du
ctiv
ity
. A
skin
g a
ma
na
ge
r o
r a
pa
rtic
ipa
nt
to e
stim
ate
pro
du
ctiv
ity
on
ce
tim
e h
as
pa
sse
d o
n t
he
jo
b is
a g
oo
d in
dic
ato
r o
f le
arn
ing
’s im
pa
ct
to a
sp
ec
ific
bu
sin
ess
ob
jec
tiv
e.
35
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Val
ue C
reat
ion
Indi
cato
rs
Ho
we
ve
r y
ou
de
fin
e v
alu
e,
me
tric
s c
an
be
ga
the
red
to
art
icu
late
th
at
va
lue
. H
ere
we
lo
ok
at
tra
inin
g’s
co
ntr
ibu
tio
n t
o in
cre
ase
d jo
b p
erf
orm
an
ce
, b
usi
ne
ss r
esu
lts
imp
ac
ted
by
tra
inin
g, a
nd
a
fin
an
cia
l R
OI
fro
m t
rain
ing
.
36
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Gen
erat
e B
alan
ced
Sco
reca
rd to
Sha
re
with
Sta
keho
lder
sTh
e s
co
rec
ard
is
a g
rea
t m
an
ag
em
en
t to
ol
to s
ho
wc
ase
va
lue
to
ke
y s
tak
eh
old
ers
.
This
re
po
rt s
ho
ws
yo
u e
ac
h le
ve
l o
f le
arn
ing
an
d t
he
Ke
y P
erf
orm
an
ce
In
dic
ato
rs w
ith
in e
ac
h le
ve
l. A
t le
ve
l five
y
ou
ac
tua
lly p
rod
uc
e q
ua
ntita
tive
RO
I m
etr
ics
inc
lud
ing
:
Be
ne
fit
to C
ost
Ra
tio
Pa
yb
ac
k P
eri
od
RO
I%
37
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Con
tinuo
us Im
prov
emen
t to
Goa
ls
Allo
ws
you
to
est
ablis
h p
re-s
et g
oal
s fo
r ea
ch q
ues
tio
n c
ateg
ory
on
th
e P
ost
eve
nt
and
Fo
llow
u
p s
urv
eys
and
co
mp
are
them
to
act
ual
per
form
ance
.
38
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Ben
chm
ark
Inte
rnal
ly a
nd E
xter
nally
Co
mp
are
per
form
ance
res
ult
s (c
lass
, co
urs
e, c
urr
icu
lum
, pro
gra
m,
lear
nin
g p
rovi
der
, clie
nt,
bu
sin
ess
un
it, l
oca
tio
n, m
eth
od
olo
gy,
and
in
stru
cto
r) le
vels
wit
hin
an
d o
uts
ide
you
r o
rgan
izat
ion
to
mo
tiva
te b
y ex
amp
le.
39
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mon
itor
Qua
lity
Pro
activ
ely-
Ala
rms
MT
M a
uto
mat
ical
ly g
ener
ates
ala
rms
wh
en p
erfo
rman
ce f
alls
bel
ow
a
cert
ain
use
r-p
rese
t th
resh
old
fo
r an
y le
arn
ing
eve
nt.
Use
rs c
an
con
fig
ure
a m
inim
um
th
resh
old
fo
r al
arm
s an
d a
re n
oti
fied
wh
enev
er
per
form
ance
fo
r an
eve
nt
falls
bel
ow
th
at m
inim
um
. Ad
dit
ion
ally
,u
sers
can
sel
ect
wh
eth
er t
hey
wo
uld
like
to
rec
eive
ala
rms
via
e-m
ail.
Chi
cago
Pro
vide
r A
John
Sm
ithV
endo
r A
40
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Pro
vide
Rea
l-Tim
e F
eedb
ack-
Sum
mar
ized
Su
mm
ariz
es a
ll p
arti
cip
ant
resp
on
ses
for
a g
iven
cla
ss.
41
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Mea
sure
Aga
inst
the
Met
hodo
logy
Be
nc
hm
ark
ac
ross
th
e k
ey
pe
rfo
rma
nc
e in
dic
ato
rs
on
yo
ur
eva
lua
tio
n fo
rms
tha
t lin
k b
ac
k t
o t
he
5
lev
els
of le
arn
ing
me
asu
rem
en
t. C
om
pa
re t
he
a
ctu
al p
erf
orm
an
ce
to
go
als
an
d t
o in
tern
al o
r e
xte
rna
l b
en
ch
ma
rks
so y
ou
pro
pe
rly
mo
nito
r u
sin
g a
ba
lan
ce
d s
co
rec
ard
ap
pro
ac
h.
42
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Con
stan
tly A
sses
s N
eeds
, Driv
ers
of
Tra
inin
g an
d B
usin
ess
Gap
sA
su
ite
of
nee
ds
asse
ssm
ent
too
ls a
llow
ing
yo
u t
o c
on
du
ct g
ap a
nal
ysis
on
lear
nin
g a
nd
an
y o
ther
to
pic
. R
esp
on
den
ts r
ecei
ve
imm
edia
te f
eed
bac
k u
po
n c
om
ple
tin
g t
he
on
line
form
sh
ow
ing
wh
ere
they
alig
n t
o b
est
pra
ctic
es.
Sp
on
sors
of
the
asse
ssm
ent
rece
ive
agg
reg
ate
dat
a th
at c
an b
e b
ench
mar
ked
an
d q
uer
ied
as
wel
l as
tact
ical
re
po
rts
to h
elp
pin
po
int
gap
s an
d
op
po
rtu
nit
ies
for
imp
rove
men
t.
43
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
How
to Im
plem
ent A
naly
tics
•E
asy
to im
plem
ent (
a fe
w h
ours
to c
reat
e an
acc
ount
)•
Cos
t effe
ctiv
e (a
sm
all p
erce
ntag
e of
you
r tr
aini
ng b
udge
t)•
Leve
rage
tech
nolo
gy (
auto
mat
e da
ta c
olle
ctio
n, s
tora
ge,
proc
essi
ng, a
nd r
epor
ting)
•C
ompr
ehen
sive
(m
easu
re a
ll le
vels
of l
earn
ing
mea
sure
men
t 1 to
5)
•In
tegr
ated
or
stan
d al
one
44
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Bes
t Pra
ctic
es in
Lea
rnin
g M
easu
rem
ent
•P
lan
your
met
rics
befo
re w
ritin
g su
rvey
que
stio
ns
•E
nsur
e yo
ur m
easu
rem
ent p
roce
ss is
rep
licab
le a
nd s
cale
able
•E
nsur
e m
etric
s ar
e in
tern
ally
and
ext
erna
lly c
ompa
rabl
e•
Use
indu
stry
-acc
epte
d m
easu
rem
ent a
ppro
ache
s
•D
efin
e va
lue
in th
e e
yes
of y
our
stak
ehol
ders
•M
anag
e th
e ch
ange
ass
ocia
ted
with
mea
sure
men
t
•E
nsur
e th
e m
etric
s ar
e w
ell-b
alan
ced
•Le
vera
ge a
utom
atio
n an
d te
chno
logy
•C
raw
l, w
alk,
run
•E
nsur
e yo
ur m
etric
s ha
ve fl
exib
ility
Sou
rce:
CLO
Mag
azin
e, O
ctob
er 2
003
45
©2
00
3 K
no
wle
dg
eA
dvis
ors
. A
ll rig
hts
re
serv
ed
.
Con
tact
Info
rmat
ion
ww
w.k
now
ledg
eadv
isor
s.co
m
For
furt
her
info
rmat
ion
on th
e in
form
atio
n co
ntac
t:Je
ffrey
Ber
kV
ice
Pre
side
nt, P
rodu
cts
and
Str
ateg
y+
1 31
2 42
3 85
99 (
phon
e)+
1 31
2 37
2 00
44 (
fax)
jber
k@kn
owle
dgea
dvis
ors.
com
