The ABC’s of ROI(Return on Investment)

John GillespieLaerdal Medical

Welcome to the Indianapolis SUN!

Traditional ROI is:

Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp

“…performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.” (“Investopedia,” n.d.)

Traditional ROI is:

Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp

“…if an investment does not have a positive ROI, or if

there are other opportunities with a higher ROI, then the

investment should be not be undertaken.” (“Investopedia,” n.d.)

“…performance measure used to evaluate the efficiency of

an investment or to compare the efficiency of a number of

different investments.” (“Investopedia,” n.d.)

Why is ROI important???

• Sustainability of your simulation program

• Help insure that the training dollars are providing value to the

organization• Help insure that the learner is receiving value for time spent in

training

Was/is this a:

• Thoughtful investment?

• Thoughtless investment?

Or

Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html

Remember this:

“…hospitals typically measure ROI from a business perspective—cost, revenues or operating efficiencies—but many benefits of clinical applications fall into quality and safety realms that do not easily translate into dollars.” (Page, 2010)

Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app/jsp/articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG

Four areas of focus:

• Organizational Buy In

• Overcoming Barriers to Simulation

• Faculty/Personnel Development

• Curriculum Integration

Organizational Buy In

• ROI (Return on Investment)

• ROE (Return on Expectation)

• Utilization Data Collection

• Evaluation Data Collection

– Leading indicators are usually pretty close to accurate

• Policy and Procedures that are specific to your organization

• Care about what keeps your organizational leaders up at night

• Create a value statement– Insure that the stakeholder gets a regularly

scheduled report

But what about ROE?

Main principles

• “Business/[organizational] objectives are seen as a starting point.” (“Trainingcheck.com,” n.d.)

• “’Return on Expectation’ (ROE) is key” (“Trainingcheck.com,” n.d.)   

                 • “Collective efforts are needed throughout an

[organization] to achieve success.” (“Trainingcheck.com,” n.d.)   

Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation/kirkpatrick-ROE-model-of-training-evaluation/

Return on Expectation

• “Proactive, business partnership approach that unified teams.”

• “Defines training as a contributor to key business results.”

• “Value defined by business stakeholders in cooperation with training.”

• “Focus on comprehensive evidence and a compelling story of value.”

• “Easy to understand, flexible, and cost-effective.”

Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retrieved from http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm

Data Collection

• To prove either ROI or ROE, you must have data to base your return on.

Types of data collection to support ROI & ROE

• ROI in Healthcare- Using data to demonstrate a positive impact on the organization through the use of simulation– This can include unexpected results

• ROI in operations- Use, or utilization– Tool for determining when you are under

utilizing/over utilizing your resources• ROI in customer service- Were our customers satisfied

with the simulation experience?– Can become a tool for measuring the effectiveness of

your instructors/simulations• ROI in academic learning- Assessment

– Process driven checklist type of scoring

Measuring Outcomes

• The effectiveness of your program is only as relevant as the effectiveness of the tool(s) that you are using to measure it by.

• Using the Kirkpatrick model, as one example, you can and should measure the effectiveness of the learning objectives met right after the simulation and then again 6 months later.

• Is the behavior reflecting what was learned during simulation (Debriefing)? Report this back to the stakeholder. Are you indicating a beneficial return on investment?

Monthly Log

WakeMed-Amar Patel

Monthly Log

WakeMed-Amar Patel

Monthly Log

WakeMed-Amar Patel

Monthly Log

WakeMed-Amar Patel

Month End Data

WakeMed-Amar Patel

First Quarter - Raw

WakeMed-Amar Patel

First Quarter - Graphed

WakeMed-Amar Patel

Targets

WakeMed-Amar Patel

How does this data help?

WakeMed-Amar Patel

An actual data collection tool……

Back to the ROI…..

How is simulation use valuable?

• Value 1- Simulation is only valuable if you use it. – A simulator that is unused/under

utilized has no value. • Worse, it is a cost center and a

financial drain on the organization.

How is simulation use valuable?

• Value 2.- Educationally effective simulation not only allows the learners to “connect the dots”, but it also can change behaviors in an organization.– The result can allow you to focus your

training dollars to effect– Reduce your malpractice costs– Reduce time spent in training– Improve patient outcomes

How is simulation use valuable?

• Value 3.- Use as a recruiting tool to draw in candidates that increase the overall value of your organization.

How is simulation use valuable?

• Value 4.- Repeatability regardless of the patient census at any given moment. – Allows for a consistent product to be

delivered to the learners.

References

Best Practice Committee. (2002). The Value of IT investments: It’s not just return on investments. Retrieved from http://www.cio.gov/documents/thevalueof_it_investments.pdf

Big Dog & Little Dog’s Performance Juxtaposition. (n.d.). Kirkpatrick’s four-level training evaluation model. Retrieved from http://www.nwlink.com/~donclark/hrd

/isd/kirkpatrick.html

Cooke, M., Irby, D., Sullivan, W., & Ludmerer, K. (2006). American medical education 100 years after the Flexner report. The New England Journal of Medicine, 355(13), 1339-1344.

Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp

Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retrieved from

http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm

Markow, M. (2011). Business terms glossary. BusinessWings. Retrieved from http://www.businesswings.co.uk/articles/Business-terms-glossary

Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app/jsp/articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG

Robert Wood Johnson Foundation. (2011). Glossary of healthcare terms. Quality/Equality. Retrieved from http://www.rwjf.org/qualityequality/glossary.jsp

SBA.gov. (2011). Essential elements of a good business plan. Retrieved from http://www.sba.gov/category/navigation-structure/starting-managing-business

/starting-business/writing-business-plan/essential-elements-good-busines

Smith, R. (2009). Embracing game technology for medical education. Retrieved from http://www.peostri.army.mil/CTO/FILES 2009_RSmith_MT3.pdf

Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html

Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation/kirkpatrick-ROE-model-of-training-evaluation/

Research in Simulation: Using Data to Help Improve a Return on Investment Amar P. Patel, MS, NREMT-P, CFC Laerdal SUN Conference Foxwoods 2012