1

Transforming EHR Interactions Using Voice Assistants

Session 164, February 13, 2019

Yaa Kumah-Crystal, MD MPH

Timothy Coffman

Vanderbilt University Medical Center

2

Yaa Kumah-Crystal, MD MPH

Discloses the following relevant relationship with commercial interests:

• Research partnership with Nuance Communications

• Use of NLP libraries for research and development

Conflict of Interest

3

Timothy Coffman

Discloses the following relevant relationship with commercial interests:

• Research partnership with Nuance Communications

• Use of NLP libraries for research and development

Conflict of Interest

4

• Principles employed in creating a voice user interface for the electronic health record

• Challenges faced in data capture

• Challenges faced in data presentation

• Assessment

• Implementation and future plans

Agenda

5

• Discuss the value of artificial intelligent (AI)-enabled natural language processing and understanding technologies in the advancement of healthcare

• Discuss the limitations of current keyboard and mouse workflows in the EHR

• Describe the usability design considerations, information request mapping and artificial intelligence/natural language understanding platform built to process voice requests

• Discuss the provider cohort experience and outcomes around satisfaction

Learning Objectives

6

Informal Poll

Own a smart phone

Use your smart phone voice assistant

• Siri, Google assistant, Bixby

Own a voice assistant device

• Alexa, Google home, Apple Homepod

7

Why Voice?

8

Documentation Modalities

Writing

~13

WPM

Typing

~40

WPM

Speech

~150

WPM

9

Why not Voice?

10

Macros

Physical exam

Insert normal physical exam

11

Vanderbilt Innovations Portfolio Research

12

Given

• Installed EHR

• EHR vendor has existing partnership with natural language

services provider (including dictation)

• Existing in-house innovation capacity– Infrastructure supporting custom development

– Team in place with necessary SW development skills

– Mission to develop expertise in new ways to interact with the EHR

Decision

• Develop new software to integrate EHR with additional natural

language services

Solutions

13

Well really, how how hard can it be?

13

14

Oh...

15

Demonstration

16

17

VEVA Processing

NLP

User Query

Speech to text

Intent Fulfillment Text to speech

Results Display

What is her A1C?

Sally’s A1C is 8.6

18

VEVA Architecture

19

• EHR’s FHIR implementation does not yet support all desired data

• Additional development required for each new query

• New phrasings of questions requiring updates to language model

• Microphone and speaker issues in a virtual desktop environment

• Latency

Technical Limitations

20

Focus on developing demonstration of capabilities supporting

• One (1) specialty clinic

• Handful of general responses

• Aligned with FHIR (DSTU2) resources

• Some EHR-proprietary services

Build in capability to extend with

• Additional capabilities for other specialties

• General responses, discovered through user feedback

• FHIR (STU3) and FHIR (R4) resources as available

• FHIR-based alternatives to proprietary services

Planning for Extension

21

• MVP

• Rapid Iteration

• Vanderbilt as a Design Shop

Technical Considerations

22

Integrating voice response involves real-time services

• Audio streaming capabilities

– Virtual desktops introduce challenges

• Automatic connections

• Latency

– Varying hardware installed at point-of-use

• Individual laptop microphones

• High-end dictation microphones

• All of the above

Infrastructure Challenges

23

• “While voice is a natural interface for humans, this may be the first

time we are using it as an interface to have a computer

communicate back to us like this. Different rules may apply.”

Design Considerations

24

• Brevity vs. Clarity

• “Blood pressure is 120 millimeters of mercury over 80 millimeters of mercury ”

• “Blood pressure is 120 over 80”

• Uncanny Valley of Words

• I ate three big red apples

• I ate red big three apples

• Quantity, Opinion, Size, Age, Shape, Color, Material, Purpose

Design Considerations

25

Use Cases

26

• Preparing to see a patient in a provider workroom

– Large screen

– PHI private

Use Case A

27

• Reviewing information with patient present

– Large screen

– PHI with patient 3rd party

Use Case B

28

• In transition in a private environment (commute)

– Small Screen (mobile)- limited visibility

– Primarily verbal navigation

– PHI private

Use Case C

29

• In transition in a public environment (hallway)

– Small screen (mobile)

– PHI considerations

(aloud only with headphones?)

Use Case D

30

Usability Study

31

• Setting: Vanderbilt University Medical Center

• Cohort: 14 Pediatric Endocrinology Providers

• Female (10) | Male (4)

• Mean age 39, range 29-65

Study Design

32

• Usability Assessment

• 6 Skills Evaluated

– Training: Hello / What can you do

– Summary

– A1C

– Weight

– Blood Pressure

– Health Maintenance

Study Design

33

• Value of information communicated

• Length of content communicated

• Value of graphical display

• Overall thoughts about voice interactions

Feedback

34

System Usability Scale: Positively Worded

Strongly Disagree

Mean 4.12 (SD .46) 3.93 (SD 1.20)

4.50 (SD .52)

3.79 (SD .70)

4.00 (SD .79)

4.43 (SD .51)

Strongly Agree

35

System Usability Scale: Negatively Worded

Mean 1.67 (SD .66)

Strongly Disagree Strongly Agree

1.36 (SD .84)

1.71 (SD .83)

1.79 (SD 1.19)

1.64 (SD .74)

1.86 (SD 1.03)

36

• Mean System Usability Scale score 80.7 (highly usable)

• Scores of >68 are considered above average

• Summary

• Relevant medications, other problems

• Weight

• Growth chart

• Blood pressure

• Percentile information

User Feedback

37

Willingness to Use System

36%64%

NoYes

4 count9 count

2

1

1

38

• Incorporation of User Feedback

• Desktop Time Comparison

• Scalability

• Alerts and Decision Support

Next Steps

39

• Understand and measure provider frustration with EHR

• Create the business case for optimizing their experience and saving providers time within their EHR workflow

• Assemble a cross-functional team to build the overarching model taking into consideration the providers’ pain points, workflow, and information needs

• Build and iterate a voice assistant prototype while users test and provide feedback

• Understand information theory and map queries and content to satisfy users’ needs

Best Practices/Recommendations

40

An Interesting Study About Voice

Nationwide Children's Hospital in Columbus, Ohio

41

It looks like you have a question.

Yes Go away Clippy!

Questions

Please remember to complete the online session evaluation

42

Contact Us