From Critique to Collaboration: Rethinking Computerized Clinical AlertsDebaleena Chattopadhyay, Romisa Rohani Ghahari,

Jon D. Duke (Co-PI), Davide Bolchini (PI)

Presented by

Debaleena Chattopadhyay & Davide Bolchini

soic.iupui.edu

NSF Award #1343973

Drug-Drug Interaction (DDI) Alerts

© New Yorker

Drug-Drug Interaction (DDI) Alerts

Efficacy of DDI Alerts

How much did the efficacy of clinical alerts—especially DDI alerts—change in the last decade?

Reviewed 42,641 orders11% (4690) produced alerts

DDI alert override rate = 88%Allergy override rate = 69%

Reviewed 18,354 orders13% (2455) produced alerts

DDI alert override rate = 95%Allergy override rate = 91%

2002

2014

Efficacy of DDI Alerts

How much did the efficacy of clinical alerts—especially DDI alerts—change in the last decade?

It did NOT.

The Problem

How to improve the design of Drug-Drug Interaction Alerts (DDI Alerts) to improve physician’s adherence?

Motivation

Drug safety alerts are critical for patient safety but largely ignored by doctors during medication prescribing

Despite efforts to improve design and reduce alert fatigue, physicians continue to distrust computerized recommendations

Transforming the Outlook

• To improve alerts, we must first look at how to improve the trust between physician and computerized advice.

• We explore the foundational principles of what physicians consider important when taking advice from peers

• We use this knowledge to create novel designs for drug safety guidance that elicit physician trust and a sense of collaboration

So what? The Broader Impact

•Potentially reduce the over 2M adverse drug events per year by improving the safety of drug prescribing

•Translating findings into real-world EMR systems through the Regenstrief Institute

•Spread adoption to industry with EHR vendors and NIST

Our Approach

User Experience

requirementsdesign

prototypingevaluating

1

Mining Requirements

Formative studies in clinical settings to unearth key factors in sharing trusted advice among doctors when making prescribing decisions

1

Our Approach

User Experience

requirementsdesign

prototypingevaluating

2

Design and Prototyping

Design and deploy novel drug safety alert interfaces to convey drug safety information to providers in a more trusted manner.

2

Our Approach

User Experience

requirementsdesign

prototypingevaluating

3

Evaluation

Evaluation studies in the lab and in central Indiana hospital to assess the effect of the proposed designs on alert compliance

3

Our Approach

User Experience

requirementsdesign

prototypingevaluating

Requirement AnalysisUnderstanding Requirements for DDI Alerts

Requirements for DDI Alerts

• Before looking at how to improve alerts, we look at how to improve the trust between physicians and computerized advice.

• A starting point to address this issue is looking at whom physicians do trust: their medical colleagues and mentors.

TRUST

Understanding Advice Sharing among Physicians• In 3 contextual inquires, we examined why clinical

advice is trusted among physicians.

• 255 minutes – 22 health care professionals during three inpatient team meetings

The Flow Model

• Two primary functional roles emerged from the model—decision makers and decision implementers.

• Attending and residents primarily served as decision makers (e.g., “I think we should go ahead and do it.”).

• But when specialists or pharmacists provided the necessary advice, they acted as decision implementers (e.g., “Has she received IV iron? Renal [says] they recommend IV iron.”).

• Medical students (and sometimes pharmacist students) primarily acted as decision implementers. Although they actively took part in the decision making process, their decision was almost always endorsed or corrected by the supervisors (attending or residents).

The Cultural Model

• The cultural model identified a pervasive mindset of an inpatient team: a strict adherence to medical hierarchy and a strong preference for experts’ opinions.

• Inter-departmental relations influenced the team’s decisions to engage in soliciting consults from domain experts (e.g., “N-surg [neurosurgeons] didn’t put in clear recs [recommendations] for what they wanted to do. [..] Maybe rad-onc [radiology oncology], and n-surghave a better way to talk to each other because we don’t get calls from them.”).

The Cultural Model (cont.…)

• Attending and residents (supervisors in the flow model) completely influenced the activity of the medical students and interns (primary decision implementers in the flow model). But…

• Whereas the flow model uncovered a supervisor-supervisee relationship evident in an inpatient meeting, the cultural model surfaced an undertone—identifying it more closely to a mentor-mentee relationship.

• We observed a strong influence of expert opinions on the team’s decisions

The Emerging Themes

Informed by our consolidated work models, we identified eight themes driving trusted advice among physicians in clinical settings

1. Specialization

2. Role in the Medical Hierarchy

3. Demonstrated Experience

4. Evidence of Understanding the Patient’s Situation

5. Empathy

6. Demonstrated Knowledge of Evidence from the Literature

7. Collaborative and Inclusive Language

8. Timeliness of the Advice

Validating Themes with Survey

• To validate with a larger sample of physicians the crucial themes emerging from our formative study, we designed and administered an online survey.

• Demographics: Of the 87 questionnaires sent, 37 were returned (22 females). Respondents were mostly less than 30 (17) or less than 40 (11) years old, and were mostly either resident (19) or attending physicians (17).

• 17 physicians worked less than five years while four worked for more than 25 years. Except four respondents, all physicians currently worked in an inpatient environment, and 20 physicians spent more than 50% of their time in an inpatient environment.

Connecting Themes to Survey Questions

Survey Results

Recommending colleagues’ hierarchical role significantly affected how much a second-year resident would trust or follow their recommendation. ID (infectious disease) consultant would be significantly more trusted than the primary intern.

Survey Results (cont.…)

When faced with conflicting recommendations coming from peers, a second-year resident would trust and follow a curbside consult from Hospitalist than one from the new ID fellow.

Survey Results (cont.…)

Themes emerging from our contextual inquiry significantly affected the likeliness of a second-year resident to trust or follow a recommendation. Specialization would be trusted and followed significantly more than role in medical hierarchy.

Survey Results (cont.…)

A treemap visualization showing the emerging nodes by number of coding references. The size of the rectangles represents the number of coding references. Self-reference was the most commonly coded frame of reference and reflective was the most commonly coded tone of communication.

Design Directions for Trust-Based Alerts• Endorsed alerts

• Transparent alerts

• Team-sensing alerts

• Collaborative alerts

• Empathy-driven alerts

• Conflict-mitigating alerts

• Agency-laden alerts

Chattopadhyay, D., Rohani Ghahari, R., Duke, J., D., & Bolchini, D. (2015). Understanding Advice Sharing among Physicians: Towards Trust-Based Clinical Alerts. Interacting with Computers (In review).

Designing Clinical AlertsToward a Model for Trusted Clinical Alerts

Work-in-Progress…

Endorsed Alerts

• Endorsed alerts

Steve Nissen, MDChair of Cardiology

ApprovedAlert

Transparent Alerts

Addendum: AMOXICILLIN 500MG. Allergy Alert Override by Smith, JD. 11/14/2014 at 8:31am.

Embed in Chart

Empathy-Driven Alerts

AllergyAllergy Warning

Next Steps

• Detailed Design of trusted DDI alerts.

• Low-fidelity, interactive prototyping of DDI alerts.

• Feedback from physicians on the DDI alerts to understand their responses.

Thank you!

bit.do/trusted-alerts

http://mypage.iu.edu/~dbolchin/trusted-advice.html

{debchatt | dbolchin} @ iupui.edu

Debaleena Chattopadhyay

Romisa Rohani Ghahari

Jon D. Duke (Co-PI)

Davide Bolchini (PI)

This research is based upon work supported by the National Science Foundation under Grant Number IIS-1343973. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

soic.iupui.edu