Design and evaluation of an interactive proof-of-concept dashboard for general practitioners

DESIGN AND EVALUATION OF AN INTERACTIVE PROOF-OF-CONCEPT

DASHBOARD FOR GENERAL PRACTITIONERS Robin De Croon, Joris Klerkx, Erik Duval

[email protected]

IEEE International Conference on Healthcare Informatics 2015, Dallas

Patient follow-up

Patients• forget follow-up meetings• do not consider a follow-up necessary• do not want to spend additional money• have the impression treatment is not working

General practitioners• hard to find patients in need of follow-up• are too busy to accommodate a prompt visit• do not have enough time

Wednesday, October 21, 2015 2

How to improve follow-up quality?

Patient oriented

• Stimulate patient empowerment

General practitioner driven

• Provide better tools to augment workflow


Wednesday,October21,2015 4

{=,<,>,>=,<=,<>}|value|{and,or}|value

Research question

“How do we help general practitioners identify patients in need of follow-up using interactive visualizations?”


Research question



Research question



Research question



User centered, rapid prototyping


Initial design

Prototype 2- Content- Missing data

ResultFinal Design

- Updated table- Consistency- bugs

Prototype 3- Internal changes

Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs

design phase usability phase evaluation phase



Initial design


ResultFinal Design





Visualization concepts

• Overview first, zoom and filter, then details-on-demand

• Data ink ratio

• Stevens’ model


BenShneiderman, TheEyesHaveIt:ATaskbyDataTypeTaxonomyforInformationVisualizations.In ProceedingsoftheIEEESymposium onVisual Languages, pages336-343,Washington.IEEEComputerSocietyPress, 1996

EdwardTufte. TheVisualDisplay ofQuantitativeInformation. 1983

S.Stevens, “Onthetheoryofscalesofmeasurement.”Science(NewYork,N.Y.),vol.103,no.2684,pp.677–680,1946

Demographics

Wednesday, October 21, 2015 12http://www.cdc.gov/nchs/data/databriefs/db81.htm

Populationpyramid Mostcommondiseases Religion

Highestdegree Maincaregiver

#dayssince lastvisit income systolicbloodpressur diabolicbloodpressure weight sugar age

Picture Name Condition Gender Age Degree Maincaregiver

Parallel coordinates


https://syntagmatic.github.io/parallel-coordinates/



Initial design


ResultFinal Design





Academic Center for General Practice

What• 3 general practitioners + 2 electronic medical record experts

• already perform audits on general practices

• Perceived usefulness

Result

• Too much demographics• Medication (groups) more important

• primary and secondary condition

• Deal with noise in data• outliers, missing elements




Initial design


ResultFinal Design





Usability iteration

What• 12 information visualization students• 10 tasks

• time-to-task & error-rate & perceived difficulty

• ± 60 minutes• Questionnaires

• initial & closing & System Usability Scale


Results usability evaluation

• System Usability Score: 68

• 7/10 tasks < 60s

Visual Updates:

• Visual update

• Consistent navigation

• Patient table




Initial design


ResultFinal Design





Final evaluation

• 9 general practitioners• 7 male + 2 females

• Concurrent Think Aloud protocol• short introduction, free interaction, 15 ~ 20 minutes

C. Lewis, Using the "thinking Aloud" Method in Cognitive Interface Design, (ibm resea ed., ser. Researchreport. Yorktown Heights, NY: IBM T.J. Watson Research Center

• System Usability ScaleBrooke, John. "SUS-A quick and dirty usability scale."Usability evaluation in industry 189.194 (1996): 4-7

• Questions from O’Leary et al. à likert questionsO’Leary, P., Carroll, N., & Richardson, I. (2014). The Practitioner’s Perspective on Clinical Pathway Support Systems. In 2014 IEEE International Conference on Healthcare Informatics (pp. 194–201)

• SWOT analysis


System Usability Scale

Wednesday, October 21, 2015

74

Bangor,A.,Kortum,P.,&Miller,J.(2009).Determiningwhatindividual SUSscoresmean:Addinganadjectiveratingscale.JournalofUsabilityStudies, 4(3),114–123

System Usability Scale


1 2 3 4 5

Learn a lot of things

Confident using the system

Very cumbersome to use

Learn to use this system very quickly

Too much inconsistency

Functions well integrated

Need the support of a technical person

Easy to use

Unnecessarily complex

Like to use this system frequently

Likert questions


1

1

2

3

5

1

3

6

4

2

2

3

3

5

6

3

3

4

2

5

2

4

2

0% 20% 40% 60% 80% 100%

Need for follow-up tool

Follow best practice

Train GPs

Correct level of detail

Recognize follow-up patients

Reducing mistakes

Useful feedback

Right kind of tool

totally disagree disagree neutral agree totally agree

O’Leary, P., Carroll, N., & Richardson, I. (2014). The Practitioner’s Perspective on Clinical Pathway Support Systems. In 2014 IEEE International Conference on Healthcare Informatics (pp. 194–201)

SWOT analysis

2722

34

34

5

2

3

4

4

2

2

3

3

4

5

Control systemPrivacy

Too little timeAverages can be dangerous

Can be improved with patient collected dataTriggers self-reflection

Ideal for research

Not needed oftenNot clear which content too show

Not much structured data availableMap uses too much screen estate

No pseudo code neededImproves team communication

Visual overviewAugment work

Check with guidelinesEase to select patients

Technologies


Limitations

• Sample size• total 15 GPs à exploratory study

• Semi realistic data• anonymous & assess perceived usefulness

• Requirements gathering• user centered, rapid prototyping

• Evaluation setting• limited time à perceived usefulness


Conclusion

• Powerful, yet easy to use

• Reduces burden to analyze patient records

• Recognize patients in need of follow-up

• Query patient with visual filters

• Needs to adapt to specific use cases

• Biggest opportunity is in research


http://cdn.makeuseof.com/wp-content/uploads/2012/12/3D-Man-Presenting-Intro-Image.jpg?a53b57

Looking for collaborations!

• Make the dashboard available in general practice• evaluate impact on collaboration• evaluate if self-reflection is triggered

• Continue working with academic GPs


Acknowledgements


Research grand: IWT 120896

Access to their products and expertise

For their experienced feedback

+ All participants!

Thank you!


http://2.bp.blogspot.com/-gZjNR3XVULs/T_ZOVgE-5lI/AAAAAAAAAg8/6YVmd5Q064o/s1600/questions11.jpg

[email protected]

information seeking mantra

34

BenShneiderman, TheEyesHaveIt:ATaskbyDataTypeTaxonomyforInformationVisualizations.In ProceedingsoftheIEEESymposiumonVisual Languages, pages336-343,Washington.IEEEComputerSocietyPress, 1996.

data ink ratio


EdwardTufte. TheVisualDisplay ofQuantitativeInformation. 1983

target user

Target Audience


• General Practitioners (in Belgium)

Independent of:

• Experience & age

• Individual or group practice

• ICT-knowledge

• Current medical software

http://marketingyoucanuse.com/wp-content/uploads/2010/12/HittingTarget.jpg

Related work: Eventflow


Monroe, M., Lan, R., Plaisant, C., Shneiderman, B. (May 2013) Temporal Event Sequence Simplification In IEEE Trans. Visualization and Computer Graphics, 19, 12 (2013), 2227-36. HCIL-2013-11

AS-IS

• Audit by Academic Center for General Practice• perceived as control

• Limited tools available:

Wednesday, October 21, 2015 38query window from Medidoc

Date post:	16-Jan-2017
Category:	Healthcare
Upload:	robin-de-croon
View:	350 times
Download:	0 times