Use of Online Resources While Using a Clinical Information System

James J. Cimino, MD; Jianhua Li, MD;Mark Graham, PhD, Leanne M. Currie, RN, MS;

Mureen Allen, MB BS; Suzanne Bakken, RN, DNSc, Vimla L. Patel, PhD

Department of Biomedical InformaticsColumbia University

Categories of Generic Question

Classification

Information Needs Study

Study 2Identify information

resourcescurrently being used

Characteristics of Resource Use

2ALog file analysis

2BSurveyCharacteristics of

Information Needs Events

Observational Study

Study 1AIdentify

information needs events

Study 1BClassify

Information Needs Events

Method Developme

nt

Understanding Resource Use

• Are clinicians using resources to answer patient-related questions?

• What resources do clinicians use?

• Do they use resources differently depending on the context?

Methods

• Log file captures clinician use of resources– Records user, function, location– User type determined from user directory– Context of use can be inferred

Logging Information Resource Use

WebCIS Logon Screen

HealthResources

Page

HealthResource

SystemLog

(anonymous)Health

ResourcesPage

HealthResource

WebCISFunctions

Logon

Log File Examples16:14:38|Resource|nobody|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs

16:16:46|WebCIS|ciminoj|156.1.5.1|lab^2003-03-02-05.19.00.819650

16:18:55|WebCIS|ciminoj|156.1.5.1|healthresource

16:21:18|Resource|ciminoj|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs

16:24:30|WebCIS|ciminoj|156.1.5.1|lab^1997-09-02-15.54.00.359121

16:25:12|WebCIS|ciminoj|156.1.5.1|pubmed

16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex

16:32:38|WebCIS|ciminoj|156.1.5.1|phar

16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex

Log File Results

• Six months of log files

• 38,763 health resources– 14,036 anonymous health resources– 19,913 health resources from WebCIS– 4,814 infobuttons

• 2,607 users– 51% resident physicians– 34% attending physicians– 5% nurses– 10% other (pharmacists, administrators, etc.)

Information Resource Usage

6%

4%

4%

4%

51%

3%3%7%

6%

1% 1%10% MicromedexPharmacy IB

Sensitivity IBCulture IB

Ovid MedlineOnline FormularyHarrison's

PubMedJournals

MedlinePlusICD9-CM

Other Resources

12% IBs

Information Use Contexts

9%

7%3%

2%

2%

2%

24%

51%

Laboratory

Radiology

Clinic Notes

Pathology

Visit Lists

Pharmacy

Discharge Sum.

Other

Context-Specific Resource Use

0%

20%

40%

60%

80%

100%

Sensitivity IB

Culture IB

Pharmacy IB

Medlineplus

ICD9-CM

Journals

PubMed

Harrison's

Formulary

Ovid Medline

Micromedex

Methods

• Log file captures clinician use of resources– Records user, function, location– User type determined from user directory– Context of use can be inferred

• Log file captures clinician use of resources– Records user, function, location– User type determined from user directory– Context of use can be inferred

• Questionnaire to determine:– Are they using resources through our system?– Why do they use resources?– What resources do they use?– What resources do they like?– What other resources do clinicians use?– Do survey results match log files?

Survey Population

• 1055 resources users in 01/03

• 250 randomly surveyed

• 45 (18%) returned– 23 attendings– 19 housestaff– 3 nurses– 1 pharmacist

Using Resources through WebCIS

• 14 anonymous only *

• 17 in WebCIS only

• 11 both

• 3 neither *

* All used resources in WebCIS

Why do they use resources?

• 43 for patient-specific issues

• 23 for general knowledge

• 3 for other reasons– writing– teaching– studying

What resources do they use? Like?

• No marked difference between user types

• Log files corroborated usage claims

• 26/45 said “do not use” (although 38/45 used)

0

5

10

15

20

25

30

35

40

Not Helpful

Somewhat Helpful

Very Helpful

What about other resources?• Reources used:

– Up To Date– MD Consult– New England Journal of Medicine– Epocrates– Google– Books

Reasons for use:28 for patient-specific issues25 for general knowledge1 for teaching

Lessons Learned• Log files representative of resource use:

– Our users tend to use “Health Resources” page– Surveys reflect observed usage

• Micromedex is most popular resource• Other resources needed (e.g. UpToDate)• Resource use is for patient-specific reasons• Most resource use is within patient context• Laboratory results are most popular context• Context does influence resource use• Infobuttons actually get used!• Infobuttons can be preferred!

Information Needs Study

Study 2Identify information

resourcescurrently being used

Characteristics of Information Needs

Events

Characteristics of Resource Use

Categories of Generic Question

2ALog file analysis

Infobutton

Manager

2BSurveyObservatio

nal Study

Study 1AIdentify

information needs events

Study 1BClassify

Information Needs Events

Classification

Method Developme

nt

Where Do We Go From Here?

• Up To Date has been added

• Infobutton Manager matches context to questions

• Focus on laboratory-related questions

• Focus on ways to exploit Micromedex

• Extend Infobutton Manager to other institutions

www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html

Conclusions From the Three Studies• Information needs arise while using clinical systems• Needs can be detected and characterized• Information needs are context-specific• Information needs are stereotypic• Satisfaction is often unsuccessful or deferred• Resource use is context-specific• Infobuttons are a viable way to resolve needs

– Offer context-specific retrievals for specific needs– Provide shortcut to popular resources

• Infobuttons are an institution- and application-independent way to improve resolution of information needs and reduce medical errors

Acknowledgments

• National Library of Medicine research grant

• National Library of Medicine training grant

• Rick Gallagher for log file support

www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html