Explorations in Internet-Based Collaborative Informatics Research: A
Cognitive Evaluation
Vimla L. Patel, PhD, DScDepartments of Medical Informatics and Psychiatry
College of Physicians and SurgeonsColumbia UniversityNew York, New York
Asynchronous Synchronous
• Telephone• Group Conference Calls• Video Conferencing• Face-to-Face Meetings
• Email - personal• Email Dist’n Lists• World Wide Web• Regular Mail
Communication
McGillEvaluation
Studies
STANFORD COLUMBIA
BWH - HARVARD MGH - HARVARD
• Academic Research and Practice• Graduate Degree Program• Clinical Information Systems• Medical Logic Standards• Decision-Support Systems• Clinical Workstations• Knowledge-Based Vocabularies• Cholesterol Guidelines
• Academic Research• Graduate Degree Program• Knowledge Representation• Advanced Server Technology• Decision-Support Systems• Knowledge Acquisition• Clinical Trials Workstations• Breast Mass Protocols
• Academic Research• Medical Informatics Training• Component-Based Software• Clinical Workstations• Integrated Guidelines• Decision-Support Systems• Knowledge Authoring Tools• Breast Cancer Guidelines
• Academic Research and Practice• Integrated Clinical Training• Decision-Support Systems• Clinical Workstations• Integrated Guidelines• Clinical Information Systems• Electronic Medical Records• Flu Vaccine Guideline
Shortliffe Cimino
BarnettGreenes
Patel
Sources of Data
• Electronic Interviews• Audio-Taped Conference Calls• Email
– Current & Archived• Progress Reports• Papers, Presentations, and Proposals• Developments at Intermed WWW Sites• WWW Access Statistics
– Access of Resources, Documents, etc..
– File Transfers Between Sites• Audiotapes of Mini Workshop(s)
Perspectives on Collaboration: E-Mail Interview
PLEASE COMMENT ON THE FOLLOWING TWO STATEMENTS:
a InterMed is a Highly Integrated Collaborative Endeavor with Largely Shared Assumptions. Work on Common Projects Require Shared Efforts and a High Degree of InterPersonal and Intergroup Integration
b InterMed is a More Loosely Coupled Cooperative Project which Each Team Provides Shared Expertise and Resources towards some Common Goal and Towards Institution-Specific Goals. Progress is Largely a Function of Individual Intiative Efforts, which are Later Coordinated within the Larger InterMed Context.
Modes of Communication Best Suited To Specific Purposes
Individual
A
B
C
D
E
F
G
Plan
CC
CC
CC
CC
CC
FTF
CC
Present
WWW
WWW
WWW
FTF
WWW
EM
WWW
Brainstorm
FTF
CC
EM
EM
CC
FTF
FTF
Conflicts
FTF
EM
CC
CC
CC
CC
FTF
Technical
EM
EM
EM
EM
EM
EM
EM
Critique
EM
EM
EM
EM
EM
EM
EM
STANFORD
COLUMBIA
DSG
PURPOSE
Categories of Activity During Conference Call
ACTIVITY Feb. 7 Feb. 21Total Time of Call 68'45" 31'35"Number of Participants 20 14Active Participants (5 Turns) 9 9Number of Turns 455 253Alternatives 17 15Clarification 193 157Criterion 2 4Goal 3 0Issue 45 18Meeting Management 14 11Project Management 74 22Summary 2 0Walkthrough 2 0
Frequency of Guideline-RelatedFrequency of Guideline-Related Email Messages per WeekEmail Messages per Week
WEEKS FROM JULY 1994 TO APRIL 1996July 94 April 96Jan 95 July 95 Dec 95 Feb 96
Numberof
Emails
0
5
10
15
20
25
30
35
40
451 32 46 49 54 65 68 76 80 82 84 86 88 90 92 94
Voca
bula
ry G
uide
lines
Exp
.
HPC
C M
eetin
g,C
omm
on G
uide
lines
Rep
rese
ntat
ion
Inte
nsiv
e W
ork
Beg
ins
onG
uide
lines
Mod
el/D
etai
led
Age
nda
AM
IA F
all C
olla
bora
tive
Subm
issi
ons
Min
i Wor
ksho
p on
Gui
delin
es
37
35
33
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
11 3 5 7 9 11 13 15 17 19 21 23
Sequence of Goals in Progress Report
Specific Goals
In Progress
Complete
MONTHS FROM JULY 1994 TO APRIL 1996July 94 April 96
Common Guideline Model
Adapt And Display Guidelines on WWW
PrototypicalGuideline/ WWW CGI
C
A B E
G
F
H
J
I ML
D
K
Sociometric Analysis of Email Communication Patterns
Stanford
DSG
Columbia
MGH
InterMedCentral
Stanford Columbia
MGH
C
AB E
F
ML
D
K
Email Communication Patterns (Jan-Feb 96)Email Communication Patterns (Jan-Feb 96)Episode 1: Vocabulary Server and Guidelines Episode 1: Vocabulary Server and Guidelines
InterMedCentral
Stanford
BWH
Columbia
MGH
C
AB E
G
F
H
J
I
D
K
Email Communication Patterns (Feb-Mar 96)Email Communication Patterns (Feb-Mar 96)Episode 2: Guidelines RepresentationEpisode 2: Guidelines Representation
InterMedCentral
Conclusions• Significant Evolution in Communication
Patterns–Greater Degrees of Integration–Focused Communication
• Increased Use of Richer Representations –Diagrams, WWW
• Integrated Plans and Coordinated Actions• Sustained Collaboration Requires Sustained
Commitment• Different Modalities — Cognitive & Social
Consequences• Learning Curve in Collaboration
• Collaboration is difficult!