Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | aidan-crowley |
View: | 213 times |
Download: | 1 times |
Seeking Synchronicity: Evaluating Virtual
Reference Transcripts
Presented by Lynn Silipigni Connaway
OCLC Members Council February 14, 2006
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
• $1,103,572 project funded by:
– Institute of Museum and Library Services $684,996 grant
– Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions
Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives
Project duration10/1/2005-9/30/2007
Four phases:I. Focus group interviews*II. Analysis of 1,000 QuestionPoint
transcriptsIII. 600 online surveys*IV. 300 telephone interviews*
*Interviews & surveys with VRS users, non-users, & librarians
Phase II:24/7 Transcript Analysis
• Generated random sample– July 7, 2004 through June 27, 2005– 263,673 sessions– 25 transcripts/month = 300 total
• 256 usable transcripts – Excluding system tests and technical problems
5 Analyses
• Geographical Distribution• Type of Library• Type of Questions
• Katz/Kaske Classification • Subject of Questions• Interpersonal Communication
• Radford Classification• Manual and NVivo coding
110, 41%
12, 4%
105, 38%
40, 14%
8, 3%1, 0%
East
Midwest
West
Australia
Canada
England
Geographical Distribution of Question Origination
Type of Library
88
76
51
38
24
12 10
10
10
20
30
40
50
60
70
80
90
100
Consortium Public University National Law Unclear State K-12
6%0%
25%
2%
30%
37%
Holdings
Inappropriate
Procedural
Research
ReadyReference
Specific Search
Type of Questions
Combined 1st and 2nd Questionsn=273 questions
Subjects of Questions64
35
3027
20
1511 11 11
9 9 8
0
10
20
30
40
50
60
70
1
Procedural
Law
General/Multidiscilinary
Science
History
Business
Education
Literature
Biography
Social Science
Recreation
Geography
Combined 1st and 2nd Questionsn=273 questions
Classification Methodology
• Qualitative Analysis
• Development/refinement of category scheme
• Careful reading/analysis
• Identification of patterns
Time intensive, but reveals complexity
Research QuestionsInterpersonal Communication Analysis
• What relational dimensions are present?
• Are there differences in relational dimensions/patterns of chat users & librarians?
• If so, what are they?• How do users & librarians compensate for lack of nonverbal cues?
• What is the relationship between content & relational dimensions in determining quality?
2 Major Themes
• Relational FacilitatorsInterpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication.
• Relational BarriersInterpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication.
Results Interpersonal Communication Analysis
Manual Coding Results
• 200 Transcripts– 177 Usable Transcripts
Librarian Relational Factors (1-200)
Librarian Relational Facilitators:Manual Analysis
132
8674
64
41
0
20
40
60
80
100
120
140
160
180
200
LF 1 RapportBuilding
LF 2 Deference LF 5 ClosingRitual
LF 4 GreetingRitual
LF 3 Rep. OfNon-Verbal
Cues
n=177 transcripts
121102
62
4027
0
20
40
60
80
100
120
140
160
180
200
CF 1 Deference CF 2 RapportBuilding
CF 3 ClosingRitual
CF 4 Rep. OfNon-Verbal
Cues
CF 5 GreetingRitual
Client Relational Facilitators:Manual Analysis
n=177 transcripts
Comparison Relational Facilitators: Manual Analysis
132
86
41
6474
102121
4027
62
0
20
40
60
80
100
120
140
160
180
200
RapportBuilding
Deference Rep. Of NVCues
Greeting Ritual Closing Ritual
Librarians Clients
n=177 transcripts
3425
0
20
40
60
80
100
120
140
160
180
200
LB 1 Relational Disconnect LB 2 Negative Closure
Librarian Relational Barriers: Manual Analysis
n=177 transcripts
78
33
0
20
40
60
80
100
120
140
160
180
200
CB 1 Closing Problems CB 2 Relational Disconnect
Client Relational Barriers: Manual Analysis
n=177 transcripts
34 3325
78
0
20
40
60
80
100
120
140
160
180
200
Librarians Clients
Relational Disconnect Negative Closure
Comparison of Relational Barriers: Manual Analysis
n=177 transcripts
NVivo Coding
• 100 Transcripts– 79 Usable Transcripts
Librarian Relational Facilitators:NVivo Analysis
n=79 transcripts
3832
63
37
26
0
10
20
30
40
50
60
70
LF1 RapportBuilding
LF2 Deference LF3 NonverbalCues
LF4 GreetingRitual
LF5 ClosingRitual
Relational Facilitators
Nu
mb
er
of
Tra
ns
cri
pts
Client Relational Facilitators:NVivo Analysis
n=79 transcripts
51
11 9
56
25
0
10
20
30
40
50
60
70
CF1 Deference CF2 RapportBuilding
CF3 ClosingRitual
CF5 GreetingRitual
CF4 NonverbalCues
Relational Facilitators
Nu
mb
er o
f T
ran
scri
pts
Comparison Relational Facilitators: NVivo Analysis
n=79 transcripts
63
38 3732
26
5156
9 11
25
0
10
20
30
40
50
60
70
RapportBuilding
Deference Nonverbal Cues Greeting Ritual Closing Ritual
Relational Facilitators
Nu
mb
er o
f T
ran
scri
pts
Librarian Client
Librarian Relational Barriers: NVivo Analysis
n=79 transcripts
2122
0
5
10
15
20
25
30
35
40
45
LB2 Negative Closure LB1 Relational Disconnect
Relational Barriers
Nu
mb
er o
f T
ran
scri
pts
Client Relational Barriers: NVivo Analysis
n=79 transcripts
40
10
0
5
10
15
20
25
30
35
40
45
CB1 Closing Problems CB2 Relational Disconnect
Relational Barriers
Nu
mb
er o
f T
ran
scri
pts
Comparison Relational Barriers: NVivo Analysis
n=79 transcripts
22 21
40
10
0
5
10
15
20
25
30
35
40
45
Negative Closure/Closing Problems Relational DisconnectRelational Barriers
Nu
mb
er o
f T
ran
scri
pts
Librarian Client
NVivo Search TotalsWord Total Percent
sorry 20 25.32
thank you 51 64.56
thanks 34 43.04
thank u 3 3.80
thanx 0 0.00
TY 0 0.00
ty 0 0.00
n=79 transcriptsIncludes words in scripts
NVivo Search TotalsPerson Word Total Percent
Librarian sorry 14 17.72
Client sorry 7 8.86
Librarian thank you 36 45.57
Librarian thanks 18 22.78
Client thank you 28 35.44
Client thanks 24 30.38
Client thank u 3 3.80
n=79 transcriptsIncludes words in scripts
Transcript Reading
• Positive VRS experience– Duration = 1 hour 11 minutes– Academic User– Question – Boston drug company - diabetes– Relational Work– Enthusiastic user– Helpful librarian
• Less than positive VRS experience– Duration = 39 minutes– Middle school or high school student– Question – physics – car acceleration– Poor reference work– Extreme negative closure
End Notes
• This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators.
• Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center.
• Project web site: http://www.oclc.org/research/projects/synchronicity/
Questions
Marie L. Radford, Ph.D.Email: [email protected]/~mradford
Lynn Silipigni Connaway, Ph.D.Email: [email protected]/research/staff/connaway.htm