Assessing and promoting computer-supported collaborative learning
Anne Meier University of Freiburg, Institute of
Psychology
• Introduction to CSCL (computer-supported collaborative learning)
• Assessing CSCL learning processes
• Supporting CSCL learning processes
• Example study: adaptive support for knowledge co-construction
Overview
The CSCL community
• a short history of CSCL…• “seeds” in the 1980s, e.g. 1989 NATO-sponsored workshop
“computer-supported collaborative learning” (Maratea, Italy)• since 1995: bi-annual CSCL conferences• since 2003: CSCL community part of International Society
of the Learning Sciences (ISLS)
• own journal: International Journal of CSCL (ijCSCL) published by Springer since 2006
• highly interdisciplinary community
Introduction to CSCL
• CSCL researchers study:• How people can learn together with the help of computers (Stahl,
Koschmann, & Suthers, 2007)• How technology can facilitate the sharing and creation of knowledge
and expertise through peer interaction and group learning processes (Restra & Laferrière, 2007)
advantages/strengths
challenges/pitfalls
What is your experience with (computer-supported) collaborative learning?
• Neo-Piagetian perspective• learning = cognitive restructuring• resolving socio-cognitive conflict arising from peer collaboration
• Cognitive elaboration perspective• learning = elaboration and integration of knowledge• very important: constructing explanations
• Neo-Vygotskian perspective• learning = appropriation, internalization• knowledge co-construction; scaffolding and fading
• Situated learning perspective• learning = increasingly “central” participation in a community of
practice• distributed cognition: persons, tools, symbols, artefacts,…
What makes collaborative learning effective?
See for example: Cohen, 1994; Dillenbourg et al., 1995; Fischer, 2002; Webb & Palincsar, 1996)
• Motivational process loss (e.g. Salomon & Globerson, 1989)
• Free-rider effect (“social loafing”)• Sucker effect
• Production blocking• having to wait for others to finish their turn• e.g. in brainstorming (Diehl & Stroebe, 1987)
• Biased information sampling (e.g. Brodbeck et al., 2007; Stasser & Titus, 1985))
• neglecting individuals’ unique knowledge• striving for consensus rather than understanding
Putting people in a (computer-supported) group does not mean that they will collaborate well!
Pitfalls of collaborative learning
• Introduction to CSCL (computer-supported collaborative learning)
• Assessing CSCL learning processes What characterizes “good” computer-supported collaborative learning?
• Supporting CSCL learning processes
• Example study: adaptive support for knowledge co-construction
Overview
Cognitive, social, and affective aspects of collaboration quality in CSCL
Communication (Clark & Brennan, 1991)
• Grounding• adapting utterances to the amount of shared knowledge/ perspective/
experience• establishing referential identity (e.g. of objects in a shared whiteboard,
of previous messages/ contributions)• establishing a shared terminology
• Conversation management • initiating conversations• managing turn-taking• ensuring that contributions are taken up
For additional literature/ references, please see Meier, Spada, & Rummel, 2007
Cognitive, social, and affective aspects of collaboration quality in CSCL
Joint information-processing
• Elaborative information-processing• eliciting and providing elaborated explanations• using the partner as a resource• elaborating on partners’ contributions
• Argumentative information-processing• constructing justified arguments and counterarguments• engaging in a critical discussion: avoiding an illusion of consensus
Cognitive, social, and affective aspects of collaboration quality in CSCL
Coordination (explicit or tacit) (e.g. Malone & Crowstone, 1994)
• Task division• identifying interdependent subtasks• blending individual and collaborative work
• Time management• agreeing on a realistic time schedule• monitoring the remaining time during the work process
• Resource management• handling the available tools efficiently• agreeing on who may use a technical feature at what time
Cognitive, social, and affective aspects of collaboration quality in CSCL
Relationship management
• maintaining equal participation• symmetric or complementary, depending on role structure
• solving conflicts constructively• epistemic vs. social conflicts
Cognitive, social, and affective aspects of collaboration quality in CSCL
Motivation
• individual task orientation• keeping up a high level of expended effort• volitional processes: focusing attention, exerting motivation control
• mutual self-regulation• mutual encouragement• monitoring performance and giving feedback
Example: Collaboration quality rating-scheme
• Development• sample from study on interdisciplinary collaboration: students of
psychology and medicine solving complex patient cases (Rummel & Spada, 2005)
Meier, A., Spada, H. & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2 , 63-86.
Example: Collaboration quality rating-scheme
Control Room
Experimental Room I Experimental Room II
Example: Collaboration quality rating-scheme
• Development• sample from study on interdisciplinary collaboration: students of
psychology and medicine solving complex patient case (Rummel & Spada, 2005)
• data- and theory-driven analyses 5 aspects/ 9 dimensions• for each dimension:
• collaboration “standard” defined and illustrated in rating handbook• collaboration quality rated on 5-point scales
Example: Collaboration quality rating-scheme
Means: Pattern 1 (task division)
00,5
11,5
22,5
33,5
Control Script Scriptplus
Model Modelplus
Means: Pattern 2 (individual task orientation, medical student)
00,5
11,5
22,5
33,5
Control Script Scriptplus
Model Modelplus
model / script > controlmodel-plus > modelInformation pooling
Task divisionTime managementTechnical coordination
model > control > scriptmodel-plus > modelIndividual task orientation
Quality of joint solution(Rummel, Spada, & Hauser, 2009)
Rummel
rummele
Example: Collaboration quality rating-scheme
• adaptation to new CSCL setting (Synergo) (Voyiatzaki et al., 2008)• descriptive framework valid in this setting as well• But: changed operationalization of dimensions and re-anchoring of scales
necessary
….. work in progress:providing adaptive feedback to students based on ratings of their collaboration quality(Meier, Voyiatzaki, Kahrimanis, Rummel, Spada, Avouris, 2008)
• Introduction to CSCL (computer-supported collaborative learning)
• Assessing CSCL learning processes
• Supporting CSCL learning processes How can beneficial collaboration processes be facilitated?
• Example study: adaptive support for knowledge co-construction
Overview
Supporting CSCL learning processes
• Earlier approaches: support “around” collaboration
• Collaboration scripts: support during collaboration
• Adaptivity: from fixed to flexible support
Supporting CSCL learning processes
• Earlier approaches: support “around” collaboration• Support prior to collaboration, e.g. training for strategic questioning
(King, 1991)• Support after collaboration, e.g. group processing approaches (Yager,
Johnson, Johnson, & Snider, 1986)
• Collaboration scripts: support during collaboration
• Adaptivity: from fixed to flexible support
after: Diziol & Rummel, accepted
Supporting CSCL learning processes
• Earlier approaches: support “around” collaboration
• Collaboration scripts: support during collaboration• provide specific instructions about task-related interaction (Kollar et al.,
2006)• Sequencing work phases• Distributing roles• Specifying activities
goal: enhance cognitive, meta-cognitive and social learning processes
• Adaptivity: from fixed to flexible support
after: Diziol & Rummel, accepted
Collaboration Scripts
Jigsawdistribution of knowledge
(e.g. expert groups & teams)exchange of information
Conflictconflicting opinions
(e.g. pro & contra-roles)argumentation
Reciprocalcognitive & metacognitive tasks
(e.g. recaller & detector)mutual regulation
Split Where Interaction Should Happen (SWISH) (Dillenbourg & Jermann, 2007)
Schema Split Compensation
Supporting CSCL learning processes
• Earlier approaches: support “around” collaboration
• Collaboration scripts: support during collaboration
• Adaptivity: from fixed scripts to flexible support• Danger of “overscripting” collaboration (Dillenbourg, 2002); instead:
taking into account students’ prior knowledge and “internal collaboration scripts”
• realizing flexible, adaptive support:• “Wizard of Oz” studies• adaptive feedback based on automated analyses of interaction (e.g.
Dönmez et al, 2005)
after: Diziol & Rummel, accepted
• Introduction to CSCL (computer-supported collaborative learning)
• Assessing CSCL learning processes
• Supporting CSCL learning processes
• Example study: adaptive support for knowledge co-construction
Overview
Example: Supporting Collaborative Inferences
F - I - R - E !
Figure from: Bauer, K., & Hesse, F. (2006). Von Kopf zu Kopf. [From head to head]. Gerhirn und Geist [Brain & Mind], 5/2006, 34-39.
Example: Supporting Collaborative Inferences
Wolfgang‘s fingerprints are on
the gun.
Wolfgang showed the guns to his guests in the
afternoon.
A B
Example: Supporting Collaborative Inferences
Wolfgang‘s fingerprints are on
the gun.
Wolfgang showed the guns to his guests in the
afternoon.
A B
Example: Supporting Collaborative Inferences
Wolfgang left his fingerprints on the weapon when he showed it to his guests.
A B
Information distribution Inference type
Person A Person B
collaborative
individual
shared
Example: Supporting Collaborative Inferences
Inference drawing frequency
0.49
0.65
0.79
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
collaborative individual shared
***
Meier & Spada, 2007
Example: Supporting Collaborative Inferences
Why is it so difficult to draw collaborative inferences?
1. individual group member holds “unconnected” information• seen as less relevant and therefore less likely to be brought up during
discussion (Fraidin, 2004)
2. inference must be drawn on the basis of newly learned information and recalled information• people tend to focus on old rather than new information (Wittenbaum, Hubbel &
Zuckermann, 1999)• recall vulnerable to disruptions in group discussion (Finlay, Hitch & Meudell,
2000)
Training Experiment: train collaboration strategies for• drawing inferences• pooling “unconnected” information• taking up new information
Example: Supporting Collaborative Inferences
No Training(n=9 )
Training Task(n=9)
Training Task+ Text (n=9)
Training Task + Text + Tutoring (n=9)
Training phase
read text on collaboration strategies
Training task (medical diagnosis):individual reading phase
discussion & solution ...with inference tutoring tool
collaborative reflection
Testing phase
Test task (murder mystery)
Meier & Spada, 2008
disease information
patient information
inference
1
2
3
4
5
6
7
8
...
New Information!
ANJA has matching information.
Example: Supporting Collaborative Inferences
disease information
patient information
inference
1
2
3
4
5
6
7
8
...
Well done!
You have drawn an important inference!
Example: Supporting Collaborative Inferences
Example: Supporting Collaborative Inferences
No Training(n=9 )
Training Task(n=9)
Training Task+ Text (n=9)
Training Task + Text + Tutoring (n=9)
Training phase
read text on collaboration strategies
Training task (medical diagnosis):individual reading phase
discussion & solution ...with inference tutoring tool
collaborative reflection
Testing phase
Test task (murder mystery)
Meier & Spada, 2008
Example: Supporting Collaborative Inferences
Inference drawing frequency
0
0.2
0.4
0.6
0.8
1
No_Training TrainingTask
TrainingTask + Text
TrainingTask + Text
+ Tutor
collaborative individual shared
**
**
performance during testing (without tutoring tool)
Meier & Spada, 2008
In a nutshell…
• Introduction to CSCL (computer-supported collaborative learning)• diverse perspectives on collaborative learning within field of CSCL• successful collaboration does not occur spontaneously!
• Assessing CSCL learning processes• focus here was on processes, rather than outcomes or preconditions• many relevant aspects: communication, information-processing,
coordination, relationship management, motivation
• Supporting CSCL learning processes• collaboration scripts: (computer-)support during collaboration• moving towards more flexible, more adaptive support
• Example study: adaptive support for knowledge co-construction• collaborative inferences are important but difficult• adaptive support yields best training results
Many thanks to the CoEmCo-Team:Hans Spada, Nikol Rummel
Dejana Diziol, Sabine HauserEva Zerpies, Malte Jansen
This work was funded byThis work was funded by
Thank you for your attention!
Brodbeck, F., Kerschreiter, R., Mojzisch, A., & Schulz-Hardt, S. (2007). Improving group decision making under conditions of distributed knowledge: The information asymmetries model. Academy of Management Review, 32, 459-479.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127–148). Washington, DC: American Psychological Association.
Cohen,E.G.:Restructuring the classroom: Conditions for productive small groups. Review of Educational Research,64,1-35.1994Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53, 497-509Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL? (pp. 61-91). Heerlen: Open
Universiteit Nederland.Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in humans and machines: Towards an interdisciplinary
learning science (pp. 189–211). Oxford: Pergamon.Dillenbourg, P. & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl &, J. Haake, Scripting computer-supported communication of knowledge. Cognitive, computational, and educational perspectives (pp. 259-288). New York: Springer.
Diziol, D., & Rummel, N. (to appear). How to design support for collaborative e-learning: A framework of relevant dimensions. In: Ertl, B. (Ed.): E-Collaborative knowledge construction: learning from computer-supported and virtual environments.
Dönmez, P., Rose, C., Stegmann, K., Weinberger, A., & Fischer, F. (2005). Supporting CSCL with automated corpus analysis technology. In T. Koschmann, D. Suthers, & T. W. Chan (Eds.), Proceedings of the CSCL 2005 (pp. 125–134). Mahwah, NJ: Lawrence Erlbaum Associates.Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts - a conceptual analysis. Educational Psychology Review. Springer.
Fischer, F. (2002). Gemeinsame Wissenskonstruktion - theoretische und methodologische Aspekte. Psychologische Rundschau, 53 (3), 119-134.Fraidin, S. N. (2004). When is one head better than two? Interdependent information in group decision making. Organizational Behavior and Human Decision Processes, 93, 102-113.Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts - a conceptual analysis. Educational Psychology Review, 18, 159-185.Koschmann, T. (Ed.). (1996). CSCL: Theory and practice of an emerging paradigm. Mahwah, NJ: Lawrence Erlbaum Associates.Koschmann, T., Hall, R., & Miyake, N. (Eds.). (2002). CSCL 2: Carrying forward the conversation. Mahwah, NJ: Lawrence Erlbaum Associates.Kraut, R. (2003). Applying social psychological theory to the problems of group work. In: J. M. Carroll (ed.), HCI models, theories, and frameworks. Towards a multidisciplinary science (pp. 325-356). Amsterdam:
Morgan Kaufmann.Malone, T. W., & Crowston, K. (1994). The interdisciplinary study of coordination. ACM Computing Meier, A., & Spada, H. (2007). Information pooling and processing in group problem solving: analysis and promotion of collaborative inferences from distributed information. In D. S. McNamara & J. G. Trafton (Eds.),
Proceedings of the 29th Annual Cognitive Science Society (pp. 473-479). Austin, TX: Cognitive Science Society.Meier, A., & Spada, H. (2008). Promoting the drawing of inferences in collaboration: insights from two experimental studies. Proceedings of the International Conference of the Learning Sciences, Utrecht, The
Netherlands, 2008.Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2, 63-86.Meier, A., Voyatzaki, E., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008). Teaching students how to improve their collaboration: Assessing collaboration quality and providing adaptive feedback in a CSCL
setting. Symposium New Challenges in CSCL: Towards adaptive script support (N. Rummel & A. Weinberger), ICLS 2008, Utrecht, The Netherlands.Resta, P. & Laferrière, T. (2007). Technology in support of collaborative learning. Educational Psychology Review, 19, 65-83.Rummel, N., Hauser, S., & Spada, H. (2007). How does net-based interdisciplinary collaboration change with growing domain expertise? In C. A. Chinn, G. Erkens & S. Puntambekar (Eds.), Mice, minds and sociecty.
Proceedings of the Computer Supported Collaborative Learning (CSCL) Conference 2007, Vol 8, II (pp. 611-620). International Society of the Learning Sciences, Inc. ISSN 1819-0146Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem-solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201-241.Rummel, N., Spada, H., & Hauser, S. (2009). Learning to collaborate while being scripted or by observing a model. International Journal of Computer-Supported Collaborative Learning, 4(1), 69-92.Salomon, G./Globerson, T.: When teams do not function the way they ought to. Internationnal Journal of Educational Research,11, 623-637. 1989.Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. Journal of the Learning Sciences, 3(3), 265-283.Stahl, G., Koschmann, T. & Suthers, D. (2007). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409-426). Cambridge,
UK: Cambridge University Press.Stasser, G. & Titus, W. (1985). Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of Personality and Social Psychology, 48, 1467-1478.Voyiatzaki, E., Meier, A., Kahrimanis, G., Rummel, N., Spada, H., & Avouris, N. (2008). Rating the quality of collaboration during networked problem solving activities, Proceedings of the 6th International Conference on
Networked Learning, pp. 409-416, Halkidiki, May 2008.Webb, N.M., & Palincsar, A.S. (1996). Group processes in the classroom. In D. Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 841-873). New York: MacmillanWittenbaum, G., Hubbell, A, & Zuckermann, C. (1999). Mutual enhancement: Toward an understanding of the collective preference for shared information. Journal of Personality and Social Psychology, 77 (5), 967-978.
References / Readings
Example: Collaboration quality rating-scheme
Meier, A., Spada, H. & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2 , 63-86.
CommunicationSustaining mutual understanding
Dialog management
Joint information processingInformation pooling
Reaching consensus
Coordination
Task division
Time management
Technical coordination
Relationship management Reciprocal interaction
Motivation Individual task orientation
Outcomes of collaborative learning
When are groups better than individuals? insights from social psychology (Kraut, 2003)
• Aggregation: combining the unique resources of individual group members• Making use of members’ complementary knowledge, perspectives, skills etc.• e.g. a cross-functional marketing team making strategic decisions based on
members’ complementary expertise
• Synergy: going beyond the resources contributed by group members• building on each others’ contributions, creating innovative ideas & solutions• e.g. a product-design team developing a new product
“assembly bonus” however: groups tend to neglect members’ unique knowledge and focus
instead on shared knowledge (Stasser & Titus, 1985)
Measuring the success of computer-supported collaborative learning
• Individual learning• types of knowledge and skills
• conceptual vs. procedural• skills: domain, collaboration, self-regulation, computer-literacy
• level of evaluation: subjective evaluation - retention – transfer
• Group-level learning• transactive memory, shared mental models facilitates future collaboration in the same group
• Interpersonal and motivational outcomes• trust, liking, friendships,…• self- and group-efficacy• interest motivation for future collaboration
Measuring the success of computer-supported collaborative learning
• What characterizes “success” in the TEL-environment you study? How do you assess it?
Assessing collaboration quality
cognitive, social, and affective processes
process gain:
assembly bonus, synergy effects
process loss:
motivation and coordination problems
group size & composition
task type
technical & informational
resources
goal structure
Group learning, e.g. transactive memory, shared mental models
Individual learning- domain knowledge and problem-solving skills- collaboration skills- technical skills
Interpersonal & motivational outcomes, e.g. trust, liking, self-efficacy, group-efficacy
institutional context
Input