Pilot Implementation of Wikis – Lessons Learned on CollaborationVerily Tan verilytanslAllan Chia allanchiabhSteven Wong stevenwonghmYew Haur Lee leeyhSIM University @unisim.edu.sg

Introduction

Collaboration

Two or more students laboring together and sharing the workload equitably as they progress toward intended learning outcomes (Barkley, Cross & Major , 2005)

Collection of individuals who are interdependent, and who share responsibility for outcomes. The team works within time constraints to produce work that involves the application of knowledge, judgment and expertise, often to the solution of a unique problem (Cohen & Bailey, 1997)

Collaboration or Teamwork is a skill highly sought after by employers. It has also been identified as a significant graduate outcome of education. (Riebe, Roepen & Santarelli, 2008)

Characteristics of successful collaboration:1. Equity of effort2. Positive interdependence3. Individual and group

responsibility4. Achievement of working

solutions or learning outcomes

Wikipedia – A Tool to Support Collaboration • Offers collaborative, constructive learning environments

(McMullin, 2005)• “Wiki” = “Quick” (Hawaiian)

• Uses simplified hypertext markup • New users only need to learn a few formatting tags.

• Includes a range of features to facilitate multi-author interactions, e.g. email or RSS notifications of page edits (Judd, Kennedy & Cropper, 2010)

• Data on contributions to every page, when it was made, and who made it, are all captured in the history page of the wiki. • Allows the lecturer to judge both the quantity and quality

of contributions by different group members (Elgort, Smith & Toland, 2008).

Measure of Collaboration in Wikis

Hughes and Narayan (2009) • Questionnaires with questions on

perceived collaboration, the frequency and type of edits in their study.

• Data were analyzed together with student responses to open ended questions

Judd, Kennedy and Cropper (2010)Data on the following was harvested• all versions of students’ wiki pages, • time students spent on edits, • number and types of edits, • number of page edits and • frequency of edits. These records were used to assess students’ level of collaboration and cooperation.

Leung and Chu (2009) Analyzed the following from the history feature • number and dates of

contributions, • version changes, and • discussions

Pilot Implementation of Wiki

Use of Wikis for Group-Based Assignment• Group-Based Assignment (GBA)• Students work on stipulated questions in groups of three• Write-up or essay (Microsoft Word document) submitted through the

university’s learning management system. • Problems

• free riders, and the• lack of real collaboration

• Use of Wiki for GBA?• MKT357: Advertising and Promotion (blended e-learning module)• Each group of students was allocated a wiki, which was password

protected, and not available in the public domain.• Email notifications of edits by members were enabled for each group.

Use of Wikis for Group-Based Assignment• Sections in the wiki page were created for each group

(scaffolds)• working areas where members could input their ideas• consolidation areas where members would collaboratively

write the essay for submission• Students were taught to color code their contributions in the

working area so that the instructor would have a quick overview of the group progress, and individual student contribution.

• Before the deadline, students are required to convert their group wiki page into a PDF document, for submission via Turnitin for plagiarism checks.

Use of Wikis for Group-Based Assignment• GBA:

• Q1: Critique of three advertisements• Q2: Collaboratively write a creative brief for a client (to

emulate the actual task advertisers perform in real life)• 90% of the marks – content; 10% - participation • The course was facilitated by four instructors. They had been

trained in a dedicated session• Assignment took place over a period of 4 weeks

Use of Wikis for Group-Based Assignment• Students were briefed on the use of the wiki during the first

face-to-face session of the course. (1.5 hours)• Big cohort of 192 students

• Students were briefed in normal classrooms in their respective tutorial groups (each group shared one laptop)

• Limited hands-on practice• Provided with Wiki manual, online demonstrations

• Sample wiki was provided, which included a project plan, simulated input by different authors in different colors, and the final essay consolidation

Purpose of Study1. Would the use of the wiki alleviate the problems of free

riders, and lack of collaboration?2. What is students’ perception of wikis?

Data CollectionQuestionnaire (15 questions)A. Students’ Perception of the Wiki

• 6 questions based on 5-point Likert scale; 1 question on the mode of communication

B. Scoring of Group Collaboration• 4 questions utilizing a scale of 1-10

Students were asked to score their group on:(1) Organization(2) Positive Interdependence(3) Equity of Effort(4) Final Goal/Outcome

Mean of the total score (four items) for each member of the group was taken as the group’s perceptual measure of collaboration. This is represented by Mean(B_Total).

C. Demographic Data• Number of edits per student per week was collected based on the wiki

page history.

Data Collection• Survey administered through the student portal, with a response rate of

98.4%. • The survey respondents (n=189; No of groups=64) :

• The majority of students (82.3%) were between 20-30 years old. • 33.3% rated themselves fairly skilled in computer skills, 49.2% rated

themselves intermediate, and 15.3%, Power User. • Group size:

• Majority of the groups consisted of three members (89.1%)• 3 groups had four members. • 1 group had one member (due to last minute student withdrawal from

the course)

Sample of Student Work

Team Wiki(http://ununhexium.unisim.edu.sg/wikis/mkt357/t06_group01/ )

Results & Discussion

Free Riders• Wiki implementation took place smoothly. The GBA

submissions and instructor grading were uneventful.• “Free riders” problem appears to be resolved• Participation score (10%) awarded to the students to each

member of the group was the same for 96.9% of the groups.

Student Usage of Wikis

Fig. 2: Variability Chart for Number of Edits vs. WeekFig. 2: Variability Chart for Number of Edits vs. Week

• Mean number of edits per student for. Week 1 – 4.63 (excluding three outliers)Week 2 - 3.62Week 3 - 5.94Week 4, the mean number of edits shot up to 21.29

• Late minute edits were not unexpected. (Judd, Kennedy & Cropper, 2010). This trend raises the doubt on the possibility of extensive collaboration between group members.

Each point represents the number of edits per student – 189 data points per weekEach point is color-coded according to the group the student belongs to – 64 colors per week

• If we define interaction as more than one member editing the wiki• 10.9% of groups had no interaction for three weeks• 26.6% for two weeks• 37.5% for one week

• Only 15 groups (23.4%) had more than one member editing the wiki every week. By segregating these groups apart from the rest, we obtain Fig. 3.

Interaction Vs. Students’ Perceived Collaboration Score

Interaction Vs. Students’ Perceived Collaboration Score

Fig. 3: Variability Chart for Number of Edits vs. Week (Comparison)Comparison of Groups with (more than one member editing the wiki every week = No) and Groups with (more than one member editing the wiki every week = Yes)

Fig. 3: Variability Chart for Number of Edits vs. Week (Comparison)Comparison of Groups with (more than one member editing the wiki every week = No) and Groups with (more than one member editing the wiki every week = Yes)

• Mean number of edits for the groups with (more than one member editing the wiki every week = Yes) for Weeks 1 to 3 are about double compared to groups with (more than one member editing the wiki every week = No).

• Week 4 - mean numbers of edits are similar between these two segregations of groups.

15 groups (with more interactions)

• If interactions can be used as a measure of collaboration, groups with (more than one member editing the wiki every week = Yes) should have a higher amount of collaboration compared with groups with (more than one member editing the wiki every week = No)

• Expect a higher Mean(B_Total) for the former compared to the latter

• Two sample t-test was carried out using an alpha value of 0.05. The p-value is 0.433, indicating no significant difference in the groups’ perceptual measure of collaboration, between groups with (more than one member editing the wiki every week = Yes) and groups with (more than one member editing the wiki every week = No)

• Interactions – not an accurate indication of collaboration?

Interaction Vs. Students’ Perceived Collaboration Score

Collaboration – Outside of the WikiCommunication Mode indicated as Most Important

% of students

Email 15.9%*SMS 14.8%Face-to-face meetings 13.2%Wiki 10.1%

• Such collaboration may explain why interactions on the wiki alone may not be an accurate measure of collaboration.

• *Simultaneous edits - A page locking mechanism prevents overriding of content by users who save their edits concurrently. Editing the group wiki near the deadline may have resulted in students co-coordinating the timing of their edits using SMS.

Students’ Perception of Wikis• 47.0% of students disagreed that the wiki was a useful

platform for collecting ideas, opinions and organizing information as a group.

• 48.7% of students disagreed that the wiki was a convenient platform for the Group-Based Assignment.• Some students found no intrinsic value in the use of wikis

at all, alluding to it causing more time and effort spent on the assignment.

• Some felt the use of wikis resulted in a misplaced focus on technical aspects, rather than the course content.

Student Perceptions of Wikis• Technical Difficulties

• 22.7% of students encountered technical difficulties using the wiki platform

• Formatting and color coding• Simultaneous edits

• Some students who encountered this error did not perform the necessary steps to save their work.

• Some students were not confident about how to deal with the error, choosing to “take turns” in editing.

Appreciation of the Potential of Wikis• “It's good as we do not have to wait for our group mates to

submit their contribution (in Word document) to one another before we can comment and edit. So time wise, it's quite good.”

• “I think I have gained a new skill and knowledge on this work sharing program. This is exceptionally useful for projects in modules like Marketing. I like this platform of sharing workload and discussion. I hope that in future, the University can look into introducing wikis to other modules.”

Conclusion & Recommendations

Conclusion• Evidence of the wiki alleviating the problem of free riders• No conclusive data on student collaboration in the wiki. • Last minute edits - possibility of extensive collaboration?• Designing learning activities that are collaborative in nature,

or simply using “collaborative” technologies like wikis, does not ensure or encourage student collaboration. (Judd, Kennedy & Cropper, 2010)

• The following needs to be addressed: • students’ negative perception• students’ tendency towards last minute interaction and

edits in the wikis

Recommendation (1)

Provide Hands-on Training and Orientation Activity (Build Trust and Belief)• 54.0% of students disagreed that the face-to-face training session

prepared them well on how to use the wiki as a collaborative platform for the GBA.

• Transition to the use of wiki was not smooth or students’ inherent resistance was not overcome by the perceived merits of the wiki

• Technical concerns or a lack of trust in technology in general, can have a negative influence on the perceived effectiveness of wiki-based collaboration.

• Not achieving an appropriate level of conceptual understanding may lead to a lack of trust in the wiki collaboration process, preventing users from embracing it fully and using it to best effect (Judd, Kennedy & Cropper, 2010; Davies, 2004).

http://www.flickr.com/photos/rhruzek/4283476436/

Recommendation (1)• Learning by participation and experience is important.

• Hands-on training for all, i.e. each student should have access to a laptop or computer

• Students need more time to adjust to the formatting features in the wiki, to learn the correct steps to take when they encounter the error page during concurrent saving of edits.

• Orientation activities • Self-introductions, or “ice-breaker activities” (Judd, Kennedy &

Cropper, 2010; Minocha et. al, 2007). • Plan should also focus on how to use the wiki in a productive manner

(Robertson, 2008). • Case studies of successful utilization of the wiki in corporations should

be cited to help users overcome initial apprehension and boost perceived effectiveness and belief in the concept (Davies, 2004).

• Trust and belief in the technology and concept of wiki will motivate students towards more active usage.

Recommendation (2)Enhance the Wiki as a Collaboration Tool• Collaboration involves communication. A good communication tool

should: (Lomas, Burke & Page, 2008)• allow natural interactions• have an easy-to-understand, intuitive interface• a new tool should have better capabilities than its predecessor. The

tool should also have a seamless integration with main applications• To engineer a less “painful” transition to the use of the wiki for

collaboration, we are exploring the following options. 1. Make use of the wiki functionality available in Blackboard – advantage of

seamless integration with the University learning management system.2. Google documents - although page history is not available, benefit of near

zero learning curve for students3. Enhance MediaWiki to incorporate a WYSIWYG editor, e.g. CKEditor

(http://ckeditor.com/)

Edit View in MediaWiki

Edit View in MediaWiki with CKEditor

Improved editing and formatting featuresSimilar to Microsoft Word

Recommendation (3)Use Wikis in Combination with other Communication Tools• Collaboration is a science in itself

(Tuckman, 1965; Pineda & Lerner, 2006).

• This complex process is not easily measured, and is unlikely to be fully captured on the wiki.

• Instructors may find the analysis of the history page of group wikis too daunting in assessing for collaboration.

http://www.flickr.com/photos/38795936@N00/308014588/

Recommendation (3)• Increase the synergy between the wiki and other communication modes

to make wiki assignments more flexible. (Davis, 2004) • Make collaboration methods like wiki, email and even face-to-face

meetings work together in the most simple and intuitive way, with the ultimate outcome of achieving collaboration.

• Wiki input can include: (Working area)• Notes of meeting• Word document attachments• Email correspondence

http://www.flickr.com/photos/maximalideal/4735417081/

Recommendation (4)Redesign the Assessment or Activity (Include Elements to Motivate Students)• Collaboration on a wiki requires time, effort and most importantly,

motivation.• Relative short time frames, modest assessment weighting may not have

motivated students sufficiently to effectively engage in the task. (Judd, Kennedy & Cropper, 2010).

• Online collaborative learning on a wiki can be excellent for (Sanger, 2010) 1. exchanging written ideas, especially when those involved are

interested and motivated2. obtaining free public reviews of students’ work.

Recommendation (4)• In redesigning our GBA, we could:

• Encourage the exchange of ideas by pairing groups together for peer reviews. One group could act as the “real life client” for the other. This could serve as a motivation for both groups.

• Obtain instructor feedback at important milestones of the wiki assignment. Instructor involvement can also pace the development of the wiki, as students become cognizant of the monitoring of the wiki –not just for submission, but during its development.

• A new assessment design would entail more time, for both student and instructor. Adjustments to the timeline and assessment weightage are recommended.

Limitation• Data in this study was based on edits and interaction in the

wiki. • More data on collaboration can be obtained from the page

history of the wiki, e.g. types of edits, time spent on edits and frequency of edits.

• The participation marks awarded by instructors was based primarily on the color coding by authors in the working areas of their group wikis. • The accuracy of this measure can be validated with the

wiki page history.

AcknowledgementsSpecial thanks to the following people from SIM University:• Technical Training Team, Educational Technology &

Production, for their unwavering support and professionalism

• A/P Sharen Liu, Director, Educational Technology & Production

• Prof Koh Hian Chye, Dean, School of Business• Ms Peggy Su, Ms Regina Yeo, Mr Bernard Law, Mr Ray

Monteverde

Power Point Slides• Slideshare

• verilytan