Nikolaos Tselios* | nitse@ece.upatras.grPanagiota Altanopoulou* | galtanopoulou@gmail.comChristos Katsanos# | ckatsanos@eap.gr

Effectiveness of a framed wiki-based learning activity in the context of HCI education

PCI 2011 | Kastoria, Greece | 30 Sep -2 Oct 2011

*Educational Science and Early Childhood Education, University of Patras# Internal Assessment and Education Unit, Hellenic Open University

Are the wikis suitable to design learning activities?

Web 2 has brought much greater attention to collaboration on the Web, and this has begun to flow into e-learning

Wikis are open-nature and can provide strong opportunities for collaboration, critical thinking and reflection Central tenets of sociocultural views of learning, and in particular of

project based learning (Duffy & Kirkley, 2004)

Wikis are also easy to learn and usable

Can be used both for distance and blended learning

2

However, without planning, design, and effective facilitation, a wiki is no more than an empty webpage (West & West, 2009)

Research Goal & Questions

Investigate if (and to what extent) a properly designed wiki based activity can enhance the learning process

Research Questions Does students’ learning performance improve through a

properly designed wiki-based activity? Are such activities more beneficial to students with lower initial

performance? Does the student’s role in the team affect her/his learning

performance?

3

Using wikis in education( West & West, 2009)

4

• Class encyclopedia• Glossary• Electronic book

Knowledge construction

Critical thought

• Case studies• Structured debate• Annotated bibliography

Contextual

application

• Organize design activities• Story creation• Organize real-world activities

Critical factors in the design of wiki-based learning (West & West, 2009)

1) Establish a purpose for the wiki project

2) Define & classify the learning goals

3) Design a rich context & problem that support the achievement of the purpose & goals

4) Prepare students for work in the new environment,

5) Promote a collaborative process through which active, social learning can take place

5

Study Methodology Participants

36 students Dept. of Education & Early Childhood Education, Upatras Compulsory project, Course: “Introduction to Web Science”

Within-subjects, Pretest-Posttest design

Demographic questionnaire (pre-) 5 questions, demographic data (e.g. age, ICT skills, wiki usage)

Knowledge assessment questionnaire (pre- & post-) 35 multiple-choice questions with 4 answer options Questions on usability evaluation & heuristic evaluation Students were not informed about the assessment 6

Activity design (1/4)

Framework: West & West (2009)

Goal: Study usability evaluation techniques in general and heuristic evaluation in particular

Learning objectives Understand impact and importance of usability Understand usage of wikis (both in general and for education) Study usability evaluation techniques Be able to conduct a heuristic evaluation Be able to form collaboration teams with specific roles assigned Be able to organize and report the results of such a process Be able to use a Wiki

7

Activity design (2/4)

Each group had to create a wiki (WikiSpaces service)

Assignment presented to the students through a wiki initiated by the researchers (http://web-usability.wikispaces.com)

The wiki included 3 parts (Usability Evaluation, Heuristic Evaluation, Actual Heuristic Evaluation of the Dept. Site) Concise instructions Indicative outline Arguments to be developed Supporting material (mainly in the form of hyperlinks)

8

Students were encouraged to use additional material & arguments (discrete evaluation criterion)

Activity design (3/4)

Procedure1. Instruction on the wiki’s basic functionality

2. Presentation of the compulsory project in the form of an already initiated wiki

3. Students worked in freely-formed groups of 4

4. Each team member had a specific role in the group Collector: obtain appropriate material relevant to the subject Coordinator: organize collected material, check its consistency,

coordinate discussion on website heuristic evaluation Editor: compose the outline of the topics according to the

objectives of the scenario Verifier: evaluate content completeness, structure and

compliance with the objectives of the project.

9

Activity design (4/4)

Evaluation criteria (included in the wiki) Coherence Content and text originality Text clarity, argumentation and reasoning Compliance to the provided structure & format guidelines Use of additional material & arguments Material appropriateness and richness Degree of collaboration Appropriate use of references

10

Wiki Screenshots

11

Results (1/3)

12

Does students learning performance improve through a properly designed wiki based activity?

More than half (56%) improved their performance by at least 50%

Highest improvement in score 65.7 (14.3 => 80)

Some (22%) did not improve or even did worse

Results (2/3)

13

Are such activities more beneficial to students with lower initial performance?

Correlationr=-0.586, p<0.001

Results (3/3)

14

Does the student’s role in the team affect her/his learning performance?

ANOVA: F(3,32)=0.732, p=0.541

Students were benefited regardless of the roles they selected themselves

Summary & Questions

Through a properly-designed wiki-based activity (West & West, 2009)

h

Future work: Compare learning gain with a control group (traditional teaching) Design more wiki-based activities in a various educational settings Investigate learners’ behavioral intention to use wiki technology

using technology acceptance models Investigate to what extent students improve in other non-cognitive

aspects (e.g. self-organization, collaboration)

15

Extra Slides

16

What about final scores in the project?

We were interested in investigating the learning gain, not the final outcome Knowledge Assessment Questionnaire Learning gain = Posttest_Score – Pretest_Score

References

Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on collaborative knowledge building with wikis. Computer-Supported Collaborative Learning, 3(2), 105-122.

Duffy, T., & Kirkley, J. (2004). Learner-Centred theory and practice in distance education cases from higher education. Mahwah, N.J.: Lawrence Erlbaum Associates.

Mindel, J., & Verma, S. (2006). Wikis for teaching and leading. Communications of the Association for Information Systems, 18(1), 1-23.

Raman, M., Ryan, T., & Olfman, L. (2005). Designing knowledge management systems for teaching and learning with wild technology. Journal of Information Systems Education, 16(3), 311-320.

West, J.A. & West, M.L. (2009). Using Wikis for Online Collaboration. San Francisco:Jossey-Bass.