Towards adaptation in e-learning 2.0

This article was downloaded by: [Moskow State Univ Bibliote]On: 20 November 2013, At: 02:59Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

New Review of Hypermedia andMultimediaPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tham20

Towards adaptation in e-learning 2.0Alexandra I. Cristea a & Fawaz Ghali aa Department of Computer Science , University of Warwick ,Gibbet Hill Rd, Coventry, CV4 7AL, Warwickshire, UKPublished online: 26 Sep 2011.

To cite this article: Alexandra I. Cristea & Fawaz Ghali (2011) Towards adaptation in e-learning 2.0,New Review of Hypermedia and Multimedia, 17:2, 199-238

To link to this article: http://dx.doi.org/10.1080/13614568.2010.541289

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/tham20

http://dx.doi.org/10.1080/13614568.2010.541289

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Towards adaptation ine-learning 2.0

ALEXANDRA I. CRISTEA* and FAWAZ GHALIDepartment of Computer Science, University of Warwick, Gibbet Hill Rd, Coventry, CV4

7AL, Warwickshire, UK

(Received 18 January 2010; final version received 15 November 2010)

This paper presents several essential steps from an overall study on shaping new ways oflearning and teaching, by using the synergetic merger of three different fields: Web 2.0,e-learning and adaptation (in particular, personalisation to the learner). These novelteaching and learning ways*the latter focus of this paper*are reflected in and finallyadding to various versions of the My Online Teacher 2.0 adaptive system. In particular,this paper focuses on a study of how to more effectively use and combine therecommendation of peers and content adaptation to enhance the learning outcome ine-learning systems based on Web 2.0. In order to better isolate and examine the effects ofpeer recommendation and adaptive content presentation, we designed experimentsinspecting collaboration between individuals based on recommendation of peers whohave greater knowledge, and compare this to adaptive content recommendation, as wellas to ‘‘simple’’ learning in a system with a minimum of Web 2.0 support. Overall, theresults of adding peer recommendation and adaptive content presentation wereencouraging, and are further discussed in detail in this paper.

Keywords: Adaptation; E-learning; Adaptive (educational) hypermedia; Web 2.0;E-learning 2.0

1. Introduction

The harnessing of collective intelligence has been a focus and branch ofartificial intelligence over the last 30 years, but was defined much earlier, in1955, by de Chardin (1955), who predicted the ‘‘noosphere’’, as the sphere ofcollective human thought, as a natural follower of the geosphere andbiospere. In fact, the idea of the ‘‘wisdom of the crowds’’ was alreadypromoted by the ancient Greek philosopher Aristotle (350 BC) in the fourthcentury BC in his work ‘‘Politics’’.

Recently, the term has been used also in connection with current webdevelopments, where communities of users represent more powerful entitiesthan disparate individuals. The current notion of collective intelligence is alsosomewhat related to the concept of the ‘‘World Brain’’, as introduced by Wells(1938) in his science fiction book. Lately, a fairly widespread literature from

*Corresponding author. Email: [email protected]

New Review of Hypermedia and Multimedia,Vol. 17, No. 2, August 2011, 199�238

New Review of Hypermedia and Multimedia

ISSN 1361-4568 print/ISSN 1740-7842 online # 2011 Taylor & Francis

http://www.tandfonline.com

DOI: 10.1080/13614568.2010.541289

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

http://www.tandfonline.com

computing a network science on collective intelligence has appeared (e.g. awork on programming collective intelligence, Segaran 2007).

Especially in the domain of e-learning, such a notion is very important, asit lifts the burden on teachers and educators to some extent, as students canlearn also from their social peers, and not only by soliciting the few availablespecialists. E-learning, or electronic learning, in general refers to any form oflearning with the help of electronic media. Here, we use the term, somewhatsimplifying, in reference to its sub-set of electronic web-based learning, thuslearning using the web as a medium. The trend of combining e-learning withWeb 2.0 has created a new direction, that of e-learning 2.0 (Karrer 2006). Oneissue with e-learning 2.0 is, however, that, whilst it channels the ‘‘wisdom ofthe crowds’’, it does not adapt to the personal needs and characteristics of alearner.

Adaptive hypermedia (Brusilovsky 1996), and especially its application inthe educational field, adaptive educational hypermedia, is a field that has beendeveloping for about 20 years, centred around personalisation of content to alearner in a Web environment. Thus, it provides an e-learning paradigm whichcaters for the dynamic, changing needs, preferences, cognitive styles, knowl-edge, etc. of an individual learner. It thus addresses a lack in Web 2.0-basedsystems. However, unlike the latter, it does not provide for user interaction,and learners, although online, learn in isolation from their peers.

Responding to these issues, in previous research we have built a firstversion of a system, My Online Teacher 2.0 (MOT 2.0), which hassuccessfully combined Web 2.0 features (such as tags, rating system, feedback,etc.) in order to support both learners (Ghali and Cristea 2009a) inpersonalised systems, as well as authors (Ghali and Cristea 2009b).

In this paper, we are aiming to extend this system, in order to establish thebest balance between Web 2.0 features (which have already proven satisfac-tory for learning, in previous research; Ghali and Cristea 2009a), personalisa-tion and adaptive peer recommendation, respectively. Therefore, we focus onthe following research questions:

. Is an e-learning 2.0 environment enhanced by the addition of content-based adaptation? (i.e. is there a positive learning outcome in an environ-ment based on collective intelligence enhanced with personalisation ofcontent?)

. Is an e-learning 2.0 environment enhanced by the addition of peerrecommendation? (i.e. is there a positive learning outcome in an environ-ment based on collective intelligence enhanced with recommended peers tocollaborate or learn from?)

The questions above can be answered both via experimental analysis andobjective measurements (such as knowledge tests). Please note that here wedon’t go into the discussion of the issues surrounding the degree ofobjectiveness of tests. Whilst they are a disputed measure, as they don’tnecessarily summarise all types of intelligence, they are also the de factostandard recognised measure for all forms of education.

200 A. I. Cristea and F. Ghali

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

. Moreover, for all the above questions, additionally, we want to find outwhat the perceived learning effect and usability of the paradigm is.

The latter needs established via subjective feedback, such as in the form of aquestionnaire. For the purpose of this analysis, a new version of MOT 2.0 wascreated, significantly extending a previous version that had already beenevaluated.

The remainder of this paper is organised as follows. Related research isconsidered in Section 2. Section 3 introduces a generic scenario, whichencompasses the overall scale of functionality desired in the targeted adaptivee-learning collaborative framework (Social Layers of Adaptation and theirOperators, SLAOS; Cristea et al. 2010) and system (MOT 2.0) from thelearner’s perspective. This scenario also is then transformed in a set ofoverarching, generic requirements. Next, in Section 4, the theoretical frame-work, SLAOS, is sketched. Following this, in Section 5, two implementationcycles of the MOT 2.0 system are presented. The first cycle is summarisedtogether with the evaluation results, which serves as an introduction andmotivation to the second cycle, first time introduced in this paper. Next, inSection 6, the hypotheses examined within this paper are presented, followedby the case study designed for their examination (Section 7). The results ofthe evaluation are then presented in Section 8, and the implications arediscussed in Section 9. Finally, conclusions are drawn in Section 10.

2. Related research

The SLAOS framework and the MOT 2.0 system bridge the gap betweenthree different research areas: adaptive hypermedia, e-learning and Web 2.0(see Figure 1). As there is little research that encompasses all, we look at somerelevant researches in the overlapping areas, as follows:

Figure 1. MOT 2.0: adaptive e-learning 2.0.

Adaptation in e-learning 2.0 201

Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

(1) Web 2.0 in e-learning;(2) adaptive e-learning;(3) peer recommendations in e-learning; and(4) adaptive collaborative learning.

Therefore, here we will analyse related research within these four areas.By looking at the research on Web 2.0 in e-learning, we can mention the work

of Isaıas et al. (2009), which mainly focuses on combining learning manage-ment systems (LMSs) and social networks. Isaıas et al. (2009) argues that theuse of social networks is critical, as they provide a method of interactionbetween students themselves and between students and teachers. Beyond this,in our work, MOT 2.0 focuses on the novelty in the area of collaborativepersonalised learning and Web 2.0-based adaptation, as well as the interactionthat occurs between the users within the system. The decision to provide thevarious tools in an integrated LMS, as opposed to connecting it to other Web2.0 applications on the Internet (in an ‘‘open world’’, as in Siemens 2008a) istypical especially in the area of learning, in order to have more control over theteaching/learning process. Although here we have also created an LMS we alsoallow for the fact that it is a potentially controversial matter and, in some cases,interfacing of various applications may be very helpful (Ankolekar et al. 2008).

On the other hand, we found that not all social network applicationsprovide a usable interfacing mechanism (public Application ProgrammingInterface or API, web services, etc.), meaning that some social networkscannot be integrated with an LMS. An example of an extremely popular Web2.0 application that has interfacing issues is Facebook*as Google CEOSchmidt (2009) noted:

If we can’t get the data, it’s very, very difficult for us to rank it. Facebook haschosen to keep much of its data behind a wall, that’s what it has decided to do.We favor openness, because we think that works best for the users.

Further related work is that of Bateman et al. (2006), who designed a newapproach for creating meta-data for learning resources, not by using semanticweb ontologies, but by allowing the creators (i.e. the students) to add new meta-datavia collaborative tagging websites. Our approach relies on the students notonly for creating the meta-data (tags in the case of MOT 2.0), but also for othersocial features, such as rating, feedback, as well as collaborative authoring.

On the other hand, work such as that of Alexander (2006) tried to answerquestions about how Web 2.0 can be used to create new ways of teaching:such as wikis, blogs and social bookmarking. For example, the socialbookmarking feature in Web 2.0 can be used in the universities betweenstaff, students and teachers, to quickly find the shared bookmarks of otherusers. In our work, MOT 2.0 uses other Web 2.0 features inside the system,rather than adding wikis, blogs, etc. to MOT 2.0.

Moreover, the work in Rollett et al. (2007) focused on using blogs, as a newway of learning, and concluded that it will be hard to pass on some Web 2.0features*such as trust, openness, voluntariness and self-organisation*to the


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

traditional existing learning method. In our approach, we control this aspectof the system by defining sets of groups, and assigning different privileges fordifferent users in different groups.

Finally, the work in Franklin and van Harmelen (2007) concentrated oncontent sharing via Web 2.0 mechanisms in higher education, and reportedsome problems, which are out of the scope of this paper but worthconsidering:

The introduction of Web 2.0 systems into HE is not without problems, as thereare ramifications in the areas of the choice of types of systems for institutionaluse; external or institutional hosting; integration with institutional systems;accessibility; visibility and privacy; data ownership, IPR and copyright formaterial created and modified by university members and external contribu-tors; control over content; longevity of data; preservation; information literacy;staff and student training; and appropriate teaching and assessment methods.

Looking at adaptive e-learning and its related researches, such as adaptiveeducational hypermedia, intelligent tutoring systems, etc, we find a greatbody of research, and quite a few applications have been written andimplemented.

Our work on the adaptive aspects of e-learning follows prior research andimplementations, such as the work of De Bra (2002), who has been creatingthe adaptive engine AHA!, that tracks user’s browsing behaviour to decidethe user’s knowledge, background and interests. However, AHA! does notsupport social features such as tagging, rating and feedback as in MOT 2.0.The personal reader in Dolog et al. (2004) uses semantic web technologies toenrich and personalise the learning resources. This work is very interesting,but is still in progress. Our approach does not rely heavily on semantic webtechniques, although import from RDF, for instance, is possible. Further-more, the work in Carro et al. (1999) describes a way of creating adaptiveweb-based courses. Adaptivity in Carro et al. (1999) is implemented bydisplaying different HTML pages based on the students’ profile, theirprevious actions and the active learning strategy. However, no socialinteraction between the students and the system is provided, as in MOT 2.0.

A part of the work has focused on benefiting from the adaptivity ine-learning via standards, as in Paramythis and Loidl-Reisinger (2004), whodefined adaptive learning as follows:

a learning environment is considered adaptive if it is capable of: monitoring theactivities of its users; interpreting these on the basis of domain-specific models;inferring user requirements and preferences out of the interpreted activities,appropriately representing these in associated models; and, finally, acting uponthe available knowledge on its users and the subject matter at hand, todynamically facilitate the learning process.

There is also a significant body of work on peer recommendations. The workin McCalla (2004) argues that e-learning systems can be adapted continu-ously, as the external environment changes. The external environment


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

includes learners, teachers, the learning subject and the technology that is beingused in the e-learning system based on the ‘‘ecological approach’’. In theecological approach, the information about the web content is attached to thecontent as the users access it. This approach can underlie the design ofapplications such as: (1) ‘‘a help seeker’’, to find a learner who can help insolving another learner’s problem; and (2) ‘‘a recommender system’’, torecommend learning content to a learner that is relevant to the learner’s task.In our work, we use the concept of the ‘‘ecological approach’’ to recommendexpert learners and learning content based on the user profile. On the otherhand, the work in Zheng and Yano (2007) proposed the use of contextawareness to support peer recommendations in e-learning systems. Thecontext awareness model has three dimensions: knowledge potential, socialproximity and technical access. Our approach does not rely on the context ofthe learner, but it instead uses user profiles to provide recommended learningcontents and recommended users.

The learning environment described by Yang (2006) consists of threesystems: (1) peer-to-peer content access and adaptation system; (2) personalisedannotation management system; and (3) multimedia real-time group discussionsystem. It uses the ubiquitous learning paradigm, with features such asidentifying the right collaborators, right contents and right services in the rightplace at the right time, based on a learner’s surrounding context, such as whereand when the learners are (time and place), what the learning resources andservices available for the learners are, and who are the learning collaboratorsthat match the learners’ needs (Yang 2006). Our approach does not rely on thecontext of the learner, but it instead uses user profiles to provide recommendedlearning contents and recommended users.

Last but not least, the work on adaptive collaborative learning is important,but not yet well-represented. Whilst collaborative learning is a major strand,applying adaptation to such learning is less represented. A related work isthat of Meccawy and Blanchfield (2008), who reported a case study fordelivering adaptation through Moodle. However, their approach does not yetadapt to social aspects of the LMS, nor to the group of learners. In MOT 2.0,we do cater to groups and social aspects*however, we do not connect to anyLMS directly, but we provide a set of converters (Ghali and Cristea 2008) whichcan import from or export to e-learning standards. The social aspects in MOT2.0 can be used in the recommendation process (i.e. recommend expert learners,recommend content with rating ]threshold, recommend a group with contentthat has highest rating, etc.). Another work (Kumar et al. 2007) reported thatstudents learned significantly more when they worked in pairs, than when theyworked alone, which reflects the importance of adaptive collaboration learningsupport. Similarly in MOT 2.0, we focus on the adaptive collaborative learningsupport by using social and grouping features. Additionally, the work inGaudioso and Boticario (2003) described how to build user models in a web-based collaborative learning environment. Their user models allow for acombination of machine learning and knowledge-based techniques. In ourwork, we build the user model based on the user knowledge as well as their


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

actions in MOT 2.0. Furthermore, the work in Alfonseca et al. (2006) suggestedthe possibility of improving collaborative learning, by grouping students inspecific ways, based on information stored in their user models. In MOT 2.0,the grouping mechanism is slightly different, as it is based on the given course.However, it is more flexible, as the teacher can create multiple groups andassign different privileges to different users based on their knowledge, or otherparameters. CoREAD (Chiarella and Lajoie 2006) is a system also belongingto the collaborative learning category. In CoREAD, students can annotate atext with highlights, in order to mark important sections of it. To some extentunsurprisingly, the results of the case study did not suggest that the stable textsignals assisted readers to produce better summaries. This may be also due tothe fact that it is unclear why a given student has highlighted a certain part of atext. In contrast, in our system, clear numerical (such as ranking) and textual(such as tags and feedback) annotations are clarifying the intent of studentswhen marking a certain document or piece thereof.

Social navigation (Dieberger 2000) is a relatively new stream within adaptivecollaborative systems, which explores methods for utilising users’ explicit andimplicit feedback in order to support navigation (Brusilovsky et al. 2004).Social navigation does not necessarily apply to educational systems, but thereare examples within which are dedicated to education (e.g. Knowledge Sea II,Brusilovsky et al. 2004). Knowledge Sea II uses a ‘‘footprint’’ approach(Wexelblat and Mayes 1999) to suggest to a current learner other pathsfrequently visited by students. Whilst the visual feedback is interesting, thesimple fact that various documents were visited is less informative than theapproach we are using in recommending documents, where we take intoaccount not the visits, but the rating of other students given to a certaindocument, as well as other meta-data, such as tags used and feedback. In thisway, a more precise guidance is possible. Comtella (Vassileva and Sun 2008) is apeer-to-peer file and bookmark sharing system aimed at e-learning. Unlike inour approach, the focus there is on increasing participation in collaborationbetween peers. In our system, due to the combination of adaptive collaborationand adaptive personalisation, collaboration is desirable, but not the ultimategoal. Moreover, no bookmark sharing is available in our system, however,recommendations for content are available at various levels of granularity, notonly at the level of whole documents. EDUCO is another social navigationsystem (Kurhila et al. 2002), which aims at helping students find other studentswhich work on a selected document, as well as identify ‘‘popular’’ documents,on which many students have been working (similar to Knowledge Sea II).Again, our approach is slightly different, as not only the fact that manystudents have worked on a document is important in our approach, but alsohow well that document (or part of it) was rated. Moreover, our system doesn’trecommend students that are working presently on a document (or part of it),but students with higher knowledge on that lesson, as will be detailed in thefollowing sections.

Summarising, whilst there are clearly many other important works in thesefour research areas, our work’s uniqueness and novelty are based on bringing


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

these areas together, and benefiting from this synergetic approach, as it isillustrated in a succinct way in Figure 1.

3. Overall collaborative adaptive e-learning 2.0 scenario

In order to understand the full range envisioned by MOT 2.0 for a learner, weillustrate it by one hypothetic generic learner scenario, below. For easycomparison, this scenario is, to the extent possible, reflected in the concreteexamples of the actual tool, presented in Figures 3�5. However, this learningscenario is more generic than the ones used in the real life classroomevaluations. These classroom evaluations are aimed at a sub-set of specificdesirable features only, and thus will later on (see Sections 7 and 8) bedescribed via targeted evaluation scenarios. The generic scenario presented inthis section is created based on feedback from previous evaluations of the first

Figure 2. Social LAOS Framework.

Figure 3. Screenshot of learning environment of Group 1.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

prototype of the tool, as well as by expanding the desired functionality tobetter represent our overall goals.

3.1 The new (e-learning 2.0) learner scenario: adaptive content and recommended

peers

Jane is a student of a course on ‘‘Data Mining’’, and she has just reached themodule (equivalent here to a chapter) on ‘‘Collaborative Filtering’’. Jane feelsat an impasse and would like some help. Unlike in a regular learningsituation, where she would need to find herself the relevant literature toreview, via various channels, or the appropriate people to talk to (teachers,colleagues, etc.), in the MOT 2.0 system, she can request help directly from




Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

her learning system, or, alternatively, the system could automatically detecther problematic state. As a result:

(1) The system can then recommend to Jane a specific item to read, e.g. the‘‘Introduction’’ from the same module. Alternatively, the system couldrecommend her another whole module to read, which would have beenan optional prerequisite, but she may have missed, or just a relatedmodule, such as the module on ‘‘Data Mining for Web personalization’’.Furthermore, the system could recommend a certain item in anothermodule, for instance, the ‘‘Data Mining for Web personalization’’module, which would be most relevant for her current situation.

(2) At the same time, or alternatively, the system can find Jane a peer to helpher. This peer could be John, who is a student of the same course, and is(perhaps only) slightly more advanced than Jane. The fact that John ismore advanced can be determined by the system via non-intrusivemethods (such as page visits, and the fact that John has simply readmore than Mary on this module), or via (more precise) intrusive methods(such as prior test results). The system facilitates for Jane to contact John(via a procedure in which both Jane and John can allow/deny the contact:e.g. Jane may contact John only if she is in the same group as he is; orJane may request the contact, and John may receive this request and beable to allow/deny it; or John may state if he is generally willing to helppeople with incoming requests*as e.g. as in Fetter and Gross (2009),where a user can set his state to ‘‘ReadyToHelp’’; etc.). An incentiveneeds to be created for John to help Jane (either just group membershipentitles group members to ask for help, or reputation within or outsidethe system, etc.) Please note that, again, we don’t mention here exactmechanisms of how this is to be implemented, but there are severalalternatives possible. A simple one is that John’s status in the systemwould change from ‘‘silver’’ membership to ‘‘golden’’, if he helps enoughpeers. Later in the paper we propose helping others as a method ofachieving deeper understanding (as in ‘‘learning by teaching’’).

(3) Also at the same time, or as an alternative, the system could recommendthat Jane join a certain group (e.g. the ‘‘Collaborative Filtering’’ group).This group could be useful because many discussions on CollaborativeFiltering have already taken place, and joining will allow Jane to accessthese; or it could be useful because this group is working on a project onthe same topic, and collaborating with them will give Jane better insightin the topic; etc. In the latter case, the system could also recommend awork distribution to Jane (e.g. Jane would be appropriate for writing ashort essay on ‘‘Data Mining leading to Collaborative Filtering’’, but notyet on ‘‘Implicit Rating’’, for instance.

The first recommendation represents an example of recommended interactionwith content, whereas the following two represent recommended interactionswith people (here, peers). In the following, we analyse in more detail whatthese scenario snippets mean in terms of required system features.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

3.2 Conclusions and required features derived from the learner scenario

(1) In terms of the overall mechanism of interaction, the first desired featureis that of user-centred approach: the student should be able to select ifthe adaptation is triggered by the system or by herself, thus selectingbetween adaptivity and adaptability. Both approaches are known to haveadvantages and disadvantages. Amongst the most known are the factthat adaptive systems require little or no effort from the user, whereasadaptable systems allow the user to be in control. However, as Fischer(2001) shows, implications reach the knowledge representation, which isinternal in adaptive systems, and external (extended to the human user)in adaptable systems, thus requiring different mechanisms (more complexin the case of the adaptive systems). Here we take the pragmatic approachthat both types should be possible, as well as any combination on the axisranging from full system control to full user control (as supported byTsandilas and schraefel 2004 and Cristea et al. 2007, amongst others).

(2) In terms of recommended interaction with content, one of the otherdesired features is that of adaptive content recommendation, at variouslevels of granularity: recommendation of whole modules, recommendationof items in modules, etc. This idea is similar to some of the adaptiveeducational hypermedia systems, however, not identical, as mostrecommend only items in the current module. However, the granularityof the recommendations is a less explored area. Moreover, therecommendations should be also based on the relation between thecurrent item (or module) and another item (or module). This relation canbe the prerequisite relation, a similarity measure, or something else.Triggers of recommendation can be user triggers (such as asking for help,selecting some options) or system triggers (tracking of user’s clicks,scrolling, time spent on a subject, tracking of completion: e.g. recom-mending another module if the current one is finished very early on, etc.).It is to be noted here that, whilst we use the term ‘‘recommendation’’when we talk about content that is suggested to a learner, one of themajor differences between recommender systems and adaptive educa-tional systems is that the latter usually presume (implicitly or explicitly)an order between the recommended elements (a path through thecontent, relations between more than two modules/items/steps), andthus the most frequently used relation is the prerequisite, whilst therecommender systems usually do not presume any order or multi-dimensional relation. Moreover, content model-based recommendationsare inspired by recommender systems and not usually found in adaptivehypermedia (Brusilovsky 1996), which use simple user model-basedrecommendations. Here, we envision a combination of the two.

(3) In terms of recommended interaction with peers, we extract the followingrequired features:a. Another desired feature is that of peer recommendation. The

recommendation of peers can also take place at different granularitylevels, such as recommending a peer that is generally more


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

advanced in the topic, at the level of the whole module or at the levelof the current item, for instance. The triggers for peer recommenda-tion are similar to the ones for content recommendation. Pleasenote that here, ‘‘peer’’ is used for both Jane’s colleagues, as well asteachers or other relevant people who are reachable via the system.This perhaps controversial notion is based on the SLAOSphilosophy of smooth transition from students to teachers andauthors (see Ghali et al. 2009)*the latter being ‘‘just’’ users withmore rights (and experience) in the system and beyond.

b. The system should allow for communication channels between peers.These may be a simple chat tool, or a more complex Voice overInternet Protocol tool (VoIP, also known as Internet Telephony),with or without video or image transmission. However, thesecommunication channels should be monitored in order to be able touse this data in recommendation of peers. Thus, the communicationdata (including the people communicating) is to be annotated for itspotential use (e.g. user John is annotated ‘‘ReadyToHelp’’, as inFetter and Gross 2009). Please note that here, again, the type ofmonitoring is not specifically mentioned, for the purpose ofgenerality. Examples can be monitoring of frequently answeringpeers, monitoring of keywords (to connect topics of conversationwith actual peers), etc. This can be then used to recommend peerswho are more likely to answer, peers who discuss topics related tothe current user’s interest, etc. All these are directly dependent, ofcourse, of the privacy settings and accepted levels of ‘‘invasion’’ ofprivacy by the system users.

c. The system should allow for group formation and dissociation, formeta-data describing the groups (e.g. in terms of members, modulesthat are read by given groups, tasks group perform, interests, etc.).

In this way, we have obtained an initial taxonomy of what the system shouldbe able to do. This taxonomy is exhaustive in terms of the space defined bythe scenario, and the three axes of the taxonomy are mutually exclusive.

In the following, the theoretical framework of the work will be sketched asa basis for the two software development cycles described.

4. Social LAOS framework (SLAOS)

A framework, the SLAOS framework (see Figure 2) has been previouslydeveloped and extended for the overlaying of social, Web 2.0 functionality,over an existing adaptive hypermedia framework, LAOS (Cristea and DeMooij 2003).

It is not the purpose of this paper to go into more detail about the SLAOSframework (Cristea et al. 2010). Here we will only revisit the components thatare relevant to the discussion of the prototypes and evaluations and we brieflyexplain the mapping between the SLAOS and its implementation via the


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

MOT 2.0 system; please refer to Cristea et al. (2010) for the completemapping.

The Resource model is the one storing the items that students are reading ata time, within a module. The social overlay over the resources is representedby meta-data such as ratings, feedback, etc. The Resource Model in SLAOS isrepresented as a collection of items in the MOT 2.0 system. The item refers tothe smallest data entity. For example, the ‘‘Introduction’’ described in theprevious learning scenario is an item.

The User model allows for the storing of data about a learner, teacher orauthor (e.g. the time spent on learning for a learner) as well as data aboutgroups, rights of both users and groups, etc. Thus, the social aspects of theuser model (overlaid over the initial adaptive personalisation model) dealwith interaction of multiple users and with groups of users, instead ofindividual users only. This is an essential deviation from regular adaptivehypermedia frameworks and systems, which traditionally deal only with the‘‘current student’’. Personalisation to this user usually takes place in a solitaryenvironment, not influenced at all by the progress, interest or any otherparameters regarding other users. The User Model in SLAOS is mapped intouser/group variable-value pairs in the MOT 2.0 system. For example, the userJane in the previous learning scenario can have a user model variable‘‘requiresHelp’’ which is set to the value of True.

The framework allows for higher level specifications, such as goals ofthe adaptation process (here, the learning process) via the Goal model andmore abstract handling of learning content via concepts in a Domain model.In the current two implementations, however, the three models Resource,Domain and Goal are implemented ‘‘collapsed’’ over each other, for

Figure 6. Scenario Steps.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

simplification. More precisely, the Domain Model in the MOT 2.0 system isintroduced as a hierarchical representation of items, grouping them intomodules. The hierarchical structure permits the reusability as the same itemcan be used in multiple modules. The module refers to the taught course. Forexample, the ‘‘Collaborative Filtering’’ module described in the previouslearning scenario represents a module. The Goal Model in MOT 2.0 assumesthat the hierarchical relations in the domain model are prerequisites, unlessconditions are added for individual items/modules to lift this assumption. Asan example of a changed condition, the previous lesson (module) on‘‘Collaborative Filtering’’ can additionally be allocated a set of recommendedmodules, based on some criteria (e.g. similarity between topics). Therecommendation is described in the next section (the second prototype).

Moreover, the framework allows for representation of external environ-mental parameters, such as weather or physical device, etc.; however thismodel is not yet implemented in the current two iterations of MOT 2.0, inorder to simplify and focus the evaluations more on Web 2.0 adaptation only.The Environment Model in SLAOS is thus mapped to the physical device inthe MOT 2.0 system. The types of physical device media can be PDA,desktop computers, laptops, etc. There is a need to adapt to the nature of thismedia, even if the user is the same, as different screen sizes can affect theinformation transmitted. As an example, in the previous learning scenario,Jane was using her laptop (with web browser) in order to access to MOT 2.0,as the default physical device. Other environmental factors can be researchedin different areas, such as weather, geographic location, eye-tracking, etc.

The Presentation Model in SLAOS is mapped onto presentation states, suchas showing/hiding the recommended learning content and the recommended(expert) learners in the MOT 2.0 system. Overall, the presentation model hasthe role to decide what, where and how something is being shown to the user.Moreover, more complex presentations can be created, such as in deciding howthe screen is to be used for the specific presentation, what is to appear where onthe screen, etc. Figures 3�5 show the three different screenshots of the MOT 2.0based on the group.

The Adaptation model creates different presentations (specified via thePresentation model), based on the combination of items from the othermodules. For instance, if a rating of an item in the Resource module is abovea threshold, and the User model shows that the user has a low amount ofknowledge about that topic, that item can be recommended by theAdaptation model algorithm, and thus presented via the Presentation model.In classical adaptive hypermedia, adaptation can be written as a statemachine, generating presentation Pt�1, or user model updates Ut�1, basedon the adaptation function At and the previous user model state Ut:

(Pt�1; Ut�1)�At

where t, t�1 refer to two consecutive states and P is thepresentation model, and U the user model


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

In the SLAOS framework, in a next state, presentation changes and usermodel updates can also take place. However, additionally, resources can beadded, dynamically extended, new Goal model features can be added, so theoverall formula is more generic, as follows:

(Pt�1, Ut�1, Rt�1, Dt�1, Gt�1, Et�1)� At(Ut, Rt, Dt, Gt, Et)where additionally, R is the resource model, D the domainmodel, G the goal model, E the environment model

Furthermore, it is to be noted that User model updates reflect both on thecurrent user’s user model update, as well as on group updates, or even generaluser model updates. The Adaptation Model in SLAOS is mapped onto therecommendation strategies in the MOT 2.0 system. As an example, see inSection 5.2 the recommended learning content strategy, and the recom-mended (expert) learners’ strategy.

5. Prototypes

In this paper, we concentrate on the second prototype. However, in order toexplain the changes introduced, as well as the new set of evaluation goals, webriefly describe the outcome of the evaluation of the first prototype, and thenmove on to the second one, as follows.

5.1 First prototype

The first prototype of MOT 2.0 (Ghali and Cristea 2009a) concentratedmainly on a high level balance of the three roles: learner, author and teacher,and what adaptation meant to these roles. The complexity of the adaptationwas not the issue (thus classical adaptation based on similarity of content wasapplied), but the focus was on the type of adaptation necessary (or possible).Thus, a version of requirement (2), adaptive content recommendation, wasimplemented for all the three roles (learner, author and teacher), and resultswere collected based on this.

Another issue analysed was group formation and rights within a group*asper requirement (3.c), for all roles (beside learner requirements, we haveelicited for the first prototype requirements for the author and the teacher forthis purpose). Thus, members of the same group were given similar rights, tobegin with, thus keeping one variable constant, in order to focus on the studyof the impact of group formation in e-learning 2.0.

This first study was concentrating on the new roles of learner, author andteacher in the Web 2.0 world, related to research such as that of Siemens(2008a). One of the main ideas derived by this first research and prototypewas that of the permeability of these roles: in a Web 2.0 environment, learnerscan become also authors, to a degree (for instance, they can add feedback toan item, they can tag items) and thus also implicitly teachers, to some extent(students who come after them benefit from these comments and tags and canlearn from them). On the other hand, authors can also be supported in their


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

authoring, by having appropriate content recommended to them (as perrequirement (2), but applied for authoring) and thus learn something fromthe system (or their peers).

Moreover, another theoretical result was as follows: if we analyse thebalance between the roles from the point of view of the rights in a Web 2.0environment, authors can be defined by the fact that they have more rightsthan students, in terms of them being able to edit more items and modules.However, students also contribute. In principle, well-performing studentsshould be able to ‘‘gain’’ more rights by their ‘‘good behaviour’’ or goodresults, and thus be able to achieve editing rights for certain items, or evenmodule creation rights, and thus becoming ‘‘full-fledged’’ authors. This ideawas further implemented and used in the second prototype, which is the mainfocus of the current paper.

The evaluations of the first prototype analysed also specifically the usefulnessof the Web 2.0 features (such as grouping, tagging, rating, subscriptions) in ane-learning context, which were overall appreciated as being useful (Ghali andCristea 2009a). These evaluations were performed both with designers, as well asstudents (as authoring (Ghali and Cristea 2009b) and learning were bothanalysed, along with the possibility of collaborative authoring and the idea ofpermeable rights). Thus, in the present study, these issues are not furtherexplored. A remaining question was that of the usefulness of a varying degree ofrights given to students, which was further taken into consideration andimplemented in the second prototype.

5.2 Second prototype

The MOT 2.0 system used in the second round of evaluations was centred onthe Learner scenario only. Hence, this paper focuses only on the learners andtheir needs. Concretely, the adaptation was enhanced with three specific typesof adaptation, which we wanted to analyse in more detail:

. two types of adaptation (i.e. system-driven adaptation):(adaptive recommendation of learning content, (corresponding to a partial

implementation of requirement (2) in the Learner scenario in Section 3);and

(adaptive recommendation of peers (corresponding to a partial implemen-tation of requirements (3.a) and (4�3.b) in the same Learner scenario inSection 3);

. one type of adaptability (i.e. user-driven adaptation):(adaptable user privileges (corresponding to a version of implementing

requirement (1) from the Learner scenario in Section 3, of giving thelearner control).

The main focus of the second experiment was on the two types of adaptationenvisioned above. This allowed us to study the effect of overlaying adaptivecontent recommendation and peer recommendation, respectively, over a Web2.0 learning environment. Please note that, although the overarching Learner


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

scenario, as introduced in Section 3, ultimately aims at combining thesevarious types of adaptation, in the evaluations we took care to isolate thedifferent types, in order to study their effects separately.

Below, a simplified version of the algorithms created for this secondprototype, corresponding to the two types of adaptation and the one type ofadaptability, is presented.

5.2.1 Recommended learning contents. Recommending learning contents is thetypical endeavour of adaptive hypermedia. Various techniques have beenanalysed and implemented (Brusilovsky 1996). However, most adaptivehypermedia systems are a ‘‘one-player-game’’, i.e. they adapt to only thecurrent user, usually without any external influence from other users. Here, thepurpose was not to use a content-based recommendation method only, but touse a method that also utilised the Web 2.0 characteristics of the environment:in this case, average rating given to an item by users who have read it. Thus, theresulting mixed methodology uses content-based parameters in the form ofthe similarity measure (here, at the granularity of whole modules) as well as therating of the module to be recommended.

RM�0;If ((Similarity (M, Mi)�Value_1) &&

(Rating (Mi)�Value_2))RM�RM @ Mi;

where:

. RM: is the Recommended Modules set for the current module, M.

. Similarity (M, Mi): is the cosine similarity between the current module M,and any other module Mi stored in the MOT 2.0 system. The similarity iscalculated between two strings, each string representing the tag (keyword)set that best describes the module. Each item is represented as a set ofkeywords describing the content of this item. As each module consists of aset of items, therefore, the module’s tags represents the union of all items’tags. We have used this simplified formula, as currently we were not aimingat obtaining necessarily the best module recommendation*a good enoughsolution suffices*the main target is the comparison of these recommenda-tions in the context of Web 2.0 features whilst adding peer recommenda-tions in the same context. Other possible formulas, beside cosine similarity,can be used, which would take into account that the rating (value) of amodule can be different from the average rating of its components.

. Mi: refers to the other modules in MOT 2.0; the variable i is the moduleidentifier.

. Value_1: is the desired threshold value of the similarity, which can beany value between 0 and 1. This is a simple adaptive content recommenda-tion procedure, where modules are recommended to users, depending onwhere they click (thus depending on what items they’ve selected). Please


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

note that Value_1 is determined by the teacher to match the courserequirements.

. Rating (Mi): represents the average rating of the items in the module. Eachmodule consists of a set of items, and each item has a rating.

. Value_2: the desired threshold value of the average rating, which canbe any value from 1 to 5 (between the maximum and minimum values).Please note that Value_2 is also determined by the teacher to match thecourse requirements.

. Concluding, content is recommended to learners (as per requirement (2) inthe Learner scenario) when the content is similar to the current module, onlyif, however, the rating of that content is high. This means that the behaviourof other users influences the possibility of content to be recommended.

5.2.2 Recommended (expert) peer learners. A specific advantage in e-learning 2.0is the fact that the collective knowledge of other users can be exploited: theuser is not a singular entity anymore, and other users can help him. Thiscorresponds to requirement (3.a) (and implicitly, (3.b)) in the Learner’sscenario, and has been implemented additionally in the second version ofMOT 2.0.

Various algorithms could have been used, but for the purpose of this study,just a simple one based on recommending users of higher (in fact,‘‘acceptable’’) knowledge has been implemented, as follows below.

RU�0;If (User (knowledge, M) �� Value)

RU�RU @ User;

where:

. RU: the Recommended Users set for the current module, M.

. M: the current module.

. User (knowledge, M): the user’s knowledge level for the current module,M.

. Value: is the desired category for the knowledge level, which can be one of‘‘beginner’’, ‘‘intermediate’’ or ‘‘expert’’.

The value used in the experiments was Value�‘‘expert’’, thus recommendingexpert learners to other users. The inclusion of a user to a certain categorywas determined automatically, via tests.

Please note that whilst this peer recommendation mechanism is notdependent on the current user characteristics, it does, however, take intoaccount the overall distribution of user knowledge. This represents anintermediary step between adaptive systems and recommender systems.

Moreover, the application of the two recommendation strategies (contentand peers) is based on the group the student belongs to. For example, learningcontent recommendation is used for Group 2, and recommended expertlearners are solely recommended to Group 3. In turn, group membership is


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

derived based on individual user characteristics (here: knowledge level, asexplained in Section 7.1).

5.2.3 User’s privileges based on the knowledge. Finally, the following adaptablestrategy was selected to determine the user rights in the e-learning 2.0environment. Based on a test, the user knowledge could be determined, andthus the exact set of rights for a user could be set. Every time the user wouldtake the test, these rights would be updated*thus the procedure was entirelyuser-driven (as per requirement (1) in the Learner scenario).

If (User (knowledge, M) �� Value){

CanView�V1;CanRate�V2;CanTag�V3;CanFeedback�V4;

}where:

. M: the current module.

. User (knowledge, M): the user’s knowledge level from the current module,M.

. Value: is the value of the knowledge level, which can be ‘‘beginner’’,‘‘intermediate’’ or ‘‘expert’’.

. V1, V2, V3, V4: is a Boolean value of 0 or 1, which determines if the userhas a privilege or not.

For instance, an expert was allowed to view, rate, tag and feedback*as hisopinion was considered to count more than that of an intermediate or abeginner. The exact settings are further detailed in the case study. To test theappropriateness of content and peer recommendation in the e-learning 2.0setting, we have separated the two types of adaptation, and formed ane-learning 2.0 control group. However, the adaptive conference of rights couldnot be entirely separated, due to the fact that Web 2.0 rights (annotation,rating, feedback) affect not only the user on which they are bestowed, but alsothe other users as well. Hence it was considered important to bestow theserights in proportion to the knowledge the learner demonstrated, as furtherexplained in Section 7.1.

With these overall algorithms implemented in the system, we could form anumber of hypotheses to be tested, which are introduced in the following.

6. Hypotheses

The overall motivation for the second version of the MOT 2.0 system is theassumption that the recommendations of other learners and learning contents


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

will increase the learners’ effectiveness, efficiency and satisfaction in thelearning process. Thus, in short, we have the following:

. Purpose of adaptation: the adaptation can be used to increase the learningoutcome using the social Web.

. Method of adaptation: the adaptation can take place by using therecommended learning content and recommended peers (experts).

. Method of evaluation: by a case study followed by a questionnaire toevaluate the hypotheses. MOT 2.0 tool will be used with real users in real-world use.

Therefore, the main Null-hypothesis that we are trying to refute is:

. H0: MOT 2.0 does not influence the learning outcome.

The counter-hypotheses to refute H0 are:

. H1: MOT 2.0 increases the learning outcome (for the learners who needhelp by further reading of related recommended content or by beinghelped by recommended peers).

. H1.1: MOT 2.0 increases the effective learning outcome.

. H1.2: MOT 2.0 increases the perceived learning outcome.

. H2: MOT 2.0 decreases the learning effort (for the learners who need helpby further reading of related recommended content or by being helpedby recommended peers)

Further hypotheses we are analysing are:

. H3: MOT 2.0 increases the satisfaction (for the learners who needhelp by further reading of related recommended content or by beinghelped by recommended peers).

. H4: MOT 2.0 is easy to learn and use (i.e. MOT 2.0 functions andscreens are easy to understand).

. H5: MOT 2.0 is easy to remember (i.e. MOT 2.0 functions and commandsare easy to remember so that the learners do not have to learn it againwhen they log in again).

7. Case study

7.1 General description of the case study

The case studies are the main evaluation methods of MOT 2.0. They are usedto field test the system components. In particular, the overall aim of thecurrent case study is to explore:

. Recommended learning content based on the learner’s profile.

. Recommended users (peers) based on the learner’s profile.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

These two adaptation mechanisms need to be analysed separately, in orderto make sure that their distinct characteristics are measured, and in ordernot to obtain skewed or correlated results. However, the two adaptationmechanisms are not applied in a ‘‘simple’’ e-learning system, but integratedin an e-learning system with Web 2.0 functionality (such as tagging, rating,etc.). Thus, the evaluation is performed in the specific context of Web 2.0e-learning or e-learning 2.0.

The measures and the criteria of this case study are: usefulness of therecommendations of users and learning contents, which includes theefficiency and effectiveness of the recommendations of users and learningcontents, as well as the satisfaction of the learners about the recommendedlearning content and the recommended other learners. The overall aim for thecase study is to analyse the validity of the hypotheses introduced in theprevious section.

We have performed the evaluations with 24 students in computer scienceat the University of Warwick. These students were studying a moduleentitled ‘‘Dynamic Web-based Systems’’. The students were a mixture of4th year MEng and 1st year MSc students. The scenario (described below)was applied to these students during their regular studies, as one of theseminars/lectures, with the topic of ‘‘Collaborative Filtering’’. The timeallocated for the seminar was 2 hours, but students could spend less (ormore) on their study, depending on their needs. The class the case studywas performed in is the last in the students’ programme (normally, between4 and 6 pm) and thus students don’t have any other class afterwards torush to, and can spend as much (or as little) time as they wish.Participation in this type of study was treated as any participation in aseminar or lecture: thus, it was not compulsory. Students would be able toleave at any time, even if they wouldn’t have finished their work, or staybeyond the 2 hours allocated, if necessary for them to finish. Also, studentswere clearly told that, whilst the topic is part of their curriculum topics,and thus is useful to learn for the final exam, none of the work theyperformed during the class is marked in any way, or affects in any waytheir final grade (including, specifically, negative feelings for the tools,methodology, etc.). The teacher in charge of the class was not present, inorder not to add any pressure on students.

7.2 Scenario steps

In order to establish the effect of recommendations of content, peers or thelack of recommendations in the context of Web 2.0, MOT 2.0 can recommendcontent, as an adaptive hypermedia system, but at the same time, it canrecommend peers that can help with the learning process (e.g. recommendingmore experienced peers, that could help the student with the currentquestions). This effect of recommendations or lack thereof is further evaluatedfor different types of students, grouped by their knowledge level. Thus:


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

. the participants take a pre-test to determine their knowledge level (for aselected domain) out of: beginner, intermediate and advanced.

Next, this knowledge level is to be used in two types of adaptation: one is peerrecommendation, and the other one is adaptation of user privileges. Thus:

. Based on the knowledge level, the participants are given a different set ofprivileges:� Beginner users can only read the learning material.� Intermediate users can read the learning material, as well as add

comments.� Advanced users are allowed to read the learning material, edit the tags,

rate the content and add comments. Additionally, advanced learners areexpected to act as peer experts on the topics that they were classified as‘‘advanced’’ on, and thus be able to answer questions from their peers.

. The participants are asked to accomplish a learning goal using MOT 2.0(i.e. to learn a specific lesson on ‘‘Collaborative Filtering’’).

. In order to achieve the learning goal, the participants are divided into threesub-groups:� Group 1: the first group, acting as a control group, would perform the

learning activity by using MOT 2.0 without any help from the contentrecommender, or the (expert) learners’ recommender. However, thisgroup, just like the others, would benefit from Web 2.0 support. Thus,e-learning 2.0 is the starting point for this research. Arguments of howe-learning 1.0 is useful for learning are to be found in prior research(Ghali and Cristea 2009a). Moreover, this group also benefited fromadaptive user privileges, dependent on their knowledge level, asdescribed in Section 5.2.

� Group 2: the second group would learn by using MOT 2.0, with the helpof recommended learning content, but without the help of therecommended learners. Thus, this group has been created to evaluatethe benefits of recommended content, in the context of e-learning 2.0.

� Group 3: the third group would learn by using MOT 2.0, without thehelp of the recommended content, but with the help of the recom-mended learners. Thus, this group’s role is to inspect the benefits of peerrecommendations, in the context of e-learning 2.0.

Additionally, efforts were made to balance the groups, as follows. The processused for division into groups aims at distributing participants of each of thethree knowledge levels, beginners, intermediate and advanced, as evenly aspossible between the three groups. This is done in order not to have one groupoutperforming the other due to its ‘‘lucky’’ repartition of students. Therepartition is based on the pre-test.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

The three screenshots in Figures 3�5 show the different views upon thelearning environment available to the three groups.

All groups view the learning content in the middle part of their screens (seeFigures 3�5).

Social actions such as rating, tagging and feedback (typical of Web 2.0settings) are available to all users in all groups (as long as their knowledge levelpermits it). This means concretely that all environments (see Figures 3�5) arepresent on the right side of the screen: rating (current rate of the item displayedin the centre of the screen, as well as allowing a user to rate), tagging (currenttags of the item displayed in the centre of the screen, as well as allowing a userto add new tags) and finally, feedback (feedback from other users, as well asallowing the current user to add his own tags).

The module structure is also available for all three groups (this is thehierarchical view of the module on the left side of each of the screens inFigures 3�5).

The differences are that Groups 1 and 2 also see other modules (Figures 3and 4), on the lower left side of the screen (all modules in the case of Group 1,with no adaptation; and recommended modules only, in the case of Group 2).

Also, the last figure (Figure 5) shows that in the case of Group 3, expertsare recommended (right middle side of the screen) and communicationfacilities are available (also right side of the screen, lower part of the screen,chat window). Thus, the learning environments of Groups 1 and 3 differ intwo aspects: ‘‘All Modules’’ and ‘‘Chat tool’’.

The design decision in this case study was mainly focusing on comparingthe learning outcome between the social environment (Figure 3), the socialenvironment with recommended learning content (Figure 4) and the socialenvironment with recommended expert learners (Figure 5).

Therefore, the design decision was focusing on excluding the recommenda-tions for Group 1 (Figure 3), adding only the recommended learning contentfor Group 2 (Figure 4) and adding only the recommended learners and thechat tool for Group 3 (Figure 5), in order to isolate the impact of therecommendations. However, all groups benefited from adaptive user privi-leges. As they were evenly distributed among groups (due to the approxi-mately even distribution of beginner, intermediate and advanced students togroups), it is reasonable to assume that this adaptation doesn’t influence thedifferentiation process.

. After learning, the participants take a post-test (which is identical to thepre-test) to determine the learning outcome by comparing the pre-test andpost-test answers for each learner.

. The effectiveness of the previous three groups (in terms of learningoutcome) is to be determined by comparing the pre-test and post-testanswers for each group.

. Finally, the participants answered a questionnaire about the systemusability.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

. During the learning activity, the system can log the following activities ofthe participants:� Date and time of reading the learning content (items).� Date and time of reading the recommended learning content (items).� Date and time of answering the pre-test.� Date and time of answering the post-test.� Comments of the participants.� Tags added by the participants.� Rating added by the participants.

. Based on the answers of the post-test, the learner’s knowledge level isupdated accordingly.

. After updating the user profile, the learner’s privileges are updated as well.

These scenario steps are depicted also in Figure 7.

8. Results

In order to establish the validity of the Hypothesis H1.1, we analyse theactual learning outcomes of the students.

Overall, the students performed better in the post-test, when compared tothe pre-test, and only in avery few (5 out of 21) cases did they perform similarly,as can be seen in Figure 8. The post-test average was 8.38, and 2.17 markshigher than the pre-test, on a scale from 0 to 10 (0*worst, 10*best mark).

In the following, we analyse first overall, and then based on groups andinitial levels (beginner, intermediate and advanced) the statistical significanceof the student results.

8.1 Comparison of pre-test and post-test for all students

Firstly, we analysed if there is a significant difference in the learning outcomefor all students in all three groups learning with MOT 2.0. The results areshown below. The test used is the paired T-test, as the marks for all students

Figure 7. The pre- and post-tests of all students, and their respective groups.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

were compared before and after the learning took place and the data werechecked for normal distribution with the Anderson�Darling test.Paired T for pre-test-post-test

N Mean StDev SE Mean

pre-test 21 6.286 1.793 0.391

post-test 21 8.381 1.465 0.320

Difference 21 �2.095 1.814 0.396

95% CI for mean difference: (�2.921, �1.270)

T-Test of mean difference�0(vs not�0): T-Value� �5.29

P-Value�0.000

From the initial 24 students that took the pre-test, only 87.5% took thepost-test. This is due to the fact that the whole activity was not strictlycompulsory. The three students that didn’t take the post-test had a very widespread of initial marks (4, 6 and 7, respectively), so the decision cannot beattributed to their knowledge level. They were also from different groups;thus, the group distribution cannot be associated with their choice. Asstudents were allowed to stay as long as they wished (there were no otherclasses after the one they took), it also was not a matter of running out oftime (at least, from the point of view of allocated time; personal timingreasons may have played a role there).

For the remaining 87.5% students that took both pre-test and post-test, theP-value clearly shows that significant learning took place for all students (thepost-test average value is higher than the pre-test average mark with 2.17, andthe difference is significant with confidence interval 95%).

Next, we analyse the learning that took place in each group (g1�g3).

Paired T for pre-test g1-post-test g1


pre-test g1 7 6.143 1.345 0.508

0

5

10

15

20

Q1 Q2 Q3 Q4 Q5

Overall

Overall

Figure 8. The overall answers to the subjective perception questionnaire (Ox-axis: questions

Q1�Q5; Oy-axis: sum of numerical values of student answers).


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

post-test g1 7 8.143 1.676 0.634

Difference 7 �2.000 1.414 0.535


T-Test of mean difference�0 (vs not�0): T-Value��3.74

P-Value�0.010



pre-test g2 6 6.00 2.45 1.00

post-test g2 6 8.50 1.52 0.62

Difference 6 �2.50 2.59 1.06

95% CI for mean difference: (�5.22, 0.22)


P-Value�0.064



pre-test g3 8 6.625 1.768 0.625

post-test g3 8 8.500 1.414 0.500

Difference 8 1.875 1.642 0.581



P-Value�0.014

The students of Groups 1 and 3 have shown statistically relevant increasesin their marks (with 2 and 1.8, respectively, in the confidence interval 95%).The students of Group 2 have a higher average increase in the mark (2.5) butthis is not statistically significant for the 95% interval*but only in the 90%interval.

We have also looked into the question if the learning outcome ishigher for students that are categorised as beginners, intermediate oradvanced (given the average of 6.2 for the lot). We have analysedbeginners with marks of 5 and below, intermediates with mark 6, andadvanced students with mark 7 and above. The results are displayed andanalysed below.

Paired T for pre-test beg-post-test beg


pre-test beg 7 4.429 0.535 0.202

post-test beg 7 7.857 1.773 0.670


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

Difference 7 �3.429 1.988 0.751



P-Value�0.004

Paired T for pre-test int-post-test int


pre-test int 6 6.000 0.000 0.000

post-test int 6 8.167 1.472 0.601

Difference 6 �2.167 1.472 0.601


T-Test of mean difference�0 (vs not�0):T-Value��3.61

P-Value�0.015Paired T for pre-test adv-post-test adv


pre-test adv 8 8.125 1.246 0.441

post-test adv 8 9.000 1.069 0.378

Difference 8 �0.875 0.991 0.350



P-Value�0.041

These results are interesting, as they show that for all three categories,beginner, intermediate and advanced students, there was a positive learningoutcome of statistical relevance in the 95% interval. However, the marksaverage difference is different for the three categories: 3.4, 2.1 and 0.8,respectively, showing the largest benefit for the beginner students, the next forintermediate students and the least for advanced students. Thus, although allstudents benefit from the learning environment, and the learning outcome ispositive in a statistically significant way, they benefit in different ways. Thesystem is bringing the students closer together to a common denominator,which is overall statistically significantly higher than where they started.

8.2 Students’ Perception Questionnaire

The following questions were used to elicit the students’ subjective perceptionof the tool. Please note that the questions here are simplified somewhat, forthe sake of space. Questions asked to students included definition of learningoutcome and neutralisation of each question. Questions appear paired withthe hypothesis they target:


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

(1) I believe that MOT 2.0 changes the learning outcome.(In-creases/ Decreases/ No influence). (H1.2)

(2)Compared to other learning systems, MOT 2.0 is: Easier/More difficult/Neither easier nor more difficult.).(H2)

(3)Compared to other learning systems, MOT 2.0 is: Better./Worse./Neither better nor worse. (H3)

(4)I believe that the interaction with the system is: Easyto learn./Hard to learn./Neither easy nor hard. (H4)

(5)I believe that the interaction with the system is: Easyto remember./Hard to remember. Neither easy nor hard.(H5)

The answers were mapped over the numerical values of {�1, 0, 1}, with ‘‘�1’’ representing the negative answer (e.g. for Question Q1, ‘‘Decreaseslearning outcome’’), ‘‘1’’ representing the positive answer (e.g. for QuestionQ2, ‘‘Easier’’) and ‘‘0’’ representing the neutral answer (e.g. for Question Q3,‘‘Neither better nor worse’’). The assumption underlying this mapping is thatthere is an implied monotonicity in the answer range, as well as a symmetric,equidistant relation between the positive and the negative answer. This is arelatively strong assumption, but which we consider compatible with the typeof answers we have selected.

From the 24 students that took the pre-test, 87.5% took also thequestionnaire (the same 87.5% that performed the post-test, as the twowere linked in a sequence in the testing environment). The overall resultsrelative to their answers to the five questions above are presented in Figure 9.

The questionnaire results represent the students’ subjective perceptionabout the MOT 2.0 system (as opposed to the test results, which arerepresenting the objective measure of the learning outcome). About 85.7%students have declared that the MOT 2.0 system changes the learningoutcome, and only 14% have declared it has no influence on their learning

Figure 9. The Durations of pre-test, post-test and learning session (in minutes) for all students

(AVG�average), as well as for the three groups.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

outcome. Students were also asked to comment on their answers, but not allof them did. Out of the 14% that declared that the system has no influence ontheir learning outcome, only added a comment, as follows:

I don’t know, I didn’t get to see the group 1 or 2 tracks.

Clearly here the issue was that the student believed the question to refer tothe learning experience of all groups, and not just his own. Question 2 directlyasks the students to compare MOT 2.0 with other learning systems in termsof difficulty of use. Question 1, on the learning outcome, was left on purposesomewhat open*we did not mention improving learning with respect tosomething specific (as in Question 2), leaving it up to the students to comparewith what is most familiar to them*which is not necessarily the traditionallearning method, because students learn in a variety of ways at our university.We also had to define the notion of learning outcome for them, as not allstudents would know what this means. These precautions unfortunately didnot prevent all possible misconceptions*and having done evaluations withstudents for many years, we are aware that it is practically impossible toremove (or think in advance of) every possible misconception. We do know,however, that the students at our university are very much used to answeringquestionnaires, and know not to be afraid of any consequences*so if theirperception would have been negative, it would have clearly shown.

Overall thus, a statistically significant majority felt the system helped inlearning, regardless of the group to which they belonged, thus acceptingHypothesis H1. The next question of equal majority of opinion (85.7% of thestudents) is the one concerning user satisfaction, which is increased by usingthe system. The statistically significant positive result confirms HypothesisH3. For Question 4, corresponding to Hypothesis H4, 81% of the studentsbelieve that the interaction with the system is easy to learn. Only one studentselected ‘‘hard to learn’’, but s/he didn’t comment on the answer to explainwhy she felt that way. The same student was the one that selected later forQuestion 5 that the system is hard to remember (thus showing consistency ofopinion). The same student also considered that the system has no influenceon the learning outcome; this perception, however, is contradicted byhis objective results (six in pre-test and eight in post-test). Thus, althoughthis student learned, he didn’t feel as if he did, which can be interpreted bothpositively (learning was unobtrusive and effortless) as well as negatively(learning could have been harder than normal; he would have expected tolearn more from the system). In terms of learning to use the tool, and thenegative feelings of this student, a colleague in the same Group 3 (withrecommended peers), who actually selected ‘‘easy to learn’’ may be shadingsome light on the potential issues with using the tool and learning how to useit, as follows from her comment:

Some elements aren’t as obvious as they could be but it’s generally very easy.I wasn’t sure how to use the chat function immediately and it would help if thefeedback box had a text input to start with. Red/green on the topic tree isn’t


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

useful to red/green colour-blind (1 in 10 men?) but the icons serve the samepurpose it seems so that’s ok.

Overall, however, a statistically significant majority confirmed that MOT 2.0is easy to learn, and thus Hypothesis H4 is thus accepted. Fewer students(76%), but still a statistically significant majority believe that the systemis also easy to remember. Only one student very briefly comments on thisquestion, as follows:

few elements so easy.

Only 67% of students believe that MOT 2.0 is easier to use than othersystems, the other 33% believing it is neither easier nor more difficult. Pleasenote that the Warwick students involved in this case study haven’t seen anyprevious version of MOT 2.0*the first prototype was tested with studentsfrom earlier years. However, Warwick students are familiar with onlinelearning systems of the type provided and maintained locally by the university(Sitebuilder, forums, blogs*see Ghali and Cristea 2009c).

Overall, all five hypotheses were accepted, based on statistically significantresults, as shown below:

One-Sample T: Q1, Q2, Q3, Q4, Q5Test of mu�0 vs not�0

Variable N Mean StDev

SE

Mean 95% CI T P

Q1 21 0.8571 0.3586 0.0782 (0.6939, 1.0204) 10.95 0.000

Q2 21 0.667 0.483 0.105 ( 0.447, 0.887) 6.32 0.000

Q3 21 0.8571 0.3586 0.0782 (0.6939, 1.0204) 10.95 0.000

Q4 21 0.810 0.512 0.112 ( 0.577, 1.042) 7.25 0.000

Q5 21 0.762 0.539 0.118 ( 0.517, 1.007) 6.48 0.000

Next, we attempted to evaluate the questionnaire results from the point ofview of the group that a respective student has been placed in. Thus, for eachgroup (G1, G2, G3), we re-examined the answers to the questions (Q1�Q5)and thus the rejection/confirmation of the respective hypotheses (H1�H5).The results of this analysis are shown below. Thus, a result of PB0.05 forQi_Gj confirms that for group Gj hypothesis Hi stands (is valid), as for allQuestions Qi the results were in average positive.

One-Sample T: Q1_G1, Q2_G1, Q3_G1, Q4_G1, Q5_G1, . . . Q5_G3Test of mu�0 vs not�0

Variable N Mean StDev

SE

Mean 95% CI T P

Q1_G1 7 1.00000 0.000000.00000(1.00000, 1.00000)* *

Q2_G1 7 0.571 0.535 0.202 ( 0.077, 1.066) 2.83 0.030


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

Q3_G1 7 0.714 0.488 0.184 ( 0.263, 1.166) 3.87 0.008

Q4_G1 7 0.857 0.378 0.143 ( 0.508, 1.207) 6.00 0.001

Q5_G1 7 1.00000 0.000000.00000(1.00000, 1.00000)* *

Q1_G2 6 0.833 0.408 0.167 ( 0.405, 1.262) 5.00 0.004

Q2_G2 6 0.667 0.516 0.211 ( 0.125, 1.209) 3.16 0.025

Q3_G2 6 1.00000 0.000000.00000(1.00000, 1.00000)* *

Q4_G2 6 0.833 0.408 0.167 ( 0.405, 1.262) 5.00 0.004

Q5_G2 6 0.500 0.548 0.224 ( �0.075, 1.075) 2.24 0.076

Q1_G3 8 0.750 0.463 0.164 ( 0.363, 1.137) 4.58 0.003

Q2_G3 8 0.750 0.463 0.164 ( 0.363, 1.137) 4.58 0.003

Q3_G3 8 0.875 0.354 0.125 ( 0.579, 1.171) 7.00 0.000

Q4_G3 8 0.750 0.707 0.250 ( 0.159, 1.341) 3.00 0.020

Q5_G3 8 0.750 0.707 0.250 ( 0.159, 1.341) 3.00 0.020

The results above thus confirm all hypotheses for all groups with aconfidence of 95%. There is one exception (in bold italics above), QuestionQ5 about Hypothesis H5 predicting that the learning effort required to useMOT 2.0 is low, for Group G2, which is the one that received help in the formof recommended content, which can only be confirmed with a confidence of90%. Three students in that group claimed the learning effort is less than forother systems, and another three claimed that it is similar. A possibleexplanation is the fact that the students had already seen other adaptivesystems based on content recommendation previous to this experiment,earlier on in their study, and may have thus felt there is a similarity there*similarity which was not so obvious for the students that were beingrecommended peers (which they hadn’t seen before) or the ones notbenefiting of either of those recommendations.

These positive results, however, do not inform on any significant differencesbetween the groups. Thus, further on, we have examined the time the learnersspent in the various activities. The results are presented in Figure 1.

Overall, as can be seen in the figure and data below, the students have spentaround half an hour (out of the allocated 2 hours) studying the module on‘‘Collaborative Filtering’’ via the MOT 2.0 system*although some clearoutliers exist (a student spending only 13 minutes, and one 58 minutes). Thus,the more representative middle value is the median of 26 minutes.

Variable Mean SE Mean StDev Minimum Median Maximum Mode

duration

of

Learning

31.90 2.68 12.30 13.00 26.00 58.00 26

Although the overall allocated time was 2 hours, the results show, however,that the average time spent really learning (as opposed to answeringquestionnaires of tests) was around half an hour*with large variationsbetween 13 and 58 minutes). Thus, the question arises of whether such a


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

relatively short learning session is not detrimental to collaboration. Forinstance, if someone would have to perform a learning task in half an hour, andat the same time help someone else, he might not be so willing to help, whereasif he would have the same assignment spread over a whole week, he might bemore inclined to assist others as well. A short term study thus may lower theamount of collaboration, whereas spread-out learning sessions would perhapsincrease the amount of collaboration. Generally speaking, whilst this type oflongitudinal studies might reveal interesting aspects of collaboration, they aremuch harder to perform. Moreover, a counter-argument would be that anintensive learning session with clear synchronous presence (and thus a higherguarantee of a quick answer) may be conducive to a higher level ofcollaboration, whereas a spread-out learning session may lower the samecollaboration, due to the uncertainty regarding the speed (and thus thetimeliness) of the result. This is similar to a forum site that has had no activityfor a while, and thus might be less conducive to posting questions to. Thus, webelieve that, whilst a longitudinal study may reveal indeed other aspects ofcollaboration, in fact, it would be a different type of collaboration altogether(geared towards asynchronicity, as opposed to the synchronous collaborationwe have explored). This is allocated to future research, as commented later onin the discussion. However, our main focus is on how adaptation can be used inthis context, and not ‘‘simply’’ on how to use Web2.0 methods in e-learning.From our experience in monitoring the case study, the students in the groupsthat were allocated communication tools were quite happy to use them, even ifthe overall time they spent in the case study is relatively short. The very fact thatthere were no immediate effects of their learning results in terms of marksmeant that they didn’t feel too much pressure, and could explore at leisure allthe features of the tool. Nevertheless, even withstanding the effect of thenovelty of the tool, and the added learning curve of learning how to use it, thestudents clearly showed a positive learning outcome.

Interestingly, on average, the students in the control group (Group 1) spentless time studying than the ones in Groups 2 and 3. As the studying time wasup to them (they all had to finish within an hour, but other than that, no fixedtime was set), this could indicate an increase in interest in studying for thestudents that benefit from recommended content or recommended peers.

To some extent this result could have been predicted. The current MOT 2.0implementation does not function as an adaptive system, where the systemfunctions as a filtering mechanism that removes unwanted or unnecessaryinformation, and thus the time spent on the study could be expected to beshorter, but as an adaptive system that provides extra information,additionally to the main content*in the form of additional content forGroup 2 and recommended peers for Group 3. However, the interestingelement was that there was no compulsion in following the system’srecommendations, as the user is in charge. Thus, this means that the studentsmay have opted to follow the system’s recommendations out of their own will.We have also traced if this was the case, and found that 62.5% of the learnersfollowed the recommendations of the system for Group 2 (with recommended


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

content). We don’t have this data on the students of Group 3, but clearly theyspent extra time continuing their discussion with their recommended peers,ending up spending the longest time. The students in Group 3 were usingrandom usernames, thus the factor of knowing each other well was lessinfluential.

Thus, students that did not belong to the control group opted to worklonger with the system, and learn for a longer period of time. Alsoremarkably, whilst their learning time is longer than that of the controlgroup, their test response time grew shorter. Indeed, students in the controlgroup spent more time in their post-test than in their pre-test*clearlythinking their answers over carefully. Whilst students in Groups 2 and 3 spenta significantly lower amount of time in their post-test (P-value�0.1 for thelatter two groups), Group 2 students halving their test time. This result isclearly based on the students’ choice, as students could remain as long as theywished to complete their activities. As the overall results show that learninghas taken place in all three groups, with statistically significant increase inlearning outcomes (with 90% or 95% confidence), the results of the timing oflearning and tests shows that students like to spend more time in learningenvironments with peer recommendation and adaptive content recommenda-tion, and that they are more confident in their test answers as a consequenceof this extra time spent. Moreover, the decrease in time spent on the testshows that the longer time spent in the system was having a learning effect,and not just being based on other reasons, such as more overhead ordistraction.

9. Discussion

As educators, arguably, we keep looking for better ways to persuade studentsto learn, and spend more time on their learning activities, and less on theirextra-curricular activities. As these initial results show, it looks like environ-ments that encompass the power of features of the social web, buildinglearning communities and adaptive learning systems may be the answer to thisaim. Offering students environments that are close to what they use every dayin their extra-curricular life makes learning more attractive. After theexperiment was over, students have very expressly conveyed their wish for usnot to take the system offline, as they still want to use it later on. Clearly,although none of the students had benefited from a full-blown system with allfeatures (as we tried to separate them in our experiment), the system wasfunctional and interesting enough to make them want to come back for more.

We can also conclude that guided interaction with their peers, incommunities and groups, is a very popular feature. It visibly made studentsspend more time in their learning environment than the students withadaptive content recommendations, and these, in turn, spend noticeably moretime than their colleagues in the control group. As great care was taken inorder to place students of various expertise and knowledge levels (beginner,intermediate and advanced) in the three groups in an even manner, the results


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

can thus only be attributed to the system features, and not to the studentsthemselves. Moreover, whilst the students might have been, to some extent,‘‘gaming’’ with the new features, the extra time spent in the system helpedthem to do their tests faster, thus learning took place during this extra time.Thus, it is possible that this corroborates the well-known wisdom thatspending more time on something leads to better understanding. In ourexperimental case, spending extra time could definitely not be attributed tostudents finding the material harder, due to the fact that followingrecommendations (of either content or recommendations of peers) wasoptional, and if it would have made their understanding more arduous, it isreasonable to think that students would have disengaged from this difficultextra work. On the other hand, the fact that they spent longer is directlyconnected to the fact that the recommendations have them the opportunity toexplore and discover more. Recommendations in MOT 2.0, unlike typicalrecommendations in adaptive hypermedia systems, do not filter and restrictthe search space, but offer extra material. Thus students following thesefurther links and discussion opportunities (for recommended peers) may alsoshow a greater degree of motivation of the students.

It should also be noted that in the experiment we have only tested theshort-term memory, as the test followed immediately after the learningsession. We expect this to be the reason that the learning outcomes ofthe students in the different groups do not differ that much from each other.We also expect that the students that have put the effort into the longerlearning sessions may benefit from the knowledge acquired in the long term.This is to be tested later in the year.

The possible high motivation of students willing to invest more time intheir study could also be explained by the fact that autonomy, relatedness andcompetence (the pillars of self-determination theory; Deci and Ryan 2002) areall being nurtured here in different degrees for the different groups. Autonomymeans allowing the user (here, learner) to make their own choices in mattersconcerning them. For instance, the learner can decide whether or not tofollow a system recommendation, be it in terms of recommended content orrecommended peers to talk to. In both cases, the learner can also decide howmuch time to invest in these extra activities. Competence means exercisingone’s abilities or improving them. Clearly all students were able to improve ontheir abilities during the learning process. Moreover, students could userating, feedback-ing and tagging mechanisms to exercise their abilities anddisplay their understanding of various topics. Finally, peers who wererecommended could exercise their abilities by teaching others*the ‘‘learningby teaching’’ (Frager and Stern 1970) paradigm being considered one of themost successful ways of learning. The last pillar, relatedness, meansconnecting with others as well as feeling socially valued. Clearly studentscould connect with others either directly*through the peer recommendationmechanism*or indirectly*via peer ratings, feedback and tagging. Moreover,and in particular, peers being recommended could definitely feel of use totheir peers and thus socially valued. These pillars of self-determination are


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

known to influence intrinsic motivation (refers to initiating an activity for itsown sake because it is interesting and satisfying in itself), as opposed toextrinsic motivation (doing an activity to obtain an external goal).

One feature which was introduced but not analysed in detail in thequestionnaires is that of adaptive user privileges. Thus, advanced usersbenefited from having more rights of Web 2.0 nature in the learningenvironment when compared to their intermediate level and beginner peers.Revisiting the results of the learning outcome evaluation, they showed that,whilst all students benefited from the MOT 2.0 system, the highest benefitwas for beginner students. This demonstrates at least that having lessprivileges did not affect them in a negative way, and perhaps allowedbeginner students to concentrate better on their learning task. However, theseresults are not conclusive, and the effect of adaptive privileges, both on toolacceptance, as well as on the learning outcomes, need further evaluation.

There was an indirect success of the repartition into groups and theadaptation of privileges based on knowledge level, as there was no obviousdissension due to the fact that some users had more rights (such as feedback,tagging, rating) or duties (such as answering peer questions) than others.However, these results cannot be seen as conclusive, and more research ofadaptive rights and privileges needs to be done.

Also, the design decision in this case study was mainly to focus oncomparing the learning outcome between the social environment (Figure 3),the social environment with recommended learning content (Figure 4) andthe social environment with recommended expert learners (Figure 5).

Therefore, the design decision was focusing on excluding the recommenda-tions for Group 1 (Figure 3), and adding only the recommended learningcontent for Group 2 (Figure 4), and adding only the recommended learnersand the chat tool for Group 3 (Figure 5), in order to isolate the impact of therecommendations. Thus, the learning environments of the Groups 1 and 3differ in two aspects: ‘‘All Modules’’ and ‘‘Chat tool’’ (for talking torecommended expert learners). Group 2 has ‘‘Recommended Modules’’instead of showing ‘‘All Modules’’, like Group 1, but it has no ‘‘Chattool’’. All groups had rating, tags and feedback.

Finally, we consider that the main added value of Web 2.0 applications is toallow users to interact with each other in various ways, beyond the classicaldiscussion groups, chat and VoiP: to allow rating, tagging, commenting oncontributions and thus to some extent to become co-authors of systems. Thisis the aspect of Web 2.0 that we use in this paper. Additionally, openness, as inother Web 2.0 systems, is also possible (in the form of links to externalresources). This also allows for a certain degree of remix of data frommultiple sources (O’Reilly 2007). Moreover, features such as self-organisationare also supported, as students and teachers all contribute to the system,determining thus the transfer of the system from one state to another, tillconvergence occurs (at least in terms of content). Comparing our approach toDownes (2005), e-learning 2.0 should be able to create and supportcommunities of practice, i.e. ‘‘a shared domain of interest’’ where ‘‘members


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

interact and learn together’’ and ‘‘develop a shared repertoire of resources’’(Wenger 1998). Issues arise, according to Downes (2005), when communitiesare ‘‘typically limited to a given group of learners, such as a university class’’,and have a ‘‘fixed start and end-point’’. Whilst this is indeed the case in ourexperiment as described in this paper, these are not limitations of the MOT2.0 system. There is nothing inherent in the system which would limit usage toa certain class only or to a certain period of time only. In fact, users of follow-up versions of MOT 2.0 can also benefit from comments and feedback fromprevious generations of students. Thus, whilst having to control ourparameters for the sake of the experimental outcomes, MOT 2.0 belongsboth to the area of participatory learning (McLoughlin and Lee 2007) as wellas to the area of Web 2.0.

Furthermore, one might claim that adaptive e-learning and Web 2.0-basedpersonal learning environments are representing two opposite and incompa-tible pedagogical approaches (because of their contrasting views on self-directed learning). This paper is trying to build a bridge between these twoislands. The empirical study with the MOT 2.0 system demonstrates thatimpact of content-based adaptation is the largest in case of students who areon the beginner’s level, which is coherent with the idea that novice learnersneed more structured scaffolding.

At the same time, adaptive (personalised) e-learning 2.0 is also related withnotions of connectivism (Siemens 2008b), especially in terms of learning beinga process of connecting specialised nodes or information sources (in our case,navigating through a hypermedia), learning resting in a diversity of opinions(expressed here via peer discussions, feedback, tagging, rating) and happeningin many different ways (here, reading information, participating in discus-sions, in feeback-ing, in rating, in tagging). Learning in such systems meansmore than just knowledge consumption, it is a whole creation process.Learners contribute to the learning environment in MOT 2.0 by addingvolatile information (such as on-going discussions) or more stable informa-tion (such as tags, feedback and rating).

Revisiting the nature and context of the study, with same-time, same-placesubjects and a very focused and artificial task with relatively shallow learningoutcomes, and comparing it with learning over a drawn-out period, in a Web2.0 environment employing a more open corpus, less teacher-imposedstructure, more open competencies to be addressed and an open network oflearners, we can make the following remarks. Firstly, a longer time period ofusage of the system would potentially relay information on long-termmemory. Additionally, however, we foresee issues determined by the factthat synchronous collaboration may become asynchronous collaboration, dueto the unavailability of some of the recommended peers at the same timeas the one at which the learning process of a given student occurs. Moreover,such a perfectly ‘‘open’’ environment suffers from the same problems as anysuch environments, including the Internet: that of broken links, disappearinglinks and information being continuously changed. To some extent, ourcontrolled environment simulated this in the shape of continuously changing


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

recommendations from the systems, from peers, from rating, feedback,tagging, etc. However, the amount of these changes is expected to be muchlarger on the perfectly ‘‘open’’ environment. We expect, however, that most ofour findings are translatable without much effort into such a perfectly ‘‘open’’environment, but further studies are needed to sustain this expectation withevidence.

10. Conclusion

Considering the initial research questions that were the driving force behindthis paper, we can conclude that:

. An e-learning 2.0 environment may be enhanced by the addition of content-based adaptation, not perhaps in terms of increased learning outcome, butin terms of attractiveness and time spent learning by students; additionally,exam time is significantly decreased.

. Moreover, such an environment is enhanced by the addition of peerrecommendation, as the time spent learning is even higher than forcontent-based adaptation; similarly, exam time is significantly decreased.

For all the above paradigms, the perceived learning effect and usability of theparadigm are significantly higher than other systems; however, no differ-entiation in perception between the paradigms could be established.Additionally, we have shown this type of system to help all students, but tohelp beginner and intermediate students more than advanced ones.

For further research, as all of these features were considered separatelyuseful, we plan to introduce them all together, as per the overall scenariospresented in this paper, and analyse their usefulness both in a quantitativeand qualitative manner. Currently, this implementation has been performedand evaluation data are being analysed.

To conclude, we can say that we have performed another step towardsharnessing the power of collective intelligence, and Web 2.0 functionality,augmenting and extending it with the specific type of personalisation andadaptation that it allows. These results feed back in the areas of Web 2.0development, but, also importantly, in the area of adaptation, illustrating thenew possibilities created by the social paradigm.

Acknowledgements

The work accomplished in this paper is supported by the GRAPPLE IST project IST-2007-

215434 and was initiated within the PROLEARN Network of Excellence.

References

B. Alexander, ‘‘Web 2.0: A new wave of innovation for teaching and learning’’, Journal of Learning, 41(2),

pp. 32�44, 2006.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

E. Alfonseca, R.M. Carro, E. Martın, A. Ortigosa and P. Paredes, ‘‘The impact of learning styles on

student grouping for collaborative learning: A case study’’, User Modeling and User-Adapted Interaction,

16(3�4), pp. 377�401, 2006.

A. Ankolekar, M. Krotzsch, T. Tran and D. Vrandecic, ‘‘The two cultures: Mashing up Web 2.0 and the

Semantic Web’’, Journal Web Semantics, 6(1), pp. 70�75, 2008.

Aristotle, 350 BC. Politics. Available online at: http://www.classics.mit.edu/Aristotle/politics.html (accessed

13 January 2009).

S. Bateman, C. Brooks and G. McCalla, Collaborative tagging approaches for ontological metadata in

adaptive e-learning systems, in The Proceedings of the Fourth International Workshop on Applications of

Semantic Web Technologies for E-Learning (SW-EL 2006) in conjunction with 2006 International

Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006), 20 June 2006, Dublin,

Ireland, pp. 3�12, 2006. Available online at http://www.win.tue.nl/SW-EL/2006/swel06-ah06-program.html

(accessed 04 December 2010)

P. Brusilovsky, ‘‘Methods and techniques of adaptive hypermedia’’, User Modelling, 6(2�3), pp. 87�129,

1996.

P. Brusilovsky, G. Chavan and R. Farzan, ‘‘Social adaptive navigation support for open corpus electronic

textbooks’’, in Proceedings of the Third International Conference on Adaptive Hypermedia and Adaptive

Web-Based Systems, W. Nejdl and P. De Bra, (Eds.), Lecture notes in Computer Science, Eindhoven, the

Netherlands: Springer, 2004, pp. 24�33.

R.M. Carro, E. Pulido and P. Rodriguez, ‘‘Dynamic generation of adaptive internet-based courses’’,

Journal of Network and Computer Applications, 22(4), pp. 249�257, 1999.

A. Chiarella and S. Lajoie, Enabling the collective to assist the individual: CoREAD, a self-organising

reading environment, in Proceedings of World Conference on E-Learning in Corporate, Government,

Healthcare, and Higher Education, T. Reeves and S. Yamashita (Eds), Chesapeake, VA: AACE, pp. 2753�2757, 2006.

A.I. Cristea and A. De Mooij, LAOS: Layered WWW AHS authoring model and their corresponding

algebraic operators, in WWW’03, Alternate Track on Education, Budapest, Hungary, 2003.

A.I. Cristea, F. Ghali and M. Joy, Social, Personalized Lifelong Learning, ETLL Book chapter, IGI Global,

2010.

A.I. Cristea, D. Smits and P. De Bra, Towards a generic adaptive hypermedia platform: A conversion case

study, Journal of Digital Information (JoDI) (Special Issue on Personalisation of Computing & Services),

8(3), 2007.

P. De Bra, ‘‘Adaptive educational hypermedia on the web’’, Communications of the ACM, 45(5), pp. 60�61,

2002.

T. de Chardin, ‘‘The Phenomena of Man’’, p. 182, London: Collins, 1955.

E. Deci and R. Ryan, (Eds.), Handbook of Self-determination Research, Rochester, NY: University of

Rochester Press, 2002.

A. Dieberger, P. Dourish, K. Hook, P. Resnick and A. Wexelblat, ‘‘Social navigation: Techniques for

building more usable systems’’, Interactions, 7(6), pp. 36�45, 2000.

P. Dolog, N. Henze, W. Nejdl and M. Sintek, The personal reader: Personalizing and enriching learning

resources using semantic web technologies, in 3rd International Conference on Adaptive Hypermedia and

Adaptive Web-based Systems, pp. 85�94, 2004.

S. Downes, 2005. E-learning 2.0, National Research Council of Canada. Available online at: http://

www.elearnmag.org/subpage.cfm?section�articles&article�29-1

M. Fetter and T. Gross, PILS: Advanced instant messaging in e-learning based on an open

implementation, in Parallel, Distributed and Network-based Processing, pp. 347�354, 2009.

G. Fischer, ‘‘User modeling in human-computer interaction’’, User Modeling and User-Adapted Interaction,

11(1�2), pp. 65�86, 2001.

S. Frager and C. Stern, Learning by Teaching, the Reading Teacher, International Reading Association,

1970.

T. Franklin and M. van Harmelen, Web 2.0 for Content for Learning and Teaching in Higher Education,

2007. Tech. rep.

E. Gaudioso and J.G. Boticario, Towards web-based adaptive learning communities, in Proceedings of the

11th International Conference on Artificial Intelligence in Education (AIED’2003), 20�24 July, Sydney,

Australia, 2003.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

http://www.classics.mit.edu/Aristotle/politics.html

http://www.win.tue.nl/SW-EL/2006/swe106-ah06-program.html

http://www.elearnmag.org/subpage.cfm?section=articlesarticle=29-1










F. Ghali and A.I. Cristea, Interoperability between MOT and learning management systems: Converting

CAF to IMS QTI and IMS CP, AH 2008, in 5th International Conference on Adaptive Hypermedia and

Adaptive Web-based Systems, Hannover, Germany, Lecture Notes in Computer Science, 5149, pp. 296�299,

2008.

F. Ghali and A.I. Cristea, MOT 2.0: A case study on the usefulness of social modeling for personalized

e-learning systems, in The 14th International Conference of Artificial Intelligence in Education (AIED’09),

June 2009, IOS Press, 2009a.

F. Ghali and A.I. Cristea, ‘‘Social reference model for adaptive web learning, advances in web based

learning-ICWL’09’’, Lecture Notes in Computer Science, 5686, pp. 162�171, 2009b.

F. Ghali and A.I. Cristea, Authoring for e-learning 2.0: A case study, in Proceedings of the IEEE

International Conference on Advanced Learning Technologies (ICALT’09), IEEE Computer Society,

pp. 170�174, 2009c.

F. Ghali, A.I. Cristea and M. Joy, Social, Personalized Lifelong Learning, E-Infrastructures and

Technologies for Lifelong Learning (ETLL), IGI Global, 2009.

P. Isaıas, P. Miranda and S. Pıfano, ‘‘Designing e-learning 2.0 courses: Recommendations and guidelines’’,

Research Reflections and Innovations in Integrating ICT in Education, 2, pp. 1081�1085, 2009.

T. Karrer, 2006. What is e-learning 2.0. Available online at: http://www.elearningtech.blogspot.com/2006/

02/what-is-elearning-20.html (accessed 04 December 2010).

R. Kumar, C.P. Rose, W. Yi-Chia, M. Joshi and A. Robinson, Tutorial dialogue as adaptive collaborative

learning support, Artificial Intelligence in Education: Building Technology Rich Learning Contexts that

Work, pp. 383�390, 2007.

J. Kurhila, M. Miettinen, P. Nokelainen and H. Tirri, EDUCO � a collaborative learning environment

based on social navigation, in Proceedings of the 2nd International Conference on Adaptive Hypermedia and

Adaptive Web Based Systems, May 2002, Malaga, pp. 242�252, 2002.

G. McCalla, The ecological approach to the design of e-learning environments: Purpose-based capture and

use of information about learners, Journal of Interactive Media in Education (Special Issue on the

Educational Semantic), 7, 2004.

B. McLoughlin and M.J.W. Lee, Social software and participatory learning: Pedagogical choices with

technology affordances in the Web 2.0 era, in ICT: Providing choices for learners and learning. Proceedings

ascilite, Singapore, 2007, Available online at: http://www.ascilite.org.au/conferences/singapore07/procs/

mcloughlin.pdf (accessed 04 December 2010).

M. Meccawy and P. Blanchfield, Combining adaptive and collaborative learning: A case study, in

IGACLE:Individual and Group Adaptation in Collaborative Learning Environments Workshop at EC-TEL

08: 3rd European Conference on Technology-Enhanced Learning, Maastricht, The Netherlands, 2008.

T. O’Reilly, What is Web 2.0: Design patterns and business models for the next generation of software,

Communications & Strategies, 1, p. 17, 2007. First Quarter. Available online at http://papers.ssrn.com/sol13/

papers.cfm?abstract_id=1008839 (accessed 04 December 2010).

A. Paramythis and S. Loidl-Reisinger, ‘‘Adaptive learning environments and e-learning standards’’,

Electronic Journal of E-Learning, 2(1), pp. 181�194, 2004.

H. Rollett, M. Lux, M. Strohmaier, G. Dosinger and K. Tochtermann, ‘‘The Web 2.0 way of learning with

technologies’’, International Journal of Learning Technology, 3(1), pp. 87�107, 2007.

E. Schmidt, 2009. How Google plans to stay ahead in search. Available online at: http://www.business-

week.com/technology/content/oct2009/tc2009102_694444.htm (accessed 14 January 2010).

T. Segaran, Programming Collective Intelligence, O’Reilly Media, 2007.

G. Siemens, 2008a. Learning and knowing in networks: Changing roles for educators and designers. Paper

105: University of Georgia IT Forum. Available online at: http://www.it.coe.uga.edu/itforum/Paper105/

Siemens.pdf (accessed 04 December 2010).

G. Siemens, 2008b. About: Description of connectivism. Connectivism: A learning theory for today’s

learner. Available online at: http://www.connectivism.ca/about.html (accessed 04 December 2010).

T. Tsandilas and M.C. Schraefel, ‘‘Usable adaptive hypermedia systems’’, New Review of Hypermedia and

Multimedia, 10(1), pp. 5�29, 2004.

J. Vassileva and L. Sun, ‘‘Evolving a social visualization design aimed at increasing participation in a class-

based online community’’, International Journal of Cooperative Information Systems (IJCIS), 17(4), pp.

443�466, 2008.

H.G. Wells, World Brain. London: Meuthuen; Garden City, NY: Doubleday, Doran, 1938.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

http://www.elearningtech.blogspot.com/2006/02/what-is-elearning-20.html

http://www.elearningtech.blogspot.com/2006/02/what-is-elearning-20.html

http://www.ascilite.org.au/conferences/singapore07/procs/mcloughlin.pdf

http://www.ascilite.org.au/conferences/singapore07/procs/mcloughlin.pdf

http://papers.ssrn.com/sol13/papers.cfm?abstract_id=1008839

http://papers.ssrn.com/sol13/papers.cfm?abstract_id=1008839

http://www.businessweek.com/technology/content/oct2009/tc2009102_694444.htm

http://www.businessweek.com/technology/content/oct2009/tc2009102_694444.htm

http://www.it.coe.uga.edu/itforum/Paper105/Siemens.pdf

http://www.it.coe.uga.edu/itforum/Paper105/Siemens.pdf

http://www.connectivism.ca/about.html

E. Wenger, 1998. Communities of practice. Learning as a social system. Systems thinker. Available online

at: http://www.co-i-l.com/coil/knowledge-garden/cop/lss.shtml (accessed O4 December 2010).

A. Wexelblat and P. Mayes, Footprints: History-rich tools for information foraging, in Proceedings of ACM

Conference on Human-Computer Interaction (CHI’99), Pittsburgh, PA, pp. 270�277, 1999.

S.J.H. Yang, ‘‘Context aware ubiquitous learning environments for peer-to-peer collaborative learning’’,

Educational Technology & Society, 9(1), pp. 188�201, 2006.

Y. Zheng and Y. Yano, ‘‘A framework of context-awareness support for peer recommendation in the e-

learning context’’, British Journal of Educational Technology, 38(2), pp. 197�210, 2007.


Dow

nloa

ded

by [

Mos

kow

Sta

te U

niv

Bib

liote

] at

02:

59 2

0 N

ovem

ber

2013

http://www.co-i-l.com/coil/knowledge-garden/cop/lss.shtml

Date post:	12-Dec-2016
Category:	Documents
Upload:	fawaz
View:	215 times
Download:	1 times

Towards adaptation in e-learning 2.0

Documents