Review of Lms Literature for Aut Lms Review Committee

Learning Management Systems: A Review

Prepared for:

LMS Review Committee

AUT University (contact [email protected] )

Prepared by: Sara Bennett

Contact details: Sara Bennett

P O Box 56053 Dominion Rd AUCKLAND

Ph 021 369 039 Email: [email protected]

Report for AUT University LMS Review group May 2011: Learning Management Systems: A Review

Page | 1

Contents

Introduction ..............................................................................................................................2

Literature Review Methods ......................................................................................................6

A Brief History of E-Learning in New Zealand .........................................................................7

Environmental Scan of Selected Universities LMS Change and Transition Decision Making...............................................................................................................................................10

Massey University ................................................................................................................11

Waikato University ...............................................................................................................13

La Trobe University ..............................................................................................................14

Macquarie University ............................................................................................................15

Queensland University of Technology ....................................................................................16

RMIT ..................................................................................................................................17

University of South Australia .................................................................................................18

Critical reflections on LMS .....................................................................................................20

Pedagogy .............................................................................................................................20

LMS, Engagement and learning outcomes...............................................................................23

System (in)flexibility ............................................................................................................24

Inclusion of Web 2.0 technologies ..........................................................................................24

Future considerations for LMS...............................................................................................29

Concluding comments ...........................................................................................................35

References ............................................................................................................................36


Page | 2

Introduction

In October 2010, the AUT University commissioned a literature review of learning management systems (LMS) as one part of a broader project reviewing LMS at AUT.

Learning management systems have been available in their current form since the early 1990s. Internationally and within New Zealand, LMSs have become nearly ubiquitous across the higher education sector as a core component of e-learning (also referred to as blended learning) (Pina 2010). A frequently used definition of an LMS describes it as “a broad term that is used for a wide range of systems that organise and provide access to online learning services for students, teachers, and administrators. These services usually include access control, provision of learning content, communication tolls, and administration of user groups” (Paulson 2002). Most, if not all, Universities in New Zealand use one or more LMS (Elgort 2005). Summarising the extent to which the higher education sector has embraced the use of LMS, Jones comments “it is broadly accepted that the almost universal response to e-learning within Universities has been a selection of an LMS.” (Jones 2009).

Learning management systems are referred to by several names in the literature, including course management systems, virtual learning environments (VLE), and e-learning courseware. The term LMS is more frequently used in the majority of US based publications, and VLE is more frequently used in Europe and Asia (Weller 2007). For the purposes of this report, the term LMS is used synonymously with VLE.

A University LMS consists of many interlinked components, as illustrated below (Wise and Quealy 2006):

LMSs resemble other systems designed for e-commerce, human resources and student records, but what makes an LMS unique is its functionality and instructional nature. Ellis describes a ‘robust’ LMS as a system which has the ability to:


Page | 3

• Centralise and automate administration • Use self-service and self-guided services • Assemble and deliver learning content rapidly • Consolidate training initiatives on a scalable web-based platform • Support portability and standards • Personalize content and enable knowledge re-use (Ellis 2009:1)

Hayward (2009, cited from Adams 2011) describes LMSs through a five level hierarchy of increasing capabilities:

1. Classroom management – facilitate delivery of notes or other learning aids for a particular lecture (e.g., lecturer creates a website to distribute materials)

2. Course management – support to span multiple class sessions across an entire course with common goals, adding tools for evaluation, feedback and discussion

3. Curriculum management – provides meta-tools (e.g., content tagging and objectives management) to handle relationships among a set of courses. These tools can be used to index a curriculum across a programme or identify common attributes across courses

4. Learning management – information is organised around the learner. This facilitates self-directed learning as students can chose from a variety of learning opportunities, and can progress at different rates over time depending on individual goals. Students may have a private area within the system to assemble selected resources (facilitating the use of an eportfolio)

5. Community management – enables borders to extend beyond the class, course, curriculum or the traditional campus learner, allowing for multiple learning contexts and organisations.

Selecting an LMS is a critical decision for any University, and is likely to have a major impact over a number of years. LMSs are available in two broad categories: propriety (paid for) and open source. Most of the propriety systems are based on Microsoft .NET and/or Java technologies (Eckstein 2010). Examples of paid-for LMS include Blackboard, JoomlaLMS, Learn.com, and Saba Learning Suite. Most open source systems are based on Apache, PHP and MYSQL, making installation simple and inexpensive (or free), and the software for each open source LMS is free to download, install, use and update, and all have comprehensive free documentation and forums (Eckstein 2010). Examples of open source LMS include Moodle, Sakai Project, Claroline, and aTutor.

This review concentrates on Blackboard and Moodle, as these are the most frequently used second generation propriety and open source LMS in use in New Zealand and Australia (Bacus 2010; University of Tasmania. 2010). Although definitive data is very difficult to access, it appears that of 33 universities across New Zealand and Australia 29 are currently using a version of Blackboard (including WebCT), 12 are using a version of Moodle, one is using Sakai and one is using Desire2Learn.

There are more than 90 different types of LMSs available (Pina 2010). Second generation LMS are characterised by a shift towards modular architecture designs, recognitions of the need for semantic exchange, integration of standards-compliant platforms and increased shift towards the ‘services’ principle, where aspects of functionality are externally exposed (Dagger, O'Connor et al. 2007). Second generation LMS remain content or teacher centric, rather than learner-centric (Yau, Lam et al. 2009).

Blackboard was established in 1997, and has grown considerably over time through the strategic acquisition of other LMS and non-LMS companies. As a result, Blackboard has a wide-reaching product line. In 2009, Blackboard Learn Version 9 was introduced, which incorporated functionality from earlier versions of Blackboard and WebCT.

Walsh and Coleman (2010) note that the latest version, Blackboard Learn 9.1 also incorporates Blackboard Connect (at an additional cost), which alerts students to deadlines,


Page | 4

due dates and academic priorities within a course. Blackboard 9.1 also allows educators to more easily incorporate videos and photos directly into text for a more complete learning experience. Finally, this version also features Blackboard Mobile Learn (also at an additional cost), which lets students connect to their online courses using various handheld devices, such as the iPhone or iPad.

Blackboard has adopted a modular approach, where the LMS is extended by the incorporation of ‘building blocks’ – applications built by third parties. A small number of building blocks are available free of charge, however most are purchased or licensed annually. Tools included within the Blackboard Learn community are course-centric, and are supplemented with the Content, Community and Outcomes systems to add institutional based content and learning object management, e-library reserves, e-portfolio, and group and user role management (Pina 2010). Due to the significant annual investment required for the Content, Community and Outcomes systems which can triple or quadruple the initial licensing fee for the Learn system many customers only use the core Learn system (Pina 2010). In contrast to Moodle, Blackboard positions itself as pedagogically neutral (Adams 2011).

In New Zealand, Blackboard has been used by many tertiary institutions as an LMS (Winter 2006).

Moodle has been freely available for download and implementation since 2002, and is developed and supported by an active community of developers, users and administrators that keep the software evolving at a steady pace (Pina 2010).

Moodle 2.0 is the latest version, and new features focus on increased usability, including: easier navigation, improved user profiles, community hub publishing and downloading, a new interface for messaging, and a feature that allows teachers to check student work for plagiarism (Walsh and Coleman 2010). Text formats also allow plug-ins for embedded photos and videos in text. Walsh and Coleman (2010) note that Blackboard 9.1 allows for this too. A major improvement in Moodle 2.0 is that anyone can set up a community hub, which is a public or private directory of courses. Also, Moodle now allows teachers to search all public community hubs and download courses to use as templates for building their own courses. Teachers can now see when a student completes a certain activity or task and can also see reports on a student’s progress in a course (Walsh and Coleman 2010).

Moodle LMS design is explicitly conceptualised to support a social constructivist framework of education, where students are actively involved in constructing their own knowledge (Moodle. 2011). The concept behind this philosophy of learning is that learners actively construct new knowledge and they learn more by explain what they have learnt to others, as well as by adopting a more subjective stance to the knowledge being created (Barr, Gower et al. 2007). Pina (2010) comments that the Moodle interface contains a feature set similar to a commercial LMS, however the focus of the interface reflects Moodle’s constructivist roots, and is focused on facilitating communications and social interaction.

In New Zealand, Moodle was reviewed by the Open Source Virtual Learning Environment Project, as part of the identification and selection of a suitable open source learning environment to develop for use in educational institutions. Moodle was shortlisted from over 30 options, and was recognised for its user friendliness, flexibility, excellent documentation and evolution to meet SCORM standards, along with accessibility to developers, modular architecture, and the existence of a lively developer community (Winter 2006).

Identifying LMS market share information is very difficult to ascertain, however there is general agreement that Blackboard and Moodle are responsible for a large section of the LMS market. A recent analysis of LMS market share for US Higher Education institutions was undertaken by the Delta Initiative, a US based higher education consultancy company which assists with the delivery of enterprise wide technology, and findings are illustrated below.


Page | 5

Figure 2: LMS Market Share for All Institutions (US) 1997 –2010. (Delta Initiative, 2010)

Similar trends are also illustrated in the Australian LMS market share. A recent report from the University of Western Australia (Bacus 2010) documents that in 2008, Blackboard accounted for 48% of the LMS market share in the Australia Higher Education sector, WebCT accounted for 43%, and Moodle for 3%. However, considerable change was anticipated across the sector, as 17 Australian Universities were reviewing their LMS options and applications, and the market share for Moodle was expected to grow to 21% after 2011 (Bacus 2010).

Increasingly, reviewers agree that there is little difference between the most recent versions of Blackboard and Moodle’s LMS (Obexer and Bakharia 2005; Feldstein 2010; Momani 2010). For example, commenting on a recent review of Blackboard 9.1 and Moodle 2.0.1 undertaken at the Ruhr University in Bochum Germany (Otto 2011), Thibault states:

Blackboard Learn 9.1 (SP3) and Moodle 2.0.1 are extremely similar in their capabilities in about 95% of the features and tools. With a few extensions and additional plugins ... the LMSs can mirror each other in 100% of the functionalities they set out to provide. (Thibault 2011).

The comparison can be found at http://moodle.ruhr-uni-bochum.de/file.php/1/Feature_Comparison_Moodle-Blackboard.pdf

Ellis (2009) recommends that careful consideration is first given to identifying how an organisation will use an LMS, before matching product functionality to requirements.

Given the essential similarities between LMS, it is important to understand what considerations Universities have given to their decision making about LMS, and what information could support AUT University as part of its review of LMS.


Page | 6

The remainder of this report is structured as follows:

• Literature review methods • A brief history of e-learning in New Zealand • Environmental scan of selected universities lms change and transition decision

making • Critical reflections on LMS • Future considerations for LMS.

Literature Review Methods

Two key approaches were taken to identifying materials for the literature review.

Initially, a search of electronic databases available through the AUT library was undertaken by AUT librarians. An initial key word strategy was agreed in advance with the AUT project team, and combinations of the following search teams were used to identify relevant materials:

Learning management system(s); blackboard; moodle; sakai; vle; lms; virtual learning environment(s); cms; clms; evaluation; comparison.

The following data bases were included in the search:

Science Direct; ERIC; Summon; Cambridge Journals; Scopus; Web of Science and Google Scholar.

A total of 79 documents were identified through this initial search. This material was provided to the researcher, and reviewed according to the following parameters:

• Currency – how could the material identified can build on and support the purpose of the review;

• Source – potential sources of information were identified and prioritised, including academic data bases and sources of unpublished literature (e.g., conference proceedings);

• Reliability and validity – all materials collected were critically reviewed, ensuring they are obtained from credible sources and are appropriate to the project’s purpose; and

• Coverage and relevance – relevance was ensured by assessing that materials included in the review were appropriate to the project’s purpose.

The review was limited to information published in English. The criteria for inclusion in the review was documents produced from 2006 onwards, as well as seminal references published earlier.

Findings from the review of this material were discussed with the Director of Learning and Teaching. Of the 79 references in the data base, 57 were accessible to the researcher – the remainder were unable to be accessed (primarily due to restricted access websites, or conference papers that were unable to be sourced). To supplement the materials identified to date, the researcher also attempted to access information through the AUT on line LMS Wiki. Some difficulties were also experiencing accessing information through this site, as some links were inaccessible.


Page | 7

In addition, difficulties were experienced by the AUT team in accessing a commercial eLearning Guild report which was intended to supplement the peer-reviewed material sourced through the AUT library literature search.

Of the literature that was able to be accessed at this stage, an initial assessment of this information indicated that the majority was not relevant to the agreed parameters of the literature review. Overall, much of the literature that was able to be accessed was not considered to be appropriately aligned with the agreed areas of focus for the review, had limited relevance, and had limited value in relation to reliability and validity.

A meeting was held in early March 2011 with the Director of Learning and Teaching to discuss the literature and the implications for the project to date. The researcher expressed concerns about the literature accessed to date, and agreement was reached to refine and rescope the project.

Agreement was reached that the review would focus primarily on a comparison between Blackboard and Moodle, and some consideration would be given to other examples of LMSs.

A second literature search was undertaken to supplement the initial search findings. This search was informed by materials provided by the Director of Learning and Teaching, and also supported by additional Google Scholar and academic data base reviews. Further references were also sought by searching the reference lists of identified materials.

The second search identified more than 150 document (peer review journal articles, reports, trade journal articles and e-learning and LMS commentator blogs), and the majority were able to be incorporated into the current review document.

A Brief History of E-Learning in New Zealand

Internationally, LMSs have been available and implemented by Universities since the early 1990s. Adoption and implementation of LMSs by New Zealand Universities has been evident since the mid to late 1990s. Since the early 2000s, there has been considerable national discussion and policy interest focused on New Zealand’s e-learning capabilities, which incorporates the use of LMSs (Shephard, Stein et al. 2008). This discussion has occurred within an overall focus on a constructivist education model, which underpins New Zealand’s approach to education and which provides rich active learning environments, and emphasises the active and personal construction of knowledge by the student through experiences and interactions within a learning environment (Papastergiou 2006; Ministry of Education. 2008). Internationally, constructivism has been a key strand of educational discourse for more than 20 years (Conole and Alveizou 2010). Summarising constructivist principles, Papstergiou (2006) notes they place importance on:

• Negotiation of learning objectives • Student control over their learning • Authentic, purposeful and contextual learning • Problem solving • Collaborative learning • Multiple, alternative perspectives • Knowledge construction and validation through action and discourse • Authentic, contextual assessment and • Development of metacognitive skills.

A brief history of key points of e-learning in New Zealand is summarised below, with particular emphasis on LMS related findings.


Page | 8

In 2001, the E-Learning Advisory Group was established to inform strategic and investment decision making and to explore issues related to the development of e-learning across the tertiary education sector. In 2002, the E-Learning Advisory Group produced a vision for e-learning in New Zealand, Highways and Pathways: Exploring New Zealand’s e-learning opportunities (Butterfield 2002) which recommended the establishment of the e-learning Collaborative Development Fund, and the launch of two electronic portals for tertiary education in New Zealand. The report identified a number of key issues, including the importance of sound pedagogical foundations to support e-learning, along with the need for adequate funding and infrastructural requirements to ensure good access and quality learning opportunities. The importance of professional development for teachers was also emphasised, to support effective learning outcomes for students.

Informed by this report, significant capability development initiatives were introduced to support e-learning by the New Zealand Government. For example, in 2003, the e-learning Collaborative Development Fund was established by the Tertiary Education Commission to enhance the ability of the tertiary education sector to use e-learning to improve the quality of education, and to give learners improved access to education. A number of projects were funded through this $28 million, four year contestable fund (Suddaby and Milne 2008). The e-learning Collaborative Development Fund produced a range of investigations into the adoption, development and delivery of e-learning in Tertiary Education Organisations, including an analysis of virtual learning environments (Catalyst IT Ltd. 2004), an analysis of e-learning in Institutes of Technology and Polytechnics (Mitchell, Clayton et al. 2005) and e-learning in industry training organisations (Clayton and Elliott 2007). The Innovation Development fund was also developed as an annual $10 million contestable fund (Suddaby and Milne 2008).

In 2004 the Interim Tertiary e-learning Framework was published, which outlined a vision for “a networked, flexible education system offering accessible, relevant, high quality learning opportunities for all New Zealanders” (p.15). The framework was developed in collaboration with representatives from the tertiary education sector, and outlined five guiding principles for e-learning in New Zealand: learner-centeredness; good practice; collaboration; innovation; sustainability and affordability. The Framework also outlined seven action areas essential for the development of sector capability to support the development of e-learning in New Zealand, including the need for professional development; development of communities of practice; and research.

Building on the Interim Framework, in 2005, the New Zealand e-learning guidelines were released (NZ E-Learning Group. 2006). The purpose of the Guidelines was to support a framework for reflective practice and quality in e-learning among tertiary education organisations, across key audience groups (managers, teaching staff and students) and across teaching activities (learning design, teaching relationships and support). The e-learning guidelines “inform staff of good practice, help in the design of learning and offer a practical entry to discussing quality teaching using e-learning” (White and Milne 2005:17). They reflect key change measures for tertiary education priorities, including the need for greater collaboration and rationalisation within the system; increased quality, performance, effectiveness, efficiency and transparency; and increased responsiveness to the needs of, and wider access for, learners (Suddaby and Milne 2008).

Focusing specifically on LMS implementation, in 2004 a consortium of New Zealand tertiary institutions lead by the Waikato Institute of Technology obtained a grant from the Ministry of Education to support the ‘Open Source Courseware Initiative New Zealand’ (Barr, Gower et al. 2007). The project examined whether faculty at three tertiary institutions perceived Moodle would support e-learning courses which would perpetuate New Zealand’s tradition of constructivist learning, meet the needs of Maori and Pasifika students, and students at risk of failing in the education sector, but be flexible enough to cater to students doing advance study. Findings indicated that Moodle was considered an effective LMS that readily


Page | 9

accommodated higher order learning activities, it was easy to use and appropriate for implementation in New Zealand tertiary institutions. Similar findings supporting Moodle as the core LMS for the New Zealand Open Source Virtual Learning Environment project were reported by the Open Polytechnic of New Zealand (Catalyst IT Ltd. 2004).

In 2004-05 the Flexible Learning Leaders in New Zealand project was developed, to consider how staff development programmes and approaches could facilitate effective adoption of information and communication technology for learning and teaching. The project involved key informant interviews with university staff involved in academic development for e-learning across New Zealand, Australia and the UK (Elgort 2005). Findings revealed that institutional adoption of an LMS was a key catalyst for supporting the adoption of e-learning practices among teaching staff. However, respondents expressed concerns that poorly thought out approaches to using LMS were of serious concern, and could have significant impact on the ability of e-learning to reach its potential.

Other research undertaken at this time also highlighted concerns about the teaching and learning aspects of e-learning in New Zealand. Marshall (2005) evaluated the capability across six (of eight) New Zealand Universities and three polytechnics to sustain and delivery e-learning. Key institutional capability weaknesses were identified directly relating to the teaching and learning aspects of e-learning. For example, learning objectives were used poorly in e-learning papers in most institutions, and there was a lack of focus on learning objectives relating to analysis, synthesis and evaluation. Marshall (2005) also identified a lack of clear relationship between e-learning technology deployed by universities and desired educational outcomes.

In 2006, the Ministry of Education commissioned research to develop a strategic framework to support professional development for e-learning within the tertiary education sector. The research was undertaken by a combined team from Otago and Massey Universities (Shephard, Stein et al. 2008). Although focused on professional development requirements, the report highlighted concerns among tertiary education organisation staff about the impact of infrastructural requirements, institutional structures and capability issues on learning and teaching outcomes, similar to those reported by Elgort (2005).

In 2007, the Ministry of Education commissioned the development of resources to assist institutional leaders to plan and manage their use of e-learning more strategically (AKO National Centre for Tertiary Teaching Excellence. 2008). While focusing on e-learning in general, the report highlights a series of key questions specific to institutions’ considerations of LMS, including:

• How should e-learning be managed within the institution – who should be responsible for the management of an LMS?

• Resourcing – how should a fully functional LMS be resourced, planned and managed? What will an LMS cost? What are costs likely to be relating to the IT environment, software, pedagogical support, and professional development for staff and students? To out-source or not?

• Staff development requirements – what expertise and training is required for e-learning? What guidelines should support this?

• Functionality – what should an LMS be able to do? Which functions are priority?

The report offers a series of case studies from various tertiary institutions around New Zealand about the use of e-learning as examples of practice of strategic leadership.

Current e-learning national policy is described in the ICT Strategic Framework for Action 2008-2012, which focuses on:

• Making optimal use of high-speed internet connections • Collaboration across the sector in the development and sharing of resources and

services;


Page | 10

• Investment in and use of shared digital repositories of content; • Use of ICT to support the provision of life-long learning opportunities • Ensuring that learners and teachers are proficient in a range of current and emerging

Its; • Ensuring that learners, teachers and administrators are well supported in their use of

ICT in their work and that they learn to make effective use of emerging tools for online collaboration and communication;

• Ensuring that all investment in ICT is measured against agreed standards of value-for-money (Ministry of Education. 2006; AKO National Centre for Tertiary Teaching Excellence. 2008).

Overall, the development of e-learning and LMS adoption in New Zealand has paralleled the drivers of the introduction of LMS internationally. Coates and colleagues (2005) summarise the key drivers that have supported universities to introduce enterprise-level LMS including:

• LMS as a means of increasing the efficiency of teaching • LMS and LMS-based resources giving the promise of enriched student learning • Student expectations for advance technologies • Competitive pressure between universities • LMS as part of a cultural shift in education to control and regulate teaching.

While LMS are evident in most universities across New Zealand, and in many tertiary institutions internationally, increasingly there agreement among e-learning researchers that the quantity and quality of learning occurring within these systems remains limited (Beer, Jones et al. 2009; Lane 2009). Specifically, questions remain unanswered about the adoption curves of LMSs, about the pedagogically sound use of these technologies and about the return on investment for institutions in light of the high investments from a human resource and financial costs perspective (Obexer and Bakharia 2005).

Within New Zealand, some universities are at various change and transition points in relation to their LMS use. While initial LMS adoption and implementation was guided and supported by early institutional and national educational policy development about e-learning, current choices appear to be guided by the need to move to more responsive models of LMS to meet emergent teaching and learning requirements. Similar transitions are occurring in other jurisdictions, for example, in the UK many institutions are in the process of reviewing their current LMS (or virtual learning environment) provision in the light of changing pedagogical requirements, more administrative integration and the emergence of new classes of social media on the web (MacNeill and Kraan 2010).

Environmental Scan of Selected Universities LMS Change and Transition Decision Making

This section of the report provides an overview of the use of LMSs at selected Australian and New Zealand Universities. Universities were selected for inclusion in the review if they were similar in key characteristics to AUT University, and had recently reviewed and/or changed their LMS. Summary information is provided about the LMS change, the process of review, timeline, costs, key drivers in the reviews and associated business cases, and the rationale for the final decision. Identifying organisational strategic and operational priorities for investment in e-learning capability is recognised as a significant challenge facing many Universities. Consequently, deciding to review an LMS, and whether or not to change an existing LMS, is a considerable investment for a tertiary institution, and “therefore it is essential to clearly identify the ‘business’ needs of the organization prior to selecting a LMS to implement i.e. what educational and/or training needs is the organization wanting to address by implementing a LMS?” (Eckstein 2010: 1).


Page | 11

A key challenge in critically considering LMS reviews is that there is no shared standard across Universities for what could, or should, be included in a review (Pina 2010). Siemens (October 6, 2006) comments that this reflects “a lack of clear focus on intentions of LMS as a learning support tool.” Criteria that are often considered in reviews are:

• Ease of use by faculty and students • Integration with a learning object repository • Functionality and tools available • Transition ease and cost from existing tool • Integration with other enterprise-wide tools • Extendibility – configuration to the University environment • Cost (Pina 2010; Siemens October 6, 2006)

Overall, there is little publically available information about tertiary institution’s decision making outcomes about LMS review and change processes, and there is even less detailed information focused on New Zealand and Australian Universities. Some information about transition between LMSs is reported in a recent environmental scan from the University of Western Australia, but this information is limited to Australian Universities (see Appendix one) (Bacus 2010). Recently, the University of Tasmania has also undertaken a review of LSMs being used by tertiary institutions in New Zealand and Australia, and the information for New Zealand has been updated by AUT University (see Appendix two) (University of Tasmania. 2010). However, neither of these environmental scans included data about the drivers of change that informed decision making by specific institutions. Information included in the following environmental scan is limited to documents available via the internet, and more detailed information should be sought directly from the tertiary institutions. Extremely limited data (in many cases, no data at all) was publically available about the costs considered in an LMS review, and this is a significant limitation.

Two New Zealand Universities have published detailed information about their LMS review processes. Massey University has published a detailed summary of the selection of their new LMS, including strategic drivers and a high level indication of costs (Brown, Paewai et al. 2010). Additionally, the University of Canterbury has published a comprehensive report of the LMS review process it underwent to compare Blackboard and Moodle (University of Canterbury. 2008).

The following tables provide a summary of the LMS decision making and transition process undertaken at selected universities across New Zealand and Australia. Universities were identified for inclusion in this environmental scan if they had similar characteristics to AUT University.

Massey University (Brown and Suddaby 2009; Brown, Paewai et al. 2010)

LMS Had adopted WebCT from 1998 onwards, transitioned to Moodle (rebranded as Stream) for three year implementation project from 2009 onwards.

Process of Review Involved multiple approaches with a priority focus of ensuring strategic drivers rather than technical considerations drove the process. Strategic drivers based up on clear principles; pedagogical considerations and Massey-specific requirements. Criteria were made available to all staff and the selection process involved extensive consultation through a variety of mechanisms, including:

• Working parities, staff surveys and in-house evaluations • Meetings with representatives from Student’s Federation • National and international benchmarking with other


Page | 12

institutions and organisations • Meetings with senior staff from LMS suppliers

Timeline Review process commenced 2007 with investment plan, development of guiding principles, pedagogical criteria and institutional requirements March – May 2008; Pedagogical reference group May 2008; Institutional endorsement June – August 2008; Business case development August – December 2008

Costs Senior Leadership Team approved business case with $4.5million budget to support infrastructure, and allow for a fivefold increase in pedagogical support. Brown et al (2010) note “Never before had such a level of resourcing been available to support technology enhanced learning within the University” (p.68).

Key drivers from review and business case

Selection and implementation of the LMS was considered a strategic opportunity for the University to redefine its delivery modes to support a more engaging and flexible learning environment. Key strategic drivers were to:

• Respond to the challenge of the ‘Google generation’ • Enhance the quality of teaching and maintain Massey’s

University preeminent status as a distance education provider

• Introduce a new flexile model of teaching that increases the level of student engagement and provides a learning experience relevant to the requirements of the knowledge society (Brown and Suddaby 2009).

Rationale for decision Moodle was the preferred LMS because it:

• Clearly signalled the intention to move from a more formally structured, content and teacher-centered approach to a blended and flexible learning environment where ‘communities of learners’ could be supported

• Provided a consistent style and institutional branding across all courses and programmes, with the flexibility for staff to teach, and students to learn, in different ways

• Offered instructional design features that could blend together print and electronic content, with a capacity to integrate new electronic tools as they become available

• Had the capacity to support Te Reo Maori and other Pasifika languages

• Was used by international leading distance education providers and large multi-national corporations, and supported the University’s aspirations for international delivery.

Final decision Recommendation to adopt Moodle provided by Pedagogical Reference Group, and endorsed by Teaching and Learning Committee of Academic Board, Extramural Students’ Society; Senior Leadership Team; and Academic Board. The goal is to implement Stream in a carefully planned manner on a programme-wide basis in order to fundamentally redesign the student learning experience (Brown, Arnold et al. 2009).


Page | 13

Key description of Institution

Largest university-level distance education provider, with over 18,000 distance learners and over 18,000 campus based learners.

Waikato University (Johnson and Walker 2007; White 2009; White and Robertson 2009)

LMS

Had previously designed and implemented WebCrossing, however support for this was stopped in 2006. Support continued in house, but this was not sustainable. 2007/2008 transition to Moodle.

Process of Review Process of review was challenging because of conflict associated with the legacy system, and lack of time and resources to engage staff in a pedagogical review process. Urgent timeframe to implement Moodle, so University decided to implement risk management strategies to support implementation, including: maximise exposure of teaching and IT staff to Moodle and raise staff awareness of transition issues; provide intensive 1:1 support to pilot staff to gather requirements for Moodle based on actual teaching experience and to inform overall configuration; administer student surveys, staff interviews and research about implementation; transition to full roll out scheduled for Summer School 2008 to further pilot test before full academic year started.

Timeline Moodle piloted and reviewed in 2007/08. Staged approach to implementation: Year One initial project aimed to deploy Moodle; establish appropriate support structures, develop policies and practices around the management of the system and provide staff and students with training to support the transition. Year two project focused on completing the transition, enhancing the environment in line with University requirements, increase volume and quality of usage and decommission the legacy LMS

Costs No information on costs able to be sourced.


Legacy LMS unsustainable, and urgency to change because of limited functionality; problems with performance; no longer supported; limited standard compliance.

Rationale for decision Implementation of new LMS supported by four critical project decisions:

• Full university-wide rollout with staged approach • Developed to ensure no loss of functionality • No automated migration of content • Delayed focus on pedagogy and encouragement not to

implement changes in teaching practice until second phase of project.

Final decision Moodle


Two Campuses: Hamilton and Tauranga, 650FTE academic staff; 10,000 FTE students.


Page | 14

La Trobe University (Curriculum Teaching and Learning Centre. 2010; La Trobe University. 2010)

LMS Blackboard WebCT CE 6.0 no longer supported, decision to move to Moodle 2 from semester one, 2011.

Process of Review Multiple aspects to evaluation of LMS including meetings, presentations, submissions and briefing papers from vendors; staff and student usability testing; accessibility testing; functional criteria; content migration; and vendor responsiveness. Criteria for review included: usability; functionality; technical fit; reliability and consistency of ongoing service; cost; and market share. Comparison between Blackboard 9.1 and Moodle

Timeline Review commenced 2009, content Migration commenced October 2010, advanced teaching with Moodle commenced early 2011 onwards

Costs No information on costs able to be sourced.


Drivers for business case included:

• Vendor ceased further development of LMS and will discontinue further support

• University wanted to align LMS implementation with change in university structure and learning and teaching goals

• Existing LMS cumbersome by current standards.

Rationale for decision Overall, high scores for both LMS options. Blackboard marginally ahead in terms of functionality, ease of use by staff, and accessibility. Wider differences seen between cost structures, flexibility and vendor responsiveness

Final decision Final rational for Moodle considered structural issues: licence cost, other cost (forex), vendor lock-in, flexibility versus risk. Also considered key contextual issues: strategic fit with Design for Learning project, staff opportunity cost versus existing skills (technical fit); content migration issues; and time required to implement.


La Trobe University is spread across seven different campuses, including an inner city site.


Page | 15

Macquarie University (Macquarie University. 2011)

LMS Blackboard WebCT version CE8 no longer supported by vendor; transition to Moodle 2.0 from December 2012 (Learn2012 Project).

Process of Review Review of options undertaken and presented to Management Advising Committee for Academic Learning Technologies in July 2010.

Timeline Commenced review 2010; implementation before December 2012

Costs No information available


Four key streams of strategic and operational considerations:

• Technical infrastructure (including integration to University systems)

• Leading the change (including resource development; communication and change management; training and support)

• Faculty and department liaison • Project management (status and governance reports,

resource management).

Rationale for decision Selection of Moodle was seen as endorsement of an approach that embraced the philosophy of growth, transformation and excellence in online learning. Also aligned to Academic Plan objective to ‘develop the University’s physical and virtual environments to provide a quality learning experience.’

Final decision Learn2012 (Moodle Implementation)


Macquarie is located in Sydney’s north-west high tech corridor, and has 31,000 students (nearly 10,000 part time), and 960 academic staff.


Page | 16

Queensland University of Technology (Mitchell and Obexer 2006; Obexer 2006; Obexer 2006; Queensland University of Technology. 2008)

LMS From Online Learning and Teaching (OLT) system to Blackboard 9

Process of Review Review strategies included: environmental scan, internal consultation about evaluation criteria and requirements for LMS; establishment of detailed and high level evaluation criteria; comprehensive evaluation of three shortlisted LMS compared to existing system; closed tender process focused on selected LMS; establishment of product profiles for selected LMS and assessment of performance against criteria. Informed by robust review processes, LMS options shortlisted to comparison between Blackboard; Desire2Learn and Moodle.

Timeline Commenced in 2005, decision to replace OLT with Blackboard by 2007

Costs Comprehensive analysis of costs indicated that despite high initial costs for the migration of content and re-training of staff (estimated at 259,765 over two years), overall and ongoing costs for maintaining Blackboard would be lower than the maintenance costs for original OLT LMS when calculated over 4 years. Estimated savings is approximately $1.4 milllion over 5 years (Obexer 2006).


OLT system needed either significant re-investment or replacement. Over time, the initial investment in OLT had been superseded by the rapid development of the open source movement, and the development of Blackboard as the ‘industry standard’ in Australia. Learning Environments Working Party also identified that the LMS system change provided a number of opportunities to advance the QUT Learning and Teaching agenda, including opportunities to: develop staff capabilities; reinvest resources required to maintain the OLT system into other activities to support learning and teaching; collaborate with others regionally and nationally; use innovative resources and tools developed by a global community; integrate with other QUT systems.

Rationale for decision Transition to Blackboard seen to provide increased functionality in some areas (e.g., communication tools, content management, student tools); being part of Blackboard developers community offers collaborative opportunities for tool development; easy integration with existing systems; building blocks allow for development and integration of innovative tools; skills of staff and students from other Universities transferable to QUT; proven scalability, stability and robustness; possibility of semi-automated content migration from OLT; over time, reduced costs in maintaining and supporting the system.

Final decision Final decision informed by recommendations from detailed evaluation and unanimous advice from the technical advisory


Page | 17

group in favour of Blackboard. No consensus reached with the academic advisory group (equally split between Blackboard and ‘wait and see’ approach), however other key stakeholders including OLT steering committee, and technical support within the University were in favour of Blackboard. Final recommendation of review was that QUT transition to Blackboard.


QUT is based in Brisbane, with 40,000 students, spread over three campuses, and 4,000 staff.

RMIT (RMIT. 2011)

LMS Transition to Blackboard 7.2 to Blackboard Learn 9.1 on 31 December 2010.

Process of Review Existing material was automatically migrated to the upgraded LMS

Timeline No information available.

Costs No information available.


Review and upgrade was seen as “integral to a reinvigoration and re-brand” of RMIT’s use of learning technologies (RMIT. 2011). Upgrade was intended to provide significant improvement for all users, more contemporary ‘fit for purpose’ system, and incorporate more features for teaching, assessment, student work and course management. Upgrade also introduced flexible and customisable options to improve the teacher/student experience, and presents in a Web interface of similar standard to popular online environments. It was also intended to enable more streamlined integration between Blackboard and other technologies.

Rationale for decision No information available about the rationale for the decision

Final decision Blackboard 9.1


RMIT has three campuses in Melbourne and two in Vietnam, and more than 70,000 students.


Page | 18

University of South Australia (University of South Australia. 2010; University of South Australia. 2010)

LMS From previous home-grown system, UniSAnet, transitioning to Moodle Learnonline. Transition to new LMS be completed by December 2011

Process of Review No information available.

Timeline No information available.

Costs No information available.


Guiding principles identified to support the achievement of the best learning outcomes for all students, including:

• learnonline will be UniSA's primary environment for delivering and engaging students with online teaching and learning, and will include Moodle LMS and other tools to ensure an integrated online experience for students.

• All UniSA courses will have a learnonline presence. • Staff using learnonline will be fully supported by the IT

help desk, online advisers and academic developers. • UniSA staff and students will be encouraged to develop

innovative online practices. • Students will undertake all summative assessment tasks

within the learnonline environment. • UniSA encourages the sharing of good teaching

practices and collaborative teaching. • All material available through learnonline will be

copyright compliant. • Students can expect access via learnonline to recordings

(audio and presentation) of lectures conducted in a main lecture theatre

• Students will have access to course material for courses they have been enrolled in for a minimum of 12 months after the completion of that study period.

• All course material software will comply with accessibility requirements

Rationale for decision No information available.

Final decision Moodle

As noted earlier, other authors have also reported difficulties in accessing detailed information about University’s choices of LMS and transitions between different LMSs (Bacus 2010). While some commentators have encouraged information about this to be shared in the public domain (see for example (Smithers 2009), this has not resulted in dissemination of detailed business cases.

However, some Universities not included in the environmental scan have published relatively detailed information about their LMS change strategies, providing information about business case and strategic drivers, as well as financial costings. Examples include:


Page | 19

• Charles Sturt University, which was the first Australian University to implement Sakai, an open source LMS. Uys (2010) describes the change and innovation process, including the model that supported change, initiatives to build learning communities within the University, professional development requirements for University staff and students, and communication strategies to support the change process.

• The University of Queensland transitioned from WebCT to Blackboard in 2004, and Steel (2005) reported key drivers in the implementation and change process, including practical leadership, cultural and project management strategies used to support implementation.

• The Open University has summarised the key drivers that supported the selection of Moodle, and the implications of this decision for the University (Sclater 2008). Key drivers that encouraged faculty adoption of the LMS are also described.

• The University of Gloucestershire published the review of its transition from WebCT to Moodle, including evaluation of student and staff usability, pedagogical implications, systems capability, administration and support, security and costs (Hill 2008).

Overall, information about costs is particularly limited. Costs relating to the implementation and use of an LMS initially include funds for purchase and maintenance of software and supporting hardware. Direct costs are the most visible expenditure, however it is also important to consider indirect costs such as those relating to the professional development and support of staff using the LMS.

Informed by their analysis of forthcoming future changes in the LMS market, Delta Initiative (2010) outline a set of key lessons for institutional decision making about LMS choices. They suggest institutions should:

• Keep their eye on the big picture: it’s not simply a decision to choose an LMS, but is also a choice about a strategic direction for technology and learning. LMS should be viewed as part of a suite of IT systems, as faculty and students are looking for a seamless interface to facets of their institutional life

• Move ahead on LMS initiatives, but be prepared for innovation and change. Pilot programmes are a useful way of testing options such as mobile, alternative approaches to LMS or social networking, alongside the main LMS initiative

• Start with what you know, and build over time. Progress in stages, not in one jump, as this allows for success over time, clarity of purpose and the ability to course correct

• Base the LMS model on your culture, as this will reflect your local culture and history, and will support flexibility and the integration of LMS other systems

• Focus on needs and use a strategic approach to planning with LMS specific questions

• An inclusive process is as important as the decision itself. LMS is more than an administrative tool, and requires a different approach to decision making. Governance issues are important, as this is an academic and IT strategic decision.

Similarly, reflecting on international trends in change and transition between LMS, Feldstein (2010) highlights that many universities which have multiple campuses, or who work in consortia, appear to be favouring open source LMS such as Moodle, Sakai and emergent LMS models such as Instructure Canvas. Feldstein and other authors (e.g., Delta Initiative, 2010) agree the LMS market is very much in transition. Reflecting on the potential changes to the LMS market that are likely to occur by 2014, Feldstein comments “between movements toward more personal/informal learning environments, the big changes that are happening in the textbook industry as content providers and delivery platform providers collide, and the breathtaking pace of innovation that continues in the consumer web market, I strongly suspect we’ll see a wave of new software that will begin to displace the classic LMS. My guess is that it will be mostly co-existing with current-generation products over the


Page | 20

next few years, but by 2014 we may see it beginning to change the whole picture for technology infrastructure in some fundamental ways” (Feldstein 2010).

Critical reflections on LMS As stated earlier, LMS are used by the overwhelming majority of New Zealand universities, and many universities internationally. However, research into the ramifications of LMSs on teaching and learning is limited and most of the published literature has focused on LMS evaluation to guide selection of a particular LMS, with a secondary focus on ‘lessons learnt’ when transitioning from a LMS system to another (Adams 2011).

Adams (2011) comments there has been little critical discussion relating to the potential shifts in teaching and learning practices, and overall there is little evidence-based research supporting continued investment in e-learning from a pedagogical perspective. The rapid adoption of LMS has occurred in a vacuum of research into their teaching and learning effectiveness (Beer, Jones et al. 2009). Overall, LMS are a “taken for granted teaching technology, based on accessibility and convenience (Weigel 2005), as well as [meeting] perceived student demand for digitally enhanced learning environments” (Adams 2011: 253).

However, there is increasing recognition that “LMS are not pedagogically neutral technologies, but rather through their very design they influence and guide teaching. As the systems become more incorporated into everyday academic practices, they will work to shape and even define teachers’ imaginations, expectations and behaviours” (Coates, James et al. 2005: 27). Mersham (2009) states “the choice of medium or LMS is a major consideration in e-learning. This software facilitates management of educational courses, including course development, presentation and administration.... E-learning pedagogy exists in direct and mutually dependent relationships with the technologies, supporting and allowing certain activities while preventing others” (p.62).

This section of the review outlines key critical reflections on LMS.

Pedagogy In a recent critical review of LMS, Adams (2011) focuses on LMS design and pedagogy, and highlights the importance of considering the question “what explicit or implicit theory of teaching and learning informed the design of a given LMS?” as a key framework for examining the pedagogical issues inherent within LMS adoption.

The pedagogical differences between open source (e.g., Moodle) and propriety (e.g., Blackboard) LMS systems are frequently cited. As noted earlier, Moodle was developed from a social constructionist pedagogy, and in contrast Blackboard positioned itself as pedagogically neutral (although this position is contested by many reviewers, e.g., Adams2011; Lane 2009). Irrespective of their pedagogical positioning, one of the initial promises of LMS development was that the software would be transformative, liberating, and would allow the learner or teacher to choose the methods and technologies that are most appropriate in a given situation. However, “the outcomes have not quite measured up to the hype” (Wise and Quealy 2006: 899). There is increasing recognition that even those LMSs which claim to be are not pedagogically neutral shells for course content – they influence and potentially limit pedagogy by presenting default formats to guide the creation of a course in certain ways (Lane 2009).

Current enterprise scale LMSs were created to manage traditional teaching tasks as if they were business processes, and were designed to focus on instructor efficiency for


Page | 21

administration processes (Lane 2009). Consequently, managerialism may an inevitable result of a centrally managed LMS, and pedagogical choices are significantly restricted. Reflecting on this, Dron (2006) comments that the inherent structure of LMSs imposes a structural hierarchy which constrains users – tools that are provided through the LMS allow and encourage some things, and prevent or discourage others. Mott (2010) refers to this as the ‘troublesome LMS’, and there is a growing body of research that illustrates that instructors overwhelmingly use content distribution and administrative tools in the LMS, while using interactive learning tools sparingly (Chang 2008; Wenmoth 2009; Naveh, Tubin et al. 2010). This is particularly evident in integrated systems (e.g., Blackboard), however Lane (2009) and other authors (Jones 2009) comment that it is also true of more constructivist systems (e.g., Moodle).

Reflecting on the use of LMSs among New Zealand tertiary sector organisations, Marshall (2005) reports that while all tertiary institutions reported using an LMS, many did not place the use of these systems within a framework of strategy and guidance to teaching staff that would transform learning. Marshall comments “there is a definite sense that existing approaches for teaching and learning are being carried over to technology without reflection and planning. A clear example of this is in the absence of linkages provided to students between the learning objectives of the course and the technologies and pedagogies they encounter” (Marshall 2005:5).

Adams (2011) also emphasises the need to consider “how do teachers and students experience teaching and learning in and with these systems, particularly relative to the educational theory the LMS is designed to operationalise?”

Wenmoth (2010) comments that the design of an LMS should be carefully considered before implementation, as there are significant links between online learning space design, and pedagogy. A University’s LMS forms the academic system equivalent of enterprise resource planning systems in terms of pedagogical impact and institutional resource consumption (Jones 2009). Wenmoth (2010) notes that frequently LMS implementation is driven by expediency, costs, limited involvement from teaching staff and post-implementation involvement of end-users to encourage uptake of use. He notes that less frequently he observes consideration of other important drivers which should support LMS vision and decision making including:

• An intensive period of facilitated discussion with teaching staff (as end users) to develop a shared perspective on the linking of pedagogy to online learning space design

• A significant modification of the LMS interface to reflect the pedagogical decisions of teachers, and to provide a coherent experience for all users across the various courses that are provided by an institution

• A strategic integration of the LMS with other appliances to provide the full range of online learning services required to provide the optimal online learning experience

• A significant stream of input from learners to provide user-feedback on usability and the user-experience

• A period of negotiation with staff to develop a set of principles to underpin the interface design across the range of course options provided by the organisation (Wenmoth 2010).

It is interesting to note that new and emergent models of LMSs such as Smartly Edu are focusing on design, with the belief that good design will make students spend more time on line, connected and engaged (Meyer 2010).

One of the negative effects of structural limitations is that they encourage limited use of the LMS, which in turn retains academic staff at the ‘novice’ user level, thereby continuing to restrict use of the LMS. This reflects the affordance of the technology, which is defined the way the application is used in real contexts by experts and novices. Social political,


Page | 22

personal, economic and other contexts constrain and define the ways in which the technology is actually used (Mersham 2009). For example, it is important to consider that academic staff may be at different levels of digital competence in their ability to consider a change in pedagogical practice. Chang (2008) states that digital competence comprises of three levels: 1) factual knowledge, about the IT system; 2) knowledge of the tool in context, or how the system is used within practice; and 3) constructive knowledge – knowing the system and its use in context, as well as being able to use it to develop and change work practices. In order for ICT to change pedagogical practice, a user has to be at the highest level of digital competence with a range of tools, and to be able to think creatively with them, to bring the experience to the next level of making qualitative change to existing pedagogical practice (Chang 2008). The under-utilisation of LMS opportunities suggests that few academic staff transition past levels 1 or 2 of digital competency.

As a response to this ‘LMS pedagogy trap’, Lane (2009) highlights the importance of introducing pedagogy to novice users of LMS before focusing on LMS features and tools, and continuing to provide pedagogical support. Lane differentiates between ‘opt-out’ LMS systems (e.g., Blackboard) and ‘opt-in’ systems (e.g., Moodle). Opt-out systems are considered more likely to overwhelm novice users as all features are available by default, and the tendency is to reduce cognitive load by using the minimal default option. However, in an opt-in system, the instructor selects each activity and presentation system from a menu list, which effectively designs the interface for the students. Fewer defaults are pre-set, which requires the instructor to think holistically about structure, context on a macro level and choice about features and tools on a micro level (Lane 2009). This, Lane suggests, will encourage the instructor to examine and explore pedagogical options more freely.

The use of LMS also fundamentally changes the relationships between teachers and students. Coopman (2009:2) states “more than previous technologies, online learning systems have the potential to enhance the collaborative performative nature of teaching, and at the same time, the potential to turn teaching into a static exercise – textualisation. As performance teaching, online classes allow for continual updating, integration of multimedia, ongoing discussion, and real-time chats. As textualised teaching, online classes become ‘plug and play’, where the person who designed the course may not even teach it and little about the class changes from term to term.”

Reflecting on the design of Blackboard, Coopman (2009) considers that the intensely hierarchical nature of Blackboard persists in producing a textualised approach to teaching and learning. The power to design the online classroom rests primarily with the LMS designers who determine the structure, secondly with the university administrators who determine which features should or should not be included, then with the instructors who determine which features should be available to students, and finally (last and least) with students who determine how they will use the interface within the structure designed by the LMS designers.

The Australian-New Zealand New Horizon report from the New Media Consortium Project (Johnson, Levine et al. 2008) also identified the skills of teachers as a significant priority challenge facing the tertiary education sector over the next few years. The report states “many teachers do not have the skills to make effective use of emerging technologies, much less teach their students to do so” (p.3), and identifies the need to provide pedagogical and capability development support to teachers. Reflecting a similar concern, Adams (n.d.) comments that for an organisation to successfully integrate an LMS that will provide long-standing benefits to students and instructors, steps need to be taken to ensure the quality of instruction is maintained and enhanced throughout the process.


Page | 23

LMS, Engagement and learning outcomes Coates and colleagues comment “if LMS are having widespread effects on the structure of university teaching, they are obviously affecting student study habits and learning” (Coates, James et al. 2005:28). There have been three main approaches to examine LMS usage to assess what, if any indications of effective learning this can demonstrate: 1) ask staff and students through surveys or interviews; 2) manually review course sites and 3) mine system analytics data. However, overall, little is known about the degree to which LMS are affecting student engagement in universities. Student engagement refers to the intellectual, emotional and practical interactions students have with educationally purposeful activities and conditions, and engagement is linked to desired outcomes of academic achievement, and satisfaction (Coates, James et al. 2005).

Clark et al state that LMSs are spaces within which social groups are created, and kept cohesive if the units within the LMS are working together – that includes the tools, academic instructors, students, and communication and collaboration spaces (Clark, Beer et al. 2010). Linking LMS usage to an assessment of engagement and relationships is intended to aid in reflecting on academic practice, and to support facilitation of engaged teaching, including incorporating reflective methodologies, and shifting the emphasis in teaching from content to dialogue (Coates, James et al. 2005).

Beer and colleagues (2009) analysed LMS systems analytics data and investigated LMS ‘clicks’ by undergraduate students as a proxy indicator of course engagement. Engagement was defined as including active and collaborative learning, participation, and communication among teachers and students. Acknowledging that learning is more complex and diverse than demonstrated in this model, findings indicated a positive correlation between the number of student clicks on LMS courses and their resulting grade. Student grades were increased where they had participated in discussion forums with teaching staff, compared to students where staff did not contribute to discussion forms. These authors suggest this emphasises the importance of employing discussion forums as part of on-line teaching, and encouraging teaching staff to adopt this LMS feature (2009; Beer, Clark et al. 2010). Similar findings have been reported by other authors (Dawson and McWilliam 2008).

Also using LSM analytic data, the University of Wollongong has developed a student network visualisation tool, SNAPP (Social Networks Adopting Pedagogical Practice), to analyse the network of interactions resulting from discussion forums and replies (University of Wollongong. 2009). The data is generated from the LMS, and includes reports on the number of sessions (log-ins), dwell time (how long the log-in lasted) and number of downloads. Network visualisations of this data provide an opportunity for a teacher to identify patterns of user behaviour, at any stage of the course. The network diagram can:

• Identify at risk or disconnected students • Identify key information brokers within a class • Identify potentially high and low performing students to support interventions being

planned even before work is marked • Indicate the extent to which a learning community is developing within the class.

The use of LMS by teachers is also an important part of student engagement – the quantity and quality of teacher presence are important and influential factors in developing and maintaining student online engagement (Dawson and McWilliam 2008). Clark and colleagues (2010) also investigated LMS analytics data to demonstrate that there are links between an academic’s approach to pedagogy, utilisation of features within an LMS and use by both academics and students, which together create involvement and engaged learning. Acknowledging that the academic is only one part of interconnected learning, Clark et al suggest the following components are necessary in combination to support engaged teachers and engaged learners:

• an academic with understanding of their teaching approach,


Page | 24

• a place within an LMS to create space for communication, collaboration and content dissemination,

• a sound understand of assessment and the need for this to be authentic • student motivation (Beer, Jones et al. 2009; Clark, Beer et al. 2010; Clark n.d.).

A recent meta-analysis of student educational outcomes from online learning found that, on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction (US Department of Education Office of Planning Evaluation and Policy Development. 2010). The difference between student outcomes for online and face-to-face classes was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. The authors noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media per se, and that engagement with instructors is important (US Department of Education Office of Planning Evaluation and Policy Development. 2010).

Similarly, Mersham (2009) comments that to be effective, e-learning must put into practice an e-pedagogy that relates to social interaction and collaboration. However, communication opportunities in e-learning are under-used and ineffective when they are grafted onto courses that are rooted in pedagogic models and practices with which they are incompatible.

System (in)flexibility Concerns about flexibility and the implications of a lack of flexibility are key critical considerations for LMS. A LMS that is relatively inflexible may cause institutional processes to be modified to align with the software, rather than the software being sufficiently flexible to meet the learning needs of the institution (Adams n.d.). The implications of the integrated or enterprise resource planning product model resonate across all LMS models. Jones (2009) comments that “whether or not your LMS is open source or commercial doesn’t change the underlying model. All LMS are based on the enterprise, or ‘one ring to rule them all’ approach.”

Jones (2011) also highlights the apparent ‘cognitive dissonance’ between the nature of LMS technology as a core part of e-learning, and the pedagogical underpinnings of good teaching and learning. He states the dissonance is enhanced by the lack of flexibility in LMS and other e-learning tools, policies and procedures.

Dron (2006) comments the systemic features of an LMS may lead to the administrative domination of teachers and learners, and not serve the best interests of the learner. To address this, the most promising technological models appear to be those that allow an LMS to be developed from an assortment of ‘smaller pieces’, rather than extending modules in within a constraining framework of a larger host system (Dron 2006; Adams n.d.). The model of ‘small pieces, loosely joined’ is congruent with a personal learning environment (see later section).

Inclusion of Web 2.0 technologies Web 2.0 tools give users the choice to interact and collaborate with each other in a social media dialogue as creators of user-generated content in a virtual community, in contrast to websites that limited users to the passive viewing of content (Diaz, Golas et al. 2010). Examples of Web 2.0 technologies include social networking sites, blogs, wikis, video-sharing sites, hosted services, web applications, and tags.


Page | 25

Web 2.0 tools and socialisation have resulted in new ways of information organisation, knowledge generation and learning facilitation, and have resulted in fundamental shifts in the relationship among knowledge, culture, learning and pedagogy (Conole and Alveizou 2010). First, the modes of inquiry encouraged by Web 2.0 practices invite the learner to conceptualise knowledge as something available to be personalised or re-appropriated. Second, Web 2.0 encourages engagement with knowledge in new ways, e.g., more animated browsing or scanning. Third, practices of knowledge production are being altered, and learners are being drawn into more collaborative inquiry methods. Overall, the application of Web 2.0 technologies support evolution and facilitation of more informal and non-formal learning contexts which blur the boundaries between categories of learners, as well as transformation of formal education contexts.

Integration of LMS with Web 2.0 technologies is a challenge and concern for traditional LMS models, and was recognised as a ‘Top 10’ IT issue in 2009:

Over the years, the LMS has evolved from a content (course) management system (CMS) to a more all-encompassing system that includes groupware and social networking tools, as well as assessment and e-portfolios to track learning across courses and semesters. Although the LMS needs to continue serving as an enterprise CMS, it also needs to be a student-centered application that gives students greater control over content and learning. Hence, there is continual pressure for the LMS to utilize and integrate with many of the Web 2.0 tools that students already use freely on the Internet and that they expect to find in this kind of system. (Agee, Yang et al. 2009).

However, how to effectively achieve this poses a significant dilemma for LMS design and adaptability. Craig (2007) comments “if there is a single fundamental question that Web 2.0 poses for the traditional LMS in the academic community, it is the appropriateness of using enterprise software to serve as a top-down distribution vehicle for academic resources while simultaneously providing the digital space to foster innovation and collaboration from below” (p.157).

Beer and Jones (2008) suggest that the Web 1.0 philosophies that are embedded within LMS create a range of inhibiters to lifelong learning and to the establishment of a learning network which is more about a conversation (Web 2.0) than content delivery (Web 1.0). These include:

• content focus: Most LMS effectively provide learner-content interaction, and progress to extend tools and offerings beyond simple content sequencing and discussion forms is limited to a ‘locked down, do-it-our-way’ platform

• organisation and instructor focus: LMS use within tertiary education has impacted more on administration rather than the fundamentals of learning and teaching

• IT culture: the technology of LMS is old and consequently the technology provided by institutions has been outstripped by the functionality and usability of technology available to individuals

• Informal learning: the parameters and boundaries of LMS limit opportunities for informal learning, where most valuable learning takes place, and is a mismatch with the complexity of learning in modern life

• Course based models: an LMS is designed to provide tools for an instructor to deliver a single course, for a single term, rather than be part of a flexible community of practice (Beer and Jones 2008).

As part of the wider debate around pedagogical issues and LMSs, Wenmoth (2011) poses the question “is a LMS (or virtual learning environment) still the best way to go?” This question is being considered by various authors and commentators, who are critically considering if LMSs are still appropriate for learners who are increasingly building online lives (e.g., (Oneto, Abel et al. 2009). Reflecting on the need for LMS to meet the needs of


Page | 26

learners, Wenmoth has identified key shifts in pedagogical scope across a continuum of issues that need to be considered in order for tertiary institutions to use LMS to effectively engage with students online, as summarised below. Figure 4: Expanding Pedagogical Scope from Wenmoth 2011

Managing complexity of learning Emergent Established

View of knowledge “knowledge as a verb”

Network-centric

Folksonomies

“Knowledge as a noun”

Subject divisions/hierarchical

Taxonomies

Main ideas of knowledge and learning

Adaptive, dynamic, connected Structured, controlled, managed

Learning intentions Contextualised, nebulous Clear objectives, outcomes

Expressed as Learning ‘ecologies’ (social, connected, distributed)

Course model

Dominant technology used Personal learning environment Learning management system

Dominant theories of learning Connectivism

Social constructivism

Constructionism

Constructivism

Behaviourism

Cognitivism

Pedagogical focus Learner-centric

Learner choice/management

Activity based, experiential

Focus on participation/collaboration

Institutional focus

Teacher managed

Organisation of classes, courses

Focus on coverage, content delivery

Communications model Many to many

Networked

One to many

Transmissive

Examples of technologies ELGG Peer to peer options eg

Del-icio-us Colloquia

MySpace Groove

Bebo

Hosted options:

Moodle, Interact Blackboard

LAMS WebCT

Similarly, Sclater (2008) comments that tertiary institutions are considering whether or not an LMS remains an appropriate medium to facilitate e-learning and are asking critical questions such as:

• Can we bring some of the social networking facilities that students find so appealing inside the institution?

• Should we use the tools hosted elsewhere on the internet by others? • Should we simply allow learners to select appropriate tools for themselves?

Some educators argue that the next requirement is a Personal Learning Environment (PLE) where students interact with and collect web resources that interoperates with an LMS. This is highlighted as a top 10 IT issue for 2010, when Ingerman and colleagues (2010) comment:

It is also important to note the growth in various external modular applications that complement the LMS, enabling students to create a more personalized learning environment. These new applications have the potential to become the building blocks to a dynamic open learning environment that moves beyond the course-centric organization and stays with students throughout their academic career.


Page | 27

Siemens (October 6, 2006) notes that PLEs are an attempt to address the limitations of an LMS, as instead of being a centralised model of design and deployment, individual departments are able to select from a collage of tools to address a particular function on the learning process, or a ‘small pieces, loosely joined’ approach. A PLE is seen to provide a more contextually appropriate toolkit, and increased adaptability to different learning approaches and environments, however there is a consequential reduction of structure in the management and implementation of learning.

The key differences between PLEs and LMS are summarised by Chatti (2010) as follows: Figure 5: Key Characteristics of PLE and LMS from Chatti 2010

Siemens (October 6, 2006) comments that “while desired, it is unrealistic to expect universities to shift significantly from an LMS to a PLE. However, online trends such as social software and Web 2.0 technologies are impacting on learner expectations. As a result, LMS designers are responding to critiques about the lack of flexibility of LMS with a shift to more open, dynamic learning models, and are adding ‘Web 2.0 features’ to their products (e.g., integration with Facebook, YouTube and other applications). However, Mott (2010) argues that the administrative structure and pedagogy of LMSs continues to impede significant teaching and learning innovations because:

• LMS are generally organised around discrete, arbitrary units of time (e.g., academic semesters) and courses typically expire and disappear at the end of semesters, thereby interrupting the continuity and flow of the learning process

• LMS are teacher-centric. Teachers create courses, upload content, initiate threaded discussions, and form groups. Opportunities for student-initiated learning activities in traditional LMS are severely limited

• Courses developed and delivered via the LMS are ‘walled gardens’, limited to those officially enrolled in them. This limitation impairs sharing content across courses, conversations between students within and across degree programmes, and all the dynamic learning affordances of the read-write web.

Wenmoth (2007) comments that some institutions have attempted to meet the needs of students and developed partial PLEs but he considers there are three key challenges remaining for education institutions to transition to a student-centric system:


Page | 28

• The locus of control and ownership of tools and features still rests with the institution not the learner

• The nature of social networking software is that it allows for the development of wide and inclusive networks. By locking software within the boundaries of an LMS the opportunity for networking is restricted to that community

• Increasing numbers of learners have independent access to Web 2.0 tools and features and will not want to be restricted to an exclusive community or system.

Mott (2010) proposes a model of an Open Learning Network (OLN) as a means to enable the LMS and PLE paradigms to co-exist and to mashup the best components of each into something that is more than just the sum of its parts. This is proposed as a balance between the either-or choice between the LMS and the PLE. The OLN has three key features:

• It is malleable • It leverages technologies that did not exist when the LMS was created in the 1990s • It is a manageable balance between the imperatives of institutional networks and the

promises of the cloud.

Mott (2010) states that the “OLN’s full potential is realised by implementing supporting technologies such as iCal for calendar aggregation, RSS for content aggregation, widgets and embed codes for republishing of live content, tags to support folksonomy-driven content discovery and compilations, and granular authorising codes to support content authoring and commentary.... The OLN can incorporate a virtually limitless number of web-based applications and third-party learning tools via web services, published application programming interfaces, and the LTI framework.... as student computing becomes increasingly mobile, OLN modules and data can be mashed up and dynamically re-presented on various devices.” Brigham Young University is in the first stages of developing and deploying an OLN. Figure 6: A fully featured OLN from Mott 2010


Page | 29

Other authors have suggested similar solutions, such as the MOOC or Massive Open Online Course, which provides participants with an opportunity to create their own PLEs and personal learning environments (Shimabukoro 2011). The MOOC is a dynamic structure which is defined by the expansion of the personal learning environments and networks. The mixed or non-LMS approach is considered more sustainable than the ‘pure’ LMS approach. The Social Learning Environment is an alternative model which focuses on a combination of PLEs and personal teaching environments supported by modular architecture (Crosslin 2010).

Future considerations for LMS

The end of the age of the LMS dinosaur may be in sight (Dron 2006).

E-learning 2.0 is a dominant theme in discussions of the future of e-learning environments (Wise and Quealy 2006). E-learning 2.0 has emerged from the development of Web 2.0, and has the following key characteristics:

• Learner-centric rather than content or teacher-centric • Promotes greater autonomy for learners, with a focus on active learning, and

learners as knowledge consumers and knowledge creators • The distinctions between teacher and student are collapsed, and power is

decentralised • Learning is controlled by the learner • Learners are encouraged to interact with others in the learning process using a range

of technologies • Open communication, freedom for sharing, social networked learning and socially

constructed knowledge is emphasised (Yau, Lam et al. 2009).

There is increasing recognition that LMS development, design and implementation needs to respond to emerging e-learning 2.0 issues. Delta Initiative describes the LMS market as being “in the midst of change”, and comments that the transition to LMS 3.0 is being driven by a combination of the following factors:

• The market would like to focus on more interactivity/knowledge creation, however these features are currently exploratory

• LMS vendors would like to be the sole platform on which all tools fit • Content portability is not solved • The open source movement is growing, but is unlikely to be a panacea to all the

issues • Similarly, the managed and hosted services are growing but is also unlikely to be a

panacea • The market is unlikely to change unless customer behaviour changes (Delta

Initiative. 2010).

Delta Initiative state that these are not ‘typical’ market dynamics, and consortiums and open source solutions are providing the most change currently (Delta Initiative. 2010). Overall, LMS market changes have been driven by vendor business concerns, rather than institutional or educational needs. This has resulted in raised concerns about the impact on institutions, which in turn has supported renewed discussions about open source solutions, and a rethink of systems and how to organise services.


Page | 30

Various commentators have identified key issues for the future consideration of LMS including the following priority issues:

1. Emergence of new alternative concepts to LMS

New models of LMS are emerging with the specific aim of disrupting the existing market, and are building on the increasing interest in alternatives to LMS where traditional LMS do not meet teaching or learning needs. These alternative models are characterised by their use of Web 2.0 applications, and integration with online media and social networking tools (Educause. 2010). Two examples of ‘disruptive’ alternative concepts to traditional LMS are Instructure and SmartlyEdu.

Instructure states “our goal is to disrupt the Learning Management System (LMS) market by setting a new standard for education technology. We’re all about openness, usability, communication, and reliability” (Instructure. 2011). Instructure challenges the status quo for LMS by providing both open source and propriety options, as this is seen to provide the most effective balance between “the value of openness and the requirement of mission-critical solution based development and support” (Instructure. 2011). Feldstein (2011) comments that Instructure is characterised by good design, ease of use, and the ability to work with Web 2.0 applications, and to integrate third-party applications. It is a different model of open source development to other open source LMS such as Moodle or Sakai, as it is more of a traditional company-run open source project, and is unlikely to cede development to an open source community.

SmartlyEdu has been developed by a group of ‘digital natives’ – “three students from around the world believe they have a solution and are working to bring this to teachers and students. Software can't be built by people who aren't going to use it; that's why mega-corps run by old men won't bring the people what they need” (Smartly Edu. 2010). It focuses meeting the communication needs of learners and teachers, and is characterised by a belief that “great design will make students spend more time online, connected, and engaged” (Meyer 2010). Delta Initiative (2010) has conceptualised the potential impact of emergent LMS models on the overall LMS market, as illustrated below:


Page | 31

Figure 5: Potential and emerging market disruptions from Delta Initiative, 2010

2. Social media and social learning

Social media is, by definition and practice, social, and social media tools are designed to facilitate active discussion, collaboration and sharing, and also to empower individuals with opportunities to engage and participate. Social media is also translocational, (it operates independent of place), and can occur in real time (synchronous) or be time agnostic (asynchronous). As learners interact through various media, a sense of connectivity, networks and community is developed.

However, various authors warn that adding ‘social tools’ to an LMS does not make it social, because as noted earlier, traditional LMS are ‘walled gardens’ and more is required to expand the boundaries about how we learn, what and with whom. For example, Orlando outlines a range of ways to incorporate social media into online education, and suggests that an LMS should be considered just one tool to deliver online learning (Orlando 2010).

Integration with formal, informal and social media and social learning is a priority for the development of future LMS (Expertus. and TrainingIndustry Inc. 2010). Chemielowski (2009 ) describes a range of broad-stroke benefits of social media to support enterprise learning, including learning and performance opportunities such as:

• The ability to enable ‘mass personalisation’ or the ability to support individuals in finding (and/or co-creating) just the right information at just the right time

• Enable access to experts – the ability to find and connect with the person who can help in a moment in need

• Enable freshness of content • Enable rapid knowledge capture


Page | 32

• Reduce operating expenses in terms of content development by shifting the model to leverage user-generated content and collaboration

• Increase volume of content available for the same or less investment due to exponentially more product producers.

Other authors suggest there is increasing need to transition to a focus on collaboration platforms that are transparent and reward sharing, to facilitate learning. Jarche (2010) suggests this will result in minimising of the use of an LMS in favour of other tools like blogs, wikis and presence tools (instant messaging and microblogging). Similarly, others suggest the need for a LMS to become federated or part of an enterprise-wide collaboration platform. For example, Pontefract (2009) states “Coaches, mentors, online buddies need to coexist within the wiki’s, blogs, discussion forums, webcam meetings, online presence, etc. which needs to coexist within the list of formal classroom and eLearning offerings which needs to coexist with your documents, knowledge management, videos, podcasts, which needs to coexist with the profiles, skills, and recent activity-feed happenings of all employees. Blow up your LMS. Find a way to integrate it into your collaboration platform.”

3. Mobile learning

Mobile devices are not well suited to traditional courseware delivery, but rather for information delivery and performance support (Rosenberg 2010). The portability of mobile devices and their ability to connect to the internet almost anywhere makes them ideal as a tool for storage of learning experiences and reference materials, as well as a general-use tool for field work (Johnson, Levine et al. 2010). They are ideal tools for ubiquitous learning – learning that develops in response to personal learning demands and that can happen everywhere and at anytime.

The ability to be always connected is supporting the development of another transformation of mobile learning – the use of organization based social networking as a mobile learning strategy. The future of mobile learning, supported by more powerful and flexible platforms and devices and anytime access to content, is facilitating the emergence of new communication channels, which are opening up opportunities to communicate and connect with other learners and experts, anywhere, time (Rosenberg 2010). Responding to the rise of mobile computing, all major LMS vendors are demonstrating or marketing mobile access (Delta Initiative. 2010).

The Delta Initiative (2010) states that there are two competing visions of the response to mobile computing. One response focuses on engagement with mobile devices and applications. The benefits of this approach include access to engaging design, free use of HW features, fast and lightweight technology, however the challenges are that the applications are device specific, which requires ‘forced bets’ on preferred devices, and high costs. In contrast, the second approach focuses on interaction with a browser, so the technology is ubiquitous, mature, and device independent. However, it may also be slower and more cumbersome to use than applications, misses out on some HW features, and is harder to access for some smart phones (versus the Ipad). Blackboard are focusing on native applications, and Moodle and other open source providers and focusing on m-site or browser technology.


Page | 33

Figure 6: Competing visions of mobile LMS from Delta Initiative, 2010

Mobile learning combined with cloud computing makes virtually limitless amounts of content instantly available to limitless numbers of users. However, Rosenberg cautions that m-learning content, either instructional or informational (e.g., a website, podcast, blog or wiki) must be the right information, presented at the right level of detail, targeted for the right user, and delivered at the right moment of need. Further attention and development is required to match users with content through profiles, preferences and tagging.

The Horizon Report 2011 identifies electronic books and mobiles as priority technology which will have a wide impact on learning and teaching within the next 12 months (Johnson, Smith et al. 2011). Johnson and colleagues (2011) comment that mobiles embody the convergence of several technologies that lend themselves to educational use, and allow very simple tools to be easily integrated into learning activities with no need for involvement of IT or other support staff. Mobiles are increasingly recognised as advantageous tools for learning, and their ubiquity, portability, and their power and impact is supported by the wide range of things that can be done with them, and their ability to access the internet anywhere. However, keeping pace with the rapid proliferation of information, software tools and devices is a key challenge for learners and teachers (Johnson, Smith et al. 2011).

4. Web 2.0 and Cloud technology integration

A recent analysis of the perceptions of current LMS and critical future functionalities among corporate and training professionals in the US reported an important area for LMS growth and development was the ability to incorporate user-generated content, and Web 2.0 usability (Expertus. and TrainingIndustry Inc. 2010).


Page | 34

Web 2.0 technologies and practices provide new mechanisms for inquiry-based and exploratory learning (Conole and Alveizou 2010). New tools are emerging for interrogating and analysing data, along with rich social and information environments to support research communities. Web 2.0 also challenges learners and teachers and blurs the boundaries of control. Emergent Web 2.0 technologies support new forms of learning including:

• inquiry-based and exploratory learning; • new forms of communication and collaboration • new forms of creativity, co-creation and production • richer contextualisation of learning (Conole and Alveizou 2010).

Integration with cloud technologies is an increasingly important trend, as the technologies we use are increasingly cloud-based, and our notions of IT support are decentralised (Johnson, Smith et al. 2011). The continuing acceptance and adoption of cloud-based applications and services is changing the software and file storage is configured and used, and also how these functions are conceptualised. Overall, Web 2.0 or cloud-based technologies further support a trend towards control of the instructional functions by the individual user (teacher and learner). These emerging technologies support and require individual creativity and autonomy, and foster the growing trend towards user-generated content and knowledge in the way that many institutionally developed products do not (Diaz, Golas et al. 2010). Aligning with an increased focus on social learning and social media, they also have the potential to promote sharing, openness, transparency, and collective knowledge construction. However, as user created content is ever increasing, giving rise to new information and ideas, there is a parallel need to create effective tools and filters for finding, interpreting, organizing and retrieving important data (Johnson, Smith et al. 2011).

Diaz and colleagues state “faculty members no longer need to wait for an LMS to develop and implement a tool, for an institution to purchase a license to use images, or for a streaming media service because many of these needs can now be met externally through a variety of cloud based tools” (Diaz, Golas et al. 2010:2).

Emerging cloud solutions have the potential to empower individual faculty with diversity of choice and the ability to independently fashion LMS tools in innovative ways. For example, Culatta (2011) applies the concepts of modularization and customization to create a ‘mashed up’ or ‘modularised’ LMS that would have the potential to be infinitely customisable as well as being able to take advantage of current ‘best of breed’ tools. In this model, an LMS becomes a ‘loosely coupled system’, where the component parts retain their own specialised functionality but work together to deliver a tailored learning experience. Culatta (2011) comments that this model has a number of advantages, including unique customisation for specific settings, selection of the best systems to support desired functionalities, and the ability to ‘hot swap’ components as requirements change.

5. Increased use of learning analytics

There is some recognition of the importance of learning analytics to support reflective practice, however the use of learning analytics is still its early stages (see for example (Beer, Jones et al. 2009; University of Wollongong. 2009). An increased focus on learning analytics in the next four to five years will harness the power of advances in data mining, interpretation and modelling to improve understandings of teaching and learning, and to more effectively tailor education to individual students (Johnson, Smith et al. 2011).

An increased focus on learning analytics not only has implications for student performance, but also for how educators perceive teaching, learning and assessment. Real-time analysis of learning analytics can support immediate alternations, and support a model of curriculum that is fluid and open to change (Johnson, Smith et al. 2011).


Page | 35

Learning analytics data can also be used with the semantic web (Web 3.0), with significant impacts for learners including the ability to access relevant up to date information more quickly, enable personalised tailored content to fit current learning requirements, and facilitate close contact with those who provide the information, while sharing information with those with similar interests (De Waard 2011).

Concluding comments This aim of this paper was to provide information to the AUT University LMS Review Committee, as one part of the LMS Review project. Specifically, the goals of this literature review were to identify what issues other universities within New Zealand and Australia have used to inform their processes of LMS change and transition, as well as to identify other significant issues that are pertinent to tertiary sector considerations of LMS change and transition. The review has assessed a broad array of information, including peer review journal articles, reports, conference presentations, trade journal articles, and e-learning and LMS commentator blogs. Collectively, this information identifies a number of issues for AUT University to consider as the LMS review is progressed, including:

• It is difficult to access complete information about LMS business cases and transition/change decision making processes across universities in New Zealand and Australia, however the available information suggests there is a growing trend to implement Moodle.

• The LMS market is in a stage of rapid change, and there is likely to be significant change over the next three to four years, with predictions of new LMS software that will displace existing LMSs.

• Increasingly, there is a growing trend to critique existing LMSs for their pedagogical limitations, and this criticism is being applied to propriety and open source software. In order to maximise the potential of a new or different LMS implemented in a university setting, significant pedagogical support is likely to be required. Supporting educators to become confident and competent with all the capabilities of an LMS will have positive outcomes for learners: the effective use of LMS by engaged teachers supports engaged learners, and engaged learners have better learning outcomes.

• Web 2.0 tools support opportunities for learner interactions and collaboration, as well as supporting the personal conceptualisation of knowledge, engagement with knowledge in new ways. Web 2.0 technologies support the transformation of formal education contexts, however at the same time they provide a significant challenge for existing models of LMSs. Commentators suggest there is limited value in attaching Web 2.0 tools to current models of LMS, as their pedagogical and architectural structures are inherently Web 1.0 and therefore incompatible with the principles and practices of Web 2.0. There is increasing interest in LMS alternatives such as personal learning environments or open learning networks which operate in a mashed up, ‘small pieces loosely joined’ approach, in order to provide increased adaptability to different learning approaches and environments.

• Careful consideration is required about how to respond to key strategic drivers that are influencing the transition to LMS 3.0, including new models of LMSs, increasing use of Web 2.0 technologies, mobile learning, cloud technologies, and the increasing use of LMS analytics, particularly in the period of further rapid change in the LMS market and LMS technologies that is predicted to occur in the very near future.


Page | 36

References

Adams, C. (2011). "Learning Management Systems as Sites of Surveillance, Control and Corporatization: A review of the critical literature " Society for Information Technology and Teacher Educatin INternational Conference 2010(1): 252-257.

Adams, C. (n.d.). Learning Managemet Systems Review, University of Alabama. Agee, A., C. Yang, et al. (2009). "Top 10 IT issues 2009." EDUCAUSE Review 44(4): 44-59. AKO National Centre for Tertiary Teaching Excellence. (2008). Taking the Lead: Strategic

Management for e-Learning. Wellington, AKO Aotearoa. Bacus, M. (2010). University of Western Australia: Summary Report on the Environmental

Scan of LMS in the Higher Education Sector. Perth, University of Western Australia. Barr, H., B. Gower, et al. (2007). "Faculty response to the implementation of an open source

learning management system in three tertiary institutions in New Zealand." Computers in the Schools 24(3/4): 125-137.

Beer, C., K. Clark, et al. (2010). Indicators of engagement Ascilite 2010. Sydney. Beer, C. and D. Jones. (2008). "Learning networks: Harnessing the power of online

communities for discipline and lifelong learning." Retrieved March, 2011, from http://davidtjones.wordpress.com/publications/learning-networks-harnessing.html.

Beer, C., D. Jones, et al. (2009). The indicators project identifying effective learning: Adoption, activity, grades and external factors. Ascilite 2009: Same places, different spaces. Auckland.

Brown, M., A. Arnold, et al. (2009). Thinking about a new LMS: Comparing different institutional models and approaches. Ascilite 2009: Same places, different spaces, Auckland.

Brown, M., S. Paewai, et al. (2010). "The VLE as a Trojan Mouse: Policy, Politics and Pragmatism." Electronic Journal of e-Learning 8(2): 63-72.

Brown, M. and G. Suddaby (2009). Policy, Politics and Pragmatism: Making the case for a new LMS. ICDE 23rd World Conference. Netherlands.

Butterfield, S. (2002). Highways and pathways: Exploring New Zealand's e-learning opportunities. Report of the E-Learning Advisory Group. Wellington, Ministry of Education.

Catalyst IT Ltd. (2004). Technical Evaluation of Selected Learning Management Systems, The Open Polytechnic of New Zealand.

Chang, C. (2008). Faculty perceptions and utilization of a learning management system in higher education, Ohio University.

Chatti, M. (2010). "LMS vs PLE." Retrieved March, 2011, from http://mohamedaminechatti.blogspot.com/2010/03/lms-vs-ple.html.

Chmielowski, B. (2009). "Value of social media to enterprise learning." Retrieved March, 2011, from http://velocitymg.com/explorations/leveraging-learning/value-of-social-media-to-enterprise-learning.html.

Clark, K. (n.d.). Academic Indicators: data mining as reflective practice, CQUniversity. Clark, K., C. Beer, et al. (2010). Academic involvement with the LMS: An exploratory study.

Ascilite 2010. Sydney. Clayton, J. and R. Elliott (2007). E-Learning Activities in Aotearoa New Zealand Industry

Training Organisations. Hamilton, Wintec, Ministry of Education, ESITO. Coates, H., R. James, et al. (2005). "A critical examinaiton of the effects of Learning

Management Systems on University teaching and learning." Tertiary Education and Management 11(1): 19-36.

Conole, G. and P. Alveizou (2010). A literature review of the use of Web 2.0 tools in higher education. Milton Keynes, UK, The Open University.


Page | 37

Coopman, S. (2009). "A critical examination of Blackboard's e-learning environment." First Monday 14(6): 1-6.

Craig, E. (2007). "Changing paradigms: managed learning environments and Web 2.0." Campus - Wide Information Systems 24(3): 152-161.

Crosslin, M. (2010). "Social Learning Environment Manifesto." Retrieved March, 2011, from http://www.edugeekjournal.com/2010/03/18/social-learning-environment-manifesto/.

Culatta, R. (2011). "The traditional LMS is dead: Looking to a modularised future." Retrieved March, 2011, from http://www.innovativelearning.com/learning_management/modular-lms.html.

Curriculum Teaching and Learning Centre. (2010). "A Recent LMS Evaluation." Retrieved March, 2011, from http://www.slideshare.net/vdit/vdit-la-trobe-4799832.

Dagger, D., A. O'Connor, et al. (2007). "Service-Oriented E-Learning Platforms: From Monolithic Systems to Flexible Services." IEEE Internet Computing 11(3): 28.

Dawson, S. and McWilliam (2008). Investigating the application of IT generated data as an indicator of learning and teaching performance. A Report produced for the Australian Learning and Teaching Council. Brisbane, Queensland University of Technology.

De Waard, I. (2011). "The semantic web and how the intelligent curriculum will enhance our learning." Retrieved March, 2011, from http://ignatiawebs.blogspot.com/2011/01/lak11-semantic-web-and-how-inteligent.html.

Delta Initiative. (2010). New Choices: The State of Learning Mangement System Market in Higher Education. Illonois, Delta Initiative.

Diaz, V., J. Golas, et al. (2010). "Privacy considerations in Cloud-based teaching and learning environments." EDUCAUSE ELI Paper 3.

Dron, J. (2006). Any color you like as long as it's Blackboard. ELearn Conference. Hawaii. Eckstein, G. (2010). Guide to Learning Management Systems Gary Eckstein Wordpress and

SEO specialist. 2011. Educause. (2010). "7 things you should know about LMS alternatives." EDUCAUSE

Quarterly. Elgort, I. (2005). E-learning adoption: Bridging the chasm. ASCLITE 2005: Balance, Fidelity,

Mobility: maintaining the momentum? Brisbane, Australia. Ellis, R. (2009). A Field Guide to Learning Management Systems. Learning Circuits. Expertus. and TrainingIndustry Inc. (2010). Survey results: The current and future state of

LMS. California, Expertus. Feldstein, M. (2010). "Academic Study of Blackboard vs Sakai at UNC School of Medicine."

Retrieved March, 2011, from http://mfeldstein.com/academic-study-of-blackboard-vs-sakai-at-unc-school-of-medicine/.

Feldstein, M. (2010). "The evolving LMS market, part ii." Retrieved March, 2011, from http://mfeldstein.com/the-evolving-lms-market-part-ii/.

Feldstein, M. (2011). "Instructure goes open source." Retrieved March, 2011, from http://mfeldstein.com/instructure-goes-open-source/.

Hill, J. (2008). VLE Evaluation: Project Report. Gloucestershire, University of Gloucestershire.

Ingerman, B., C. Yang, et al. (2010). "Top-10 IT Issues 2010." EDUCAUSE Review 45(3): 45-60.

Instructure. (2011). "Our Open Source Strategy." Retrieved March, 2011, from http://www.instructure.com/blog/2011/02/28/our-open-source-strategy/.

Jarche, H. (2010). "Identifying a collaboration platform." Retrieved March, 2011, from http://www.jarche.com/2010/05/identifying-a-collaboration-platform/.

Johnson, E. and R. Walker (2007). The Promise and Practice of E-Learning within Complex Tertiary Environments. 7th IEEE International Conference on Advanced Learning Technologies. Niigata Japan.

Johnson, L., A. Levine, et al. (2008). The Horizon Report: 2008 Australia-New Zealand Edition. Austin, Texas, The New Media Consortium.


Page | 38

Johnson, L., A. Levine, et al. (2010). The 2010 Horizon Report. Austin, Texas, The New Media Consortium.

Johnson, L., R. Smith, et al. (2011). The 2011 Horizon Report. Austin, Texas, The New Media Consortium.

Jones, D. (2009). "One ring to rule them all": Limitaitons and implicaitons of the LMS/VLE product model. The weblog of (a) David Jones. 2010.

Jones, D. (2009). Thoughts on "Insidious pedagogy". The weblog of (a) David Jones. 2010. Jones, D. (2011). "The dissonance between the constructivist paragidm and the

implementation of institutional e-learning." Retrieved March, 2011, from http://davidtjones.wordpress.com/2011/03/02/the-dissonance-between-the-constructivist-paradigm-and-institutional-e-learning/.

La Trobe University. (2010). LMS Replacement Project. Moodle Implementation - Information Sheet for Academic Staff. Melbourne, La Trobe University.

Lane, L. (2009). "Insidious pedagogy: How course management systems impact teaching." First Monday 14(10).

MacNeill, S. and W. Kraan (2010). Distributed Learning Environments: A briefing paper. London, JISC CETIS.

Macquarie University. (2011). "Macquarie University Learning and Teaching Centre: Learn2012 Project (Moodle Implementation)." Retrieved March 2011, from http://www.mq.edu.au/ltc/projects/learn2012/index.htm.

Marshall, S. (2005). Determination of New Zealand tertiary institution e-learning capability: An application of an e-learning maturity model. Report on the e-learning maturity model evaluation of the New Zealand tertiary sector. Report to the New Zealand Ministry of Education. Wellington, Victoria University of Wellington.

Mersham, G. (2009). "Reflections on e-learning from a communictions perspective." Journal of Distance Learning 13(1): 51-70.

Meyer, A. (2010). "What Blackboard and Moodle get very wrong." Retrieved March, 2011, from http://smartlyedu.com/blog/posts/14-what-blackboard-and-moodle-get-very-wrong.

Ministry of Education. (2006). ICT Strategic Framework for Action 2006-2007. Wellington Ministry of Education.

Ministry of Education. (2008). Briefing to the Incoming Minister. Wellington, Ministry of Education.

Mitchell, D., J. Clayton, et al. (2005). E-Learning In New Zealand Institutes of Techology/Polytechnics: Final Report. Hamilton, Wintec Waikato Institute of Technology.

Mitchell, G. and R. Obexer (2006). LMS Implementation Project: Change Management Plan. Brisbane, Queensland University of Technology.

Momani, A. M. (2010). Comparison between two learning management systems: Moodle and Blackboard.

Moodle. (2011). "About Moodle." Retrieved March, 2011, from http://docs.moodle.org/en/About_Moodle.

Mott, J. (2010). "Envisioning the Post-LMS Era: The Open Learning Network." EDUCAUSE Quarterly 33(1).

Naveh, G., D. Tubin, et al. (2010). "Student LMS use and satisfaction in academic institutions: The organizational perspective." Internet and Higher Education 13(3): 127-133.

NZ E-Learning Group. (2006). e-Learning Guidelines: Guidelines for the support of e-learning in New Zealand tertiary institutions. Palmerston North, Massey University.

Obexer, R. (2006). OLT System Review Recommendation. Brisbane, QUT Teaching and Learning Support Services.

Obexer, R. (2006). OLT System Review Recommendation: Learning Environments Working Party Feedback. Brisbane, QUT Teaching and Learning Support Services.


Page | 39

Obexer, R. and A. Bakharia (2005). OLT System Review: Environmental Scan Report. Brisbane, Queensland University of Technology.

Oneto, L., F. Abel, et al. (2009). Making today’s Learning Management Systems adaptive. Orlando, J. (2010). "Integrating social media into online education." Retrieved March 2011,

from http://www.facultyfocus.com/articles/online-education/integrating-social-media-into-online-education/.

Otto, R. (2011). Feature Comparison Moodle-Blackboard. Bochum, Germany, Ruhr-Universitat.

Papastergiou, M. (2006). "Course Management Sytems as tools for the creation of online learning environments: Evaluation from a social constructivist perspective and implications for design." international Journal on E-Learning 5(4): 593-622.

Paulson, M. (2002). "Online Educaiton Systems: Discussion and definition of terms." from http://www.nettskolen.com/forskning/Definition%20of%20%terms.pdf.

Pina, A. (2010). An Overview of Learning Management Systems. Learning Management System Technologies and Software Solutions for On-line Teaching: Tools and Applications. Y. Kats. Pennsylvania, IGI Global.

Pontefract, D. (2009). "The Standalone LMS Is Dead." Retrieved March, 2011, from http://trainingwreck.com/.

Queensland University of Technology. (2008). "Review of Online Learning and Teaching." Retrieved March, 2011, from http://www.tils.qut.edu.au/initiatives/learningmana/OLTReview.jsp.

RMIT. (2011). "Teaching and Learning: Blackboard Upgrade." Retrieved March, 2011, from http://www.rmit.edu.au/browse/Staff%2FLearning%20and%20Teaching%2FSupport%20and%20resources%2FTeaching%20technologies%2FBlackboard%20upgrade/.

Rosenberg, M. (2010). "Marc my words: thinking about mobile learning in the age of the ipad." Retrieved March, 2011, from http://www.learningsolutionsmag.com/articles/477/marc-my-words-thinking-about-mobile-learning-in-the-age-of-ipad.

Sclater, N. (2008). "Large-Scale Open Source E-Learning Systems at the Open University UK." EducAUSE Research Bulletin 12(10): 1-13.

Sclater, N. (2008). "Web 2.0, Personal Learning Environments, and the future of LMS." EducAUSE Research Bulletin 13: 1-13.

Shephard, K., S. Stein, et al. (2008). Professional Development for e-Learning: A framework for the New Zealand tertiary education sector. Part C: Massey Literature REview: Professional Development for e-learning: Adoption, implementation, and improvement. Wellington, Ministry of Education.

Shimabukoro, J. (2011). "Why LMSs aren't the answer." Retrieved March, 2011, from http://etcjournal.com/2011/02/22/why-lmss-arent-the-answer/.

Siemens, G. (October 6, 2006). "A review of learning management system reviews. Connectivism blog [Web log message]]." from http://www.connectivism.ca/?p=243.

Smartly Edu. (2010). "Smartly is a reinvention of what is now known as "online learning"." Retrieved March, 2011, from http://www.smartlyedu.com/newsletters/2010/08/14.html.

Smithers, M. (2009). "Public LMS Evaluations." Retrieved March, 2011, from http://www.masmithers.com/2009/09/20/public-lms-evaluations/.

Steel, C. (2005). Game for Change? Balancing an enterprise-level LMS implementation. ASCLITE 2005: Balance, Fidelity, Mobility: maintaining the momentum? Brisbane, Australia.

Suddaby, G. and J. Milne (2008). "Coordinated, collaborative and coherent: Developing and implementing e-learning guidelines wihtin a national tertiary education sytem." Campus - Wide Information Systems 25(2): 114-122.

Thibault, J. (2011). "Blackboard vs Moodle 2.0: 100+ metrics compared head to head by @MoodleRUB." Retrieved March, 2011, from


Page | 40

http://www.moodlenews.com/2011/blackboard-vs-moodle-2-0-100-metrics-compared-head-to-head-by-moodlerub/.

University of Canterbury. (2008). Learning Management Systems Review: Final Report and Recommendations. Christchurch, University of Canterbury.

University of South Australia. (2010). "Learnonline." Retrieved March, 2011, from http://www.unisa.edu.au/learnonline/.

University of South Australia. (2010). "UniSA's Guiding principles for Oneline learning." Retrieved March, 2011, from http://www.unisa.edu.au/learnonline/staff/Guiding%20Principles.asp.

University of Tasmania. (2010). ACODE LMS Update Hobart, University of Tasmania. University of Wollongong. (2009). "Learning Networks: About SNAPP." Retrieved March,

2011, from http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html. US Department of Education Office of Planning Evaluation and Policy Development. (2010).

Evaluation of Evidence-based practices in on-line learning: A meta-analysis and review of online learning studies. Washington, US Department of Education.

Uys, P. (2010). "Implementing an open source management system: A critical analysis of change strategies." Australian Journal of Educational Technology 26(7): 980-995.

Walsh, K. and O. Coleman. (2010). "Moodle 2.0 versus Blackboard 9.1 - a brief comparison." Retrieved March, 2011, from http://www.emergingedtech.com/2010/11/moodle-2-versus-blackboard-brief-comparison/.

Weller, M. (2007). Virtual Learning Environments: Using, choosing and developing your VLE. Oxford, Routledge.

Wenmoth, D. (2007). Understanding the OLE-PLE relationship, CORE Education. Wenmoth, D. (2009). "VLE's slow to take off." Retrieved March, 2011, from http://blog.core-

ed.org/derek/2009/01/vles-slow-to-take-off.html. Wenmoth, D. (2010). "Designing an online environment for learning." Retrieved March,

2011, from http://blog.core-ed.org/derek/2011/03/making-effective-use-of-vles.html. Wenmoth, D. (2011). "Making effective use of VLEs." Retrieved March, 2011, from

http://blog.core-ed.org/derek/2011/03/making-effective-use-of-vles.html. White, D. (2009). Jumping Ship Without Drowning: Managing Change of an Institutional

LMS. Hamilton, Waikato University. White, D. and N. Robertson (2009). Jumping Ship Without Drowning: Managing Change of

an Institutional LMS. Hamilton, Waikato University. White, P. and J. Milne (2005). Survey of support and guidance for e-learning in New Zealand

tertiary institutions. Palmerston North, Massey University. Winter, M. (2006). Learning Management Sytems for the Workplace: A research report.

Christchurch, Core Education Limited. Wise, L. and J. Quealy (2006). At the limits of social constructivism: Moving beyond LMS to

re-integrate scholarship. Ascilite 2006: Who's learning? Whose technology? Sydney. Wise, L. and J. Quealy (2006). LMS Governance Project Report. Melbourne, Melbourne-

Monash Collaboration in Institutional Technologies. Yau, J., J. Lam, et al. (2009). A Review of e-Learning Platforms in the Age of e-Learning 2.0.

Hybrid Learning and Education, Proceedings. F. L. Wang, J. Fong, L. Zhang and V. S. K. Lee. Berlin, Springer-Verlag Berlin. 5685: 208-217.


Page | 41

Appendix One

LMS Uptake of Australian Universities (Bacus 2010)

Australian Universities From Moving to In

Australian Catholic University [ACU] WebCT CE 8.0 + D2L e-portfolio

unknown status

Australian National University [ANU] WebCT CE 4.1 Moodle 1.9 2009 - 2010

Bond University [Bond] Blackboard 7.0 Blackboard

Central Queensland University [CQU] Blackboard 6.3* Moodle 1.9 2010

Charles Darwin University [CDU] Blackboard 6.3 Bb Learn 9.0

Charles Sturt University [CSU] Sakai 2.4 Sakai 3.0 2011

Curtin University of Technology [CURTIN] Blackboard Bb Learn 9.0

Deakin University [Deakin] WebCT Vista Desire2Learn

Edith Cowan University [ECU] Blackboard 8.5 Bb Learn 9.1 2010 - 2011

Flinders University [FLINDERS] WebCT Vista 4.2*

under review

Griffith University [GRIFFITH] Blackboard 8.0 Bb Learn 9.1 2011

James Cook University [JCU] Blackboard Bb Learn 9.0

La Trobe University [LATROBE] WebCT CE 6.0 Moodle 2.0 2011

Macquarie University [MACQUARIE] WebCT CE 6.0 under review

Monash University [MONASH] WebCT Vista 4.0 under review

Murdoch University [MURDOCH] WebCT CE 6.0 under review

Queensland University of Technology [QUT] Blackboard 7.2* Bb Learn 9.0 -

RMIT University [RMIT] Blackboard 7.2 Bb Learn 9.1 2011

Southern Cross University [SCU] Blackboard 7.1* Bb Learn 8.0

Swinburne University of Technology [SWINBURNE] Blackboard 8.0 Bb Learn 9.0 Mar 2010


Page | 42

University of Adelaide [ADELAIDE] Blackboard 8.0 Bb Learn 9.0 2010

University of Ballarat [BALLARAT] WebCT CE 8.0 Moodle 1.9 2011

University of Canberra [CANBERRA] WebCT CE 4.1* Moodle 1.9 2008

University of Melbourne [MELBOURNE] Blackboard 8.0 under review

University of New England [UNE] WebCT CE 6.0 Moodle 2.0 2011

University of New South Wales [UNSW] Bb Learn 9.0 Bb Learn 9.1 2010

University of Newcastle [NEWCASTLE] Blackboard Academic Suite

Blackboard

University of Notre Dame Australia - The [UNDA] Blackboard 7.0 Blackboard

University of Queensland [UQ] Blackboard 7.3 Bb Learn 9.0 -

University of South Australia [UniSA] Home grown system - UniSAnet*

Moodle 1.9

University of Southern Queensland [USQ] Moodle 1.9 Moodle 2.0 2011

University of Sydney [SYDNEY] WebCT CE 8.0 Bb Learn 9.1 2011

University of Tasmania [TASMANIA] WebCT Vista under review

University of Technology Sydney [UTS] Bb Learn under review

University of the Sunshine Coast [USC] Blackboard Academic Suite

Blackboard

University of Western Australia [UWA] WebCT CE 8.0 under review

University of Western Sydney [UWS] WebCT CE 8.0 Bb Learn 9.1 2011

University of Wollongong [UOW] WebCT Vista 8.0 under review

Victoria University [VU] WebCT CE 6.0 under review


Page | 43

Appendix Two: LMS in New Zealand University of Tasmania environmental scan (University of Tasmania. 2010), updated by AUT University

New Zealand

Blackboard

Version

BB – WebCT

Version

Moodle

Version

Other Change to BB9 - When

Change to Moodle-When

Considering Change

AUT University

Bb9 Under review

University of Auckland

Cecil Homegrown, One faculty with Moodle

No change being considered

University of Canterbury

1.9 Changes 2009

Massey University

1.9 No 2009 Done

University of Otago

8 ?

University of Waikato

1.9 No 2008 No

Vic Uni of Wellington

BB BB 9

Lincoln Moodle 1.9

Date post:	29-Nov-2014
Category:	Documents
Upload:	rui-guimaraes-lima
View:	185 times
Download:	1 times

Review of Lms Literature for Aut Lms Review Committee

Documents