+ All Categories
Home > Documents > Online Report

Online Report

Date post: 06-Apr-2018
Category:
Upload: rachit-goel
View: 227 times
Download: 0 times
Share this document with a friend

of 177

Transcript
  • 8/2/2019 Online Report

    1/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840

    The University of Nottingham

    The School of Computer Science

    & Information Technology

    Computer Assisted Assessment within a Virtual

    Learning Environment

    G6DDIS MSc Dissertation

    5th September 2004

    Project supervisor: Elizabeth BrownAuthor: Jonathan Marshall

  • 8/2/2019 Online Report

    2/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 2

    Submitted September 2004, in partial fulfilment of

    the conditions of the award of the degree M.Sc. in I.T.

    Jonathan Marshall

    School of Computer Science and Information Technology

    University of Nottingham

    I hereby declare that this dissertation is all my own work, except as indicated in the

    text:

    Signature ______________________

    Date _____/_____/_____

  • 8/2/2019 Online Report

    3/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 3

    Abstract

    A Virtual Learning Environment (VLE) is an Information Technology tool to assist

    traditional classroom based teaching and learning. VLEs are commercially available, but

    the license to use these is becoming increasingly expensive, and would be costprohibitive to use on a small scale. This project has focused on developing a VLE to

    assist teaching students studying computing courses within the Further Education sector.

    The system is tailored specifically to meet the requirements of both the tutors and

    students. Computer Assisted Learning has been integrated within the VLE to provide a

    method for formatively assessing learning throughout the duration of the taught courses.

    The results from the assessments are computed allowing timely feedback to be provided

    in graphical and tabular format. Furthermore, two channels of communication have been

    provided as a discussion forum and online form to improve interactions between the

    tutors and students, and among students. The VLE also allows submission of assignments

    via an uploading procedure to a data repository. Learning resources are also available for

    students to download and print out. Usability testing has been conducted to measure

    users satisfaction, efficiency and effectiveness of the developed system. On the basis of

    this study, practical implications of utilising a VLE within education were discussed, and

    what constitutes an effective VLE was also suggested.

  • 8/2/2019 Online Report

    4/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 4

    Acknowledgement

    My unreserved thanks to my supervisor for this project, Elizabeth Brown who has

    stimulated my intellectual curiosity and inspired me to develop this work.

    I am grateful for the financial support contributed towards my course fees provided by

    my employer Broxtowe College of Further Education, and to my line manager Barbara

    Mercer for the many alterations she made to my teaching timetable to accommodate

    lecture times, without her help this MSc course would not have been possible.

    I wish to thank my parents Margaret and Keith, and my parents in-law Yao-hsin and Yu-

    Lan, without their help with my newborn child, Jasmine, this dissertation would not have

    been possible; my gratitude towards all family members cannot be measured or

    estimated. Their emotional support and love provided me with the strength to face all the

    difficulties encountered during the last two years of study.

    I cannot end my acknowledgement without mentioning my wife, Mei-I, who over the

    years we have been together, has had a profound influence and unwavering confidence in

    my abilities, without her support I would have never completed this qualification. I thank

    her for her tolerance and patience for the many hours that I have spent studying; I strive

    to grow and taste a fraction of her academic success; here words are but empty symbols

    of inexpressible feeling.

  • 8/2/2019 Online Report

    5/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 5

    Table of contents

    INTRODUCTION........................................................................................................................................ 8

    WHAT IS A VIRTUAL LEARNING ENVIRONMENT (VLE)?............................................................................ 9What constitutes a VLE?..................................................................................................................... 10What does a VLE do?.......................................................................................................................... 11

    History ................................................................................................................................................ 13 Comparison to the traditional classroom ........................................................................................... 14Social Environment............................................................................................................................. 14Social Interaction................................................................................................................................ 15Online Learning.................................................................................................................................. 15

    ASSESSMENT ............................................................................................................................................ 16Multiple choice questions.................................................................................................................... 18

    MOTIVATION FOR THIS STUDY .................................................................................................................. 20Rationale............................................................................................................................................. 21 What is the problem that is being solved; ........................................................................................... 26Student motivation .............................................................................................................................. 28

    SYSTEMS ANALYSIS .............................................................................................................................. 29

    SYSTEM DEVELOPMENT LIFE CYCLE MODELS ......................................................................................... 30Model Comparison.............................................................................................................................. 30

    REQUIREMENTS ANALYSIS ....................................................................................................................... 33Student Profile .................................................................................................................................... 33Student requirements .......................................................................................................................... 35System administrator requirements..................................................................................................... 36Tutor requirements.............................................................................................................................. 37System requirements ........................................................................................................................... 38

    SYSTEM SPECIFICATION ............................................................................................................................ 40DATA FLOW DIAGRAMS ........................................................................................................................... 41

    Context Data Flow Diagram VLE (Figure 3) .................................................................................. 43Level 1 DFD (Figure 4) ................................................................................................................... 44 Login/Logout Level 2 (Child Diagram) DFD (Figure 5) ................................................................. 45

    Add/Read Message - Level 2 DFD (Figure 6) .................................................................................... 45 Add/Read Message - Level 2 DFD (Figure 6) .................................................................................... 46Create/Take Assessments Level 2 DFD (Figure 7) .......................................................................... 46Create/Take Assessments Level 2 DFD (Figure 7) .......................................................................... 47Generate Results Level 2 DFD (Figure 8) ....................................................................................... 48View Learning Materials Level 2 DFD (Figure 9) .......................................................................... 48View Learning Materials Level 2 DFD (Figure 9) .......................................................................... 49Upload Assignment Level 2 DFD (Figure 10) ................................................................................. 50

    Input/Edit Personal Details Level 2 DFD (Figure 11) .................................................................... 50Input/Edit Personal Details Level 2 DFD (Figure 11) .................................................................... 51

    ENTITY RELATIONSHIP MODELLING...................................................................................................... 52PLANNING ................................................................................................................................................ 54SOLUTION OVERVIEW ............................................................................................................................... 54

    Interface Design.................................................................................................................................. 54 Screen design layout ........................................................................................................................... 57System Flowchart................................................................................................................................ 60

    Prototype............................................................................................................................................. 65 Database design.................................................................................................................................. 65

    IMPLEMENTATION................................................................................................................................ 69

    IMPLEMENTATION OF THE SOFTWARE....................................................................................................... 71Programming Languages.................................................................................................................... 72 Programming...................................................................................................................................... 77

  • 8/2/2019 Online Report

    6/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 6

    VLE Files ............................................................................................................................................ 79Problems encountered ........................................................................................................................ 81

    EVALUATION........................................................................................................................................... 81

    TESTING THE VLE WITH DIFFERENT AND ENVIRONMENTS ....................................................................... 84Usability testing. ................................................................................................................................. 85

    STATISTICAL SURVEYS OF USER EVALUATION QUESTIONNAIRES

    .............................................................. 90Statistical analyses.............................................................................................................................. 91IMPROVEMENTS ........................................................................................................................................ 92

    DISCUSSION.............................................................................................................................................. 94

    THEORETICAL AND PRACTICAL IMPLICATIONS ......................................................................................... 94LIMITATIONS ............................................................................................................................................ 95HOW DOES THIS VLE COMPARE TO EXISTING SYSTEMS?.......................................................................... 95FUTURE DIRECTION .................................................................................................................................. 96

    CONCLUSION........................................................................................................................................... 97

    REFERENCES........................................................................................................................................... 98

    BIBLIOGRAPHY .................................................................................................................................... 103

    APPENDICES .......................................................................................................................................... 105

    APPENDIX 1 - SCHEME OF WORK........................................................................................................... 105APPENDIX 2 SCREEN DESIGN .............................................................................................................. 106APPENDIX 3 - VLE PROGRAM LISTINGS ................................................................................................ 116

    Styles/styles.css (Cascading Style Sheet source code) ...................................................................... 116index.php (source code) .................................................................................................................... 117topheader.php (source code)............................................................................................................. 118leftnav.php (source code).................................................................................................................. 120

    footer.php (source code) ................................................................................................................... 125 template.php (source code) ............................................................................................................... 126

    subjectselection.php (source code) ................................................................................................... 126Assessment/assessmentChoice.php (source code)............................................................................. 127

    Assessment/assessment.php (source code)........................................................................................ 128 dbConnect.php (source code) ........................................................................................................... 132Login/login.php (source code) .......................................................................................................... 132 Login/logoff.php (source code) ......................................................................................................... 132 LearningResources/SoW.php ............................................................................................................ 133 ContactForm/ContactForm.php (source Code)................................................................................ 134StudentDetails/personalDetails.php (source code)........................................................................... 137StudentDetails/accountFunctions.php .............................................................................................. 140StudentDetails/createAccount.php.................................................................................................... 141StudentDetails/results.php ................................................................................................................ 143StudentDetails/resultGraph.php ....................................................................................................... 144StudentDetails/submitWork.php........................................................................................................ 145StudentDetails/validateRTF.php....................................................................................................... 147

    AboutMe/timetable.php..................................................................................................................... 149 AboutMe/personalProfile.php........................................................................................................... 151 AboutMe/cv.php................................................................................................................................ 152

    APPENDIX 4 - TEST DATA ...................................................................................................................... 155APPENDIX 5 - USABILITY QUESTIONNAIRE ............................................................................................ 157

    Part 1 Functionality.......................................................................................................................... 157Part 2 User Input .............................................................................................................................. 157Part 3 System Output ........................................................................................................................ 157Part 4 User Guidance and Help ....................................................................................................... 158 Part 5 Consistency............................................................................................................................ 158

  • 8/2/2019 Online Report

    7/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 7

    Part 6 Flexibility............................................................................................................................... 159 Part 7 Error Correction/Handling and Robustness.......................................................................... 159 Part 8 Overall System Usability........................................................................................................ 160

    APPENDIX 6 - HISTOGRAM OF USABILITY QUESTIONS ............................................................................ 161

  • 8/2/2019 Online Report

    8/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 8

    Introduction

    The use of Information Technology (I.T.) within educational environments has

    significantly increased since the arrival of the personal computer. I.T. has been used to

    enhance teaching and learning within the classroom environment. It has been the rapid

    expansion in the use of the Internet that has led to a new era in collaborative and

    interactive learning that is dynamic and not restrained by geographical location (Ma,

    Vogel, and Wagner, 2000). I.T. has affected all aspects of education not just teaching and

    learning. Patalong (2003) identifies that recently there has been major transformations

    throughout most stages of education, moving from talk and chalk tradition classroom

    teaching to student centred, autonomous learning. There has been a large increase in the

    student population which has also more diverse, and the significant reduction in the cost

    of computing technology has seen a huge increase in the use of IT.

    This project investigates how a Virtual Learning Environment (VLE) may improve

    teaching and learning, and identifies which aspects are most beneficial to students.

    Furthermore, a VLE will be developed to be employed as a learning aid for students

    within the Further Education environment. The Internet will provide a platform to deliver

    the VLE as it has many benefits over stand-alone applications that are discussed later infurther detail. The main consideration being that the Internet is a cost effective

    distribution method, but pedagogically, it is important that there is dynamic interaction

    between the real and virtual learning environments (VLE) for quality student learning to

    occur.

    The term Online delivery refers to distributing learning materials over the Internet and

    provides the opportunity to increase both the quantity and quality of interaction between

    tutor and student and among students, and improve access to education and training. But

    a critical success factor is the pragmatic access to I.T. such as equal access to computers

    and the Internet for all tutors and students. Lo Choi Yuet Ngor (2001) states that the

    Internet is widely available in educational establishments around the world offering new

    and diverse possibilities for delivery and communications,and provides learners with a

  • 8/2/2019 Online Report

    9/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 9

    more flexible approach to learningat times and places to suit their lifestyle. Online

    delivery also helps to reduce the cost and improve the cost-effectiveness of education. It

    also empowers learners to improve their knowledge and to learn more quickly, by

    accelerating the adoption of new information and new programs (ibid.).

    For the student, an online learning environment is a convenient way of accessing a range

    of material related with a course of study. Since the vast majority of all of the

    asynchronous components of a typical course of study the study materials, the

    administrative information and the exchanges between academic staff and students are

    today electronic, an online VLE offers no more than a logical modular structure into

    which these components can be fitted (Maccoll, 2001).

    What is a Virtual Learning Environment (VLE)?

    Since the rapid growth of the Internet in the early 1990s, many new software tools and

    products have been developed to take advantage of its numerous benefits. Since the mid-

    1990s the education community has seen the emergence of software products categorised

    as VLEs that aim to support teaching and learning activities and these have harnessed

    the potential of the Internet for their delivery (Virtual Learning Environments, n.d.).

    A VLE is a teaching and learning tool designed to enhance the learning experience by the

    inclusion of computers and the Internet in the learning process. To access the VLE users

    are required to register their personal details and are assigned a student ID. The tutor has

    additional user rights to create, upload and modify curriculum content and track student

    performance.

    The main components many commercial VLE software packages include curriculum

    mapping which involves dividing the curriculum into manageable sections that can be

    delivered and assessed, also they include monitoring students performance, online

  • 8/2/2019 Online Report

    10/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 10

    support, and electronic communication (e-mail, threaded discussion forums, and Internet

    links to outside curriculum resources).

    What constitutes a VLE?

    A textbook cannot be considered a learning environment. But, reading a book, attending a

    lecture, discussing thoughts with other students, writing an assignment for the tutor, do

    constitute a learning environment. Similarly, a set of static Web pages does not constitute

    a VLE unless there is social interaction about or around the information provided. Social

    interaction may be described as synchronous (e.g. discussion forums, chat rooms etc.)

    versus asynchronous (e.g. electronic mail, bulletin boards) communication, between one-

    to-one, one-to-many or many-to-many, and can use several mediums; text-based, audio

    and video, and also includes indirect communication such as sharing files, although it canbe difficult to maintain data integrity when providing simultaneous write access to a file

    (Dillenbourg, 2000).

    Online interactions between students and tutors form the basis of an effective VLE and

    offer the user a variety of communication and information presentation, and assessment

    tools through the convenience of a conventional Web browser interface. Implementation

    requires the use of a programming language that supports the generation of dynamic Web

    pages from a server-side database that stores information and maintains user state and be

    accessed via a standard Web server. Many educational institutions would benefit from the

    advantages offered by a VLE provided over the network to support their teaching and

    learning activities.Creating online resources that are motivating, interesting, intuitive and

    educationally sound requires considerable expertise, and is time consuming and therefore

    costly. Effective VLEs should allow tutors to create content-rich resources rapidly and

    without the need to learn many new technical skills.A typical Web-based VLE should

    provide an integrated set of information management tools, which facilitates easy upload

    of learning materials and offer a consistent interface which may be customised by the

    user to suit their requirements.

  • 8/2/2019 Online Report

    11/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 11

    Dillenbourg (2000) suggest that the principal components of a VLE should allow the

    following features:

    Mapping of the syllabus into convenient elements that can be easily assessed and

    recorded;

    Monitoring of student activity and accomplishment against the elements of the

    syllabus;

    Support of online learning, including access to learning resources, assessment and

    guidance;

    Online support from the tutor and peer group using asynchronous or synchronous

    communication methods;

    Other general communications, including email, discussion forums, bulletin

    boards and chat rooms;

    Links to other information systems, both internally and externally on the Internet.

    What does a VLE do?

    The principle functions that a successful VLE needs to provide are controlled access to

    learning materials that have been mapped to the subject specification that may be

    separately assessed and recorded. Student activity and achievement requires tracking

    against the subject to direct, guide and monitor learner progress. Immediate guidance oflearning and assessment procedures to provide a mode of communication between the

    learner, the tutor and other learning support specialists thus providing direct support and

    feedback for learners, as well as peer-group communications- this helps to build a sense

    of group identity and community of interest. Also, a VLE would benefit from having

    links to administrative systems, thus reducing the change of data being redundant by

    unnecessary duplication. The features of a good VLE is shown in Table 1.

  • 8/2/2019 Online Report

    12/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 12

    Features Examples

    Communication between tutors and

    students

    E-mail, discussion boards and virtual

    chat facilities which support various

    types of communication: synchronous

    and asynchronous, one-to-one, one-

    to-many and many-to-many.

    Self-assessment and summative

    assessment

    multiple-choice assessment with

    automated marking and immediate

    feedback

    Delivery of learning resources and

    materials

    Through the provision of learning and

    teaching materials, images and video

    clips, links to other Web resources,

    online discussion and assessment

    activities.

    Shared work group areas Allows designated groups of students

    to upload and share files as well as

    communicate with each other.

    Support for students Could take the form ofcommunication with tutors or other

    students, provision of supporting

    materials such as course information

    and Frequently Asked Questions.

    Student tools Individual student Web pages, drop

    boxes for the upload of course-work,

    electronic diaries and calendars.

    Management and tracking of

    students

    Usernames and passwords to ensure

    that only registered students can

    access the course; analysis of

    assessment undertaken by students or

    their use of materials within the VLE.

  • 8/2/2019 Online Report

    13/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 13

    Consistent and customisable look

    and feel

    A standard user interface that is easy

    for students to understand and use.

    Courses can be individualised with

    colours, graphics and logos but the

    essential mode of use remains

    constant.

    Navigation structure Structured delivery of information

    supported by a standard navigation

    toolbar. Most VLE software assumes

    that students will work their way

    through linear sequences of

    instructional material. Others are

    more flexible and will accommodate

    alternative information structures,

    e.g. multi-path case studies.

    Table 1 Features of an effective VLE, resource from Virtual Learning

    Environments (n.d.)

    History

    Dillenbourg (2000) mentions that the history of educational technology illustrates that

    new technology (television, computers, hypertexts, multimedia, Internet, and virtual

    reality) raise a wave of naive expectations regarding to the intrinsic effects of these

    technologies. Still every medium has some specificity, but this specificity can only

    expressed in terms of potential effects. A medium affords specific types of interactions:

    computers enable individualised instruction but not all software is individualised,

    computers are able to analyse the learner behaviour, to manipulate interactive models, but

    most programmes do not perform this function.

    As Maccoll (2001) stated that VLEs began to appear in the second half of the 1990s.

    They were initially client software and not delivered via the Web. As the software needs

  • 8/2/2019 Online Report

    14/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 14

    to reach a wider audience the Web has gradually become the dominant delivery platform

    due to the fact that it can be served and viewed on any platform connected to the Internet.

    As the uptake in VLE has increased, it is proving less difficult providing networked

    personal computers with Web browsers than it is providing the same computers with

    specialised client software. However, one of the main advantages in developing courses

    on the Web is the ability to hyperlink resources stored locally on an Intranet or externally

    on the Internet.

    Comparison to the traditional classroom

    It is important to ask the question of whether computer-based learning is more effective

    than learning in a conventional classroom. This question is deemed irrelevant for the vast

    majority of those, who view technologies as a tool to assist the tutor and not as substitute(Dillenbourg, 2000).When comparing computer-based learning and the conventional

    teaching method, we do not measure the inherent effectiveness of humans against

    computers, but the quality of two learning methods as implemented by two techniques.

    The conventional physical learning environment generally integrates courses, resources

    such as libraries, laboratories, formal communication (question and answers, notice

    boards) and informal communication, an administration (largely paper based) process etc,

    where similarly, a VLE integrates a variety of tools supporting similar functions to those

    that exist in the physical environment: information, communication, collaboration,

    learning and management.

    Social Environment

    Hiltz, Coppola, Rotter and Turoff (2000) state that the instructional aspect of a VLE

    refers to providing study materials and learning resources via the Internet that is relevant

    to the pedagogical theories from educational research. Whereas, the interactive aspect of

    a VLE refers to the opportunity for the students to be actively involved with online

    collaborative (group) learning e.g. case studies, group discussions using forums, instant

    messages, or chat rooms. In addition, students may use e-mail for one-to-one

    communications between tutors and students, for seeking advice, clarifications, and

    asking individual questions. The interactive aspect is pertinent to group interaction/social

  • 8/2/2019 Online Report

    15/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 15

    theories from social psychology and sociology. Understanding students perceptions and

    skills within these two aspects is considered essential in assessing students readiness

    toward online courses.

    Social Interaction

    Dillenbourg (2000) identifies the first obvious opportunity of virtual learning

    environments is that they support social interaction, in many ways: synchronous versus

    asynchronous, text-based versus audio or video, one-to-one versus one-to-many. Again,

    these affordances define potential effects, not actual ones. For instance, we often

    encounter tutors who believe that, since their students use e-mail, they will start to ask

    frequent and intelligent questions. However, this is rarely the case as most email

    conversation is about the management of learning (finding resources, negotiatingdeadlines, asking for an appointments etc.). The same applies to educational forums, in

    which it is very difficult to sustain the flow of messages. The problem is not due to the

    technology, but to the educational context. Students will not start communicating with the

    tutor for the sake of communication.

    Online Learning

    It is relatively easy for students to submit assignments electronically and sit tests or

    assessments online; this has led to many educational institutions exploiting the positive

    aspects of computing technology to provide their courses online to relieve tutors from the

    time related aspects of marking.

    Initial computer-based learning programs tended to be electronic books that did not meet

    the original claims of interaction or user-centred learning. Technology is now

    commonplace which allows authors to design and produce teaching materials which

    offers the learner a choice of learning methods and routes, implementing a range of

    audio, visual, and text materials, and opportunities to interact meaningfully with the

    content, unfortunately the resulting course materials are often prohibitively time

    consuming to prepare, and require considerable educational design expertise.Sharing

    computer-based teaching material within groups is difficult to achieve, although the

    Internet, and CD-ROMs overcome interoperability problems in sharing computer-based

  • 8/2/2019 Online Report

    16/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 16

    data, but it is the quality of teaching material is a the major concern. Students that are

    familiar with using online learning tend to rely heavily on the Internet for their research,

    rather than using books, journal articles, encyclopaedias etc., this can have an adverse

    affect on the final quality of their work as materials published on the Internet are not

    regulated for quality.

    On-line courses using a resource-based model have an underlying aim of learning how to

    learn; which encourage knowledge management skills such as selecting and searching for

    information which helps the learners discover how and where to find answers and

    understand solutions, and to transform and present ideas. The Internet provides both the

    cause and an ideal means for doing this.

    Therefore, contemporary approaches to teaching and learning are dominated by the

    importance of interactivity in the learning process, the requirement for knowledge

    management skills and the capacity for team working, and the trend towards resource-

    based rather than packaged learning.

    Assessment

    Each subject area will have a self assessment associated with it, which will enable the

    students to monitor their learning progress and provide an opportunity for formative

    feedback relating to the areas of knowledge that require further work (Patalong, 2003). At

    the end of each assessment, hyperlinks to Web pages will be provided to those questions

    that were answered incorrectly to promote students knowledge in the area of difficulty.

    Additionally, suggestions on how to search for the particular subject area will be made

    available for students who were still unsure, and as a final stage, answers to the

    incorrectly answered questions will be provided at the end of the assessment with

    statistical feedback, thus allowing students to monitor their progress. An opportunity for

    reassessment will be provided for the formative assessments, but this will be provided

    before the answers are given to encourage the students to research the answers by

    themselves. Assessment will be online, multiple-choice questions, which will be

    automatically marked and graded by the VLE. It is not intended that these assessments

  • 8/2/2019 Online Report

    17/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 17

    will replace face-to-face teaching; the objective is for them to supplement teaching and

    provide the students opportunity to consolidate the learning that has taken place within

    the conventional classroom.

    Black and William (1998) use the term Assessment as referring to the group of

    activities undertaken by both tutors and students in assessing themselves, providing both

    grades and feedback to modify teaching. They consider assessment to be Formative

    when the evidence is actually used to adapt the teaching to meet the students needs. It is

    generally accepted that summative assessment is a tool with the primary objective of

    grading the student.

    Recently, Farrell and Leung (2004) suggests that assessment plays a critical role in the

    educational process as both a means of grading and supplying valuable feedback to the

    student. The embracing of technology as an integral part of the delivery of education has

    increased the availability of effective assessment tools to be utilised by the student,

    encouraging self-assessment at all stages of the learning experience. It is accepted

    wisdom that testing for the purpose of feedback should be a routine part of the ongoing

    classroom activity rather than an interruption. Traditionally both formative and

    summative methods of assessment have been reliant on the tutor to supply the feedback.

    With the ever-increasing demand on the tutors time unintentional delays in supplying

    feedback to the student can occur to the disadvantage of both the student and the tutor. To

    maximise the effect and value of the feedback it must be instantaneous, which is an

    inherent characteristic of online assessment tools due to the nature of the encompassing

    technology. The formative assessment feedback cycle should occur early in the learning

    experience and often for the duration of the subject.

    Moreover, Sclater and Howie (2003) categorise assessment into three types

    1. Credit weighted (summative) assessment which may be either formal

    examinations or continuous assessment;

    2. Self (formative) assessment which may be authenticated self-assessment or

    anonymous self-assessment;

  • 8/2/2019 Online Report

    18/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 18

    3. Diagnostic assessment; enabling a student and/or tutor to evaluate the students

    knowledge by pre-testing before the course is commenced or post-testing in order

    to evaluate the effectiveness of the learning.

    Therefore, the system developed in this study will use an innovative approach to a Web

    based self-assessment tool based on the traditional multiple choice questions format. The

    generated students score is designed to reflect their understanding of the reviewed topic.

    Additionally, the aforementioned three types of assessment will be integrated into the

    system.

    Multiple choice questions

    Farrell and Leung (2004) argue that multiple choice questions are extensively employed

    in education for both a formative and summative assessment. The acceptance and use ofmultiple choice questions as an assessment tool can be attributed to their ability to assess

    broad fields of learning in a compact system whilst being quick to assess and provides

    reasonable feedback to the students with minimal associated costs. Computer Assisted

    Assessment (CAA) is most commonly associated with multiple-choice questions, due to

    the fact that it is relatively easy to adopt the multiple choice questions format to the

    digital environment, and also having the appeal of being able to produce fully integrated,

    automated tests.

    On completion of the multiple choice questions test the resulting score with the identified

    correct answers is instantaneously provided as feedback. The incorrect answers are also

    identified with a brief explanation. It is also possible to guide the student to the

    appropriate source for further study. This method is effective but is dependent on the

    student answering the questions honestly, without guessing. The resulting data will be

    recorded directly to a database. It is advantageous to collect the data using this method as

    the technology is ideal for collecting and analysing raw data.

    Marking open-ended questions

    Computers can easily assess and grade closed type questions; the responses being closed

    and therefore restricted to a finite answer limiting the possible responses. Open-ended

  • 8/2/2019 Online Report

    19/177

  • 8/2/2019 Online Report

    20/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 20

    However, Bull and Danson (2004) suggest that there is a certain amount of scepticism

    about the ability of multiple-choice questions to test anything more than fundamental

    understanding, and that multiple choice questions assess declarative knowledge, usually

    by the least demanding process of recognition.

    As a system that assesses open-ended questions would be beyond the scope of this project

    it was decided that a system that allows assessment by multiple choice questions would

    be more suitable. Other functionality could be provided to allow the collection of

    assessments that contain open-ended questions; this could be implemented by allowing

    students to upload assignments to a data store.

    Motivation for this studyThe reason behind selecting a project of this nature is to assist with my present

    employment. I currently work as an I.T. Lecturer within Further Education and for

    several years used HTML to create static Web pages to deliver online materials to the

    students. The initial motivation for using the Internet to distribute resources is that many

    of the students do not keep (or fail to file) printed materials from lessons. Organising the

    lesson content allows the students to access all materials used throughout the duration of

    the course, and reduces the need for printing duplicate materials. I have found this

    method successful and wanted to further develop the use of the Internet to assist teaching

    and learning.

    Futhermore, my teaching is very focused towards summative assessment methods with

    less regard to formative assessment. Therefore, utilising a VLE that incorporates online

    assessment enables the students to undertake more formative assessments, and thus

    provide both parties with valuable feedback. The feedback would enable the content of

    the course to be adapted to suit the needs of the students, and provide a predictive

    indication of future achievement.

  • 8/2/2019 Online Report

    21/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 21

    Rationale

    The motivation to employ Information and Learning Technologies (ILT) is not to cover

    more material within the duration of the course, but to engage students with significant

    intellectually learning resources to assist them in achieving higher levels of learning, and

    thus enhanced results.

    As McCray (2000) mentioned the potential of ILT as an effective alternative medium by

    which to carry out some of the activities undertaken in the conventional classroom

    setting, especially content delivery. Some of the activities can be successfully transferred

    outside the physical classroom releasing valuable class time that may be reallocated to

    more beneficial learning activities to improve the overall learning outcomes.

    Benefits

    Using a VLE to assist teaching within Further Education would provide many tangible

    benefits to both the students and tutor.Some of the reported benefits of the use of ILT

    within teaching and learning include improved cognitive learning processes and enhanced

    educational outcomes.

    Winn (2002) found that research into the educational effectiveness of VLEs shows that,under the right environment, it can have a positive impact on students of varying ages,

    studying a wide variety of subjects. The learning activities within a VLE are important

    components of the course and they promote learning and move students towards the

    programme objectives.

    Reported benefits (Inglis, 1999) to Students of using a VLE:

    Remote access to learning materials;

    Allow electronic submission of work, reducing the need for paper wastage;

    Real-time communications allowing peer support and additional tutor support;

    Discussion forum provides a reference for students who may be encountering

    similar problems;

  • 8/2/2019 Online Report

    22/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 22

    Can monitor performance from both the online testing and summative results

    posted by tutor;

    Timesavings - Reduced search time, less travel time to fetch or deliver materials.

    Improving access to education and training Internet provides access (anytime

    and any place) to information that might have been mislaid;

    Scheduling flexibility - Allows organisation of own time and inclinations with

    respect to educational interaction;

    Improved communication - Increased access to tutors and fellow students outside

    of the classroom;

    More ways to contribute and achieve better grades - Improved opportunities to

    contribute knowledge to the benefit of the group which is not limited to in-class

    discussion;

    Anonymity - Ability to comment anonymously without the scrutiny of direct face-

    to-face communication.

    Reported benefits for Tutors and educational establishments:

    Improving the quality of teaching, learning and services - online delivery allows

    the increase of both the quantity and quality of interaction between tutor and

    student and among students;

    Long term time savings - Reduced time to compile materials due to a higher level

    of reusability and sharing of learning materials;

    Better teaching quality a more structured approach to classroom delivery has a

    direct impact on the quality of teaching;

    Improved student performance based on a greater personal choice; which also

    leads to better teaching evaluations;

    Scheduling flexibility - The opportunity to interact and help engage students

    without the rigidity imposed by conventional class scheduling; Better student response - Students perceive a higher teaching quality;

    Reducing the (long-term) costs of education;

    Improving the cost-effectiveness of education;

    Empowering learners to know more and learn faster.

  • 8/2/2019 Online Report

    23/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 23

    Benefits to tutor(s):

    Allow easy distribution of materials;

    Provides a simple method to formatively test;

    Improved motivation as students performance can be monitored.

    A VLE is used to enhance (rather than replace) conventional teaching, so the students

    will therefore receive more input than they did when they only had a traditional

    information skills session. Patalong (2003) suggests that by imposing exam conditions for

    the Computer Assisted Assessment (CAA), academic staff indicated to students that the

    ability to retrieve, evaluate and provide a considered answer is a serious exercise.

    More than this, Dillenbourg (2000) states that as the students can easily retrieve more

    information from a larger variety of information sources via the Internet, although this

    does not guarantee effectiveness of learning.This affordance differentiates Web-based

    environments from previous systems. The effectiveness depends upon the way the

    designer take advantage of this opportunity, which raises two questions for the designer

    of VLE content:

    1. How to address the problems concerning quality control and difficulty to findinformation; either to pre-select information by gathering the information

    considered as relevant, or to provide access to the Web but to demonstrate how to

    search for information and be critical with selection;

    2. What role does the designer allocate to information access in the learning

    process? There is a risk that designers confuse setting up a learning environment

    and providing access to information. The risk of simple knowledge transmission

    is quite high if one looks at University Web sites, but it is not present at primary

    school level.

    CAA allows automation of the marking process and more instantaneous results; marks

    (statistics and/or graphically) being automatically displayed to students immediately upon

    completing the assessment (Jacobsen & Kremer, 2000).

  • 8/2/2019 Online Report

    24/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 24

    Disadvantages

    Patalong (2003) highlights the concerns about academic honesty; online assessment

    should only count for a small portion of the overall course grade. Unless direct measures

    are implemented to prevent communication between students, online assessment is bettersuited to formative assessment that does not contribute to the final course grade.

    Summative assessment needs to have more rigorous control measures to eliminate people

    from seeking assistance, Zhang (2002) mentions about mixing the questions up

    randomly, to prevent students communicating electronically by referring to question

    numbers or having the same questions as those on their neighbours screens. The

    assessment can be password protected so that it could not be accessed beforehand. On

    arrival to the assessment room students can present some form of identity and complete

    attendance forms, once these have been complete and are ready to begin the test, the

    password was may be released. Soon as everyone has begun the test, the password for the

    test can be changed, to prevent anyone trying to text message via mobile phone the first

    password to friends for assistance. Also, students should be prevented from leaving the

    room until the password had been changed.

    There is the problem of combining online assessment with a discussion forum, the

    students undertaking the assessment could use the discussion forum to distribute the

    answers. Methods to combat this problem would be either to prevent access to the

    discussion forum whilst undertaking the assessment, which does not prevent the students

    from using other external discussion forums, or to randomise the order of the questions.

    The second method is preferred providing that the students are not aware of this measure,

    as it provides a minor barrier that is easy to implement. But this is the main reason for not

    using a system of this nature for important summative assessments.

    Lee (2002) states one of the chief advantages of using Web-based teaching is that it

    provides real-time information, and many processes of communication and collaboration

    between students and tutors are increasingly computer-based. From the students

    viewpoint, perhaps the most significant and detrimental factors to the success of a VLEs

  • 8/2/2019 Online Report

    25/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 25

    are stress, association with the use of technology, and dissatisfaction towards the

    technology itself. It is suggested that the success of any VLE depends on the level of

    computing skills and attitudes of students. Students with anxiety towards new

    technologies or are less prepared for computer use are most likely to experience

    difficulties in a VLE.

    As a result, Brace-Govan (2003) attempts to look at how it is possible for the tutor to

    monitor and gauge educational activity within discussion forums. It was found that it is

    difficult to monitor students communications in a text-based discussion forum,

    especially if an online discussion is successful and has significant contributions.Rovai

    (2000, p. 290 in Brace-Govan) argue, online instructors must deliberately structure

    interactions to overcome the potential lack of social presence and noted how critical

    timing is in this.

    In addition, Zhaos (2003) empirical studies show that there are a few common problems

    that online students have encountered with VLEs, which include:

    Technical problems, including limitations with bandwidth, problems with browser

    software, etc.;

    Inaccessible instructors- it is not always possible to coincide online activities; Lack of relevant and/or comprehensive online course materials;

    Reduced intimacy associated with face-to-face traditional delivery methods in

    classrooms;

    The feedback received was delayed;

    Reduced interactivity and interaction with the tutor;

    Tutors inability to monitor prompts from students;

    Course contents and assessments remains unchanged to suit the different learning

    style;

    Online students needs are different;

    Undertaking a generic approach without considering appropriate learning design

    in the application of information technologies.

  • 8/2/2019 Online Report

    26/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 26

    Dillenbourg (2000) mentions that the Internet is growing exponentially and it provides

    tutors and students with unlimited access to vast quantities of information. This offers

    many advantages but also some negative issues that often go unconsidered. The quality of

    information is unchecked; there is often no validity assessment to determine the accuracy

    of the published information. Theremay also be an overflow of information making it

    difficult to locate relevant information. Unless a specific Internet filter is used, students

    may encounter pages that contradict the taught ethical and moral values. The information

    on the Internet is unstructured, there is a distinct lack of meta-information (e.g. the

    author, if the source is reliable, the period the information remains valid, whether this

    version is the latest etc.), and the Internet has become a large unorganised file repository.

    The semantic Web movement has evolved to combat many of the weaknesses of the

    Internet and to provide an inherit structure to its content.

    The problem that needs to be addressed

    This project needs to include several features that are apparent in many commercially

    available VLEs: A detailed VLE comparison has been extensively examined by Eduserv

    chest (2003). The features need to allow communication between all users of the system,

    asynchronous communication implemented as email, and synchronous communication

    implemented as a discussion forum. It is preferable to provide a form to enable users to

    email the tutor, although this does not allow a copy of the email to be saved as

    conventional email systems (i.e. the sent items folder), it does have the benefit of

    providing instant communication (rather than having to open their own personal Web-

    based email account) which would encourage students to ask more questions. A copy of

    the message would be appended to the reply, which minimises this drawback. The

    mailto html command was discarded due to the fact that not all users have an

    operational default mail application.

    Learning materials also should be distributed as Web pages that can be viewed, saved to

    the local hard disks or printed out. As the materials are displayed as HTML, Cascading

    Style Sheets will be utilised to control the dimensions of the displayed output, this also

    minimises the effort required to change the style of the site. Creating hyperlinks to word-

  • 8/2/2019 Online Report

    27/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 27

    processed documents or even Adobes Portable Document Format (pdf) is uncomplicated

    but usually results in longer download times and increased network traffic, which may

    result in unacceptable waiting times if students are accessing the materials

    simultaneously. Learning materials have previously been organised according to an

    outline scheme of the course; this is displayed as a table showing the week number,

    theoretical content of the lesson, practical tasks including assessments, and downloadable

    resources (Appendix 1).

    In order to monitor students progress informal formative assessment will be provided,

    these are undertaken whilst logged into their account allowing results to be computed and

    immediate feedback provided. The results will be retained for progress monitoring, and

    be displayed both graphically and in tabular form. Jacobsen and Kremer (2000) state that

    a VLE needs to provide useful statistical and graphical summaries of individual

    assessments, current course grades and group results. This information is useful for tutors

    in analysing student learning and to gauge the effectiveness of individual assessment

    questions.The online assessment will be composed of closed questions in the form of

    multiple-choice questions. If open question answers are required then the students can

    download a Rich Text File from the Learning Resources area, complete the questions and

    then submit the work by uploading the document from the local machine to a data store in

    the VLE.

    As the VLE will contain personal details about the users a certain level of access security

    is required to comply with the Data Protection Act 1998. A secure login with a username

    and password text field (the characters being replaced by a symbol) will be sufficient. If

    the user forgets their password then an email facility will provide a reminder to prevent

    the user from being locked out from the system.

    It is also important to consider other non-functional aspects to the VLE, Zhao (2003)

    states that there are four requirements that are important in providing quality online

    teaching and learning courses; these being effectiveness, technological infrastructure to

    support adequate access, student and academic satisfaction.

  • 8/2/2019 Online Report

    28/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 28

    Although effectiveness is difficult to quantify, is may be measured with usability testing.

    It would also be beneficial to gather opinions of users after they have been using the VLE

    for a short period of time and adapt the VLE to suit their needs.

    Student satisfaction depends upon many variables within the physical (and virtual)

    learning environments; these include the quality of the course, adequate interaction with

    the tutor and appropriate peer collaboration, and appropriate service from I.T. technical

    support. Both asynchronous (e-mail) and synchronous (discussion forum)

    communications between students and tutors, and amongst students, are central to online

    learning in VLEs; well-timed feedback, whether from tutors or peers are vital to the

    success of learning, including learning online.

    It is essential to consider the benefits of either buying commercially available software or

    whether to develop the VLE in-house. Ellaway, Dewhurst, and Cumming (2003) identify

    some of arguments for using a commercially available VLE, these include: known

    purchase price as opposed to development costs of creating in-house which may be

    considerable more that first estimated, an ensured level of interoperability between

    known and reduced redundancy as it is likely that standard systems will be upgraded and

    developed in the future. Also, applied pedagogy, good practice and experience are easier

    to share between users of the same brand of VLE. The reasons for developing a VLE in-

    house include: matching the needs of the individual educational institution, maintaining

    control of the design and development process, and preserving in-house knowledge and

    expertise.

    Student motivation

    The role of tutors working within education is to provide students with the necessary

    knowledge whether theoretical, practical or technical to equip them with the skills

    beneficial in their future careers. De Lange, Suwardy, and Mavondo (2002) demonstrate

    that student satisfaction with using a VLE for this knowledge transfer is significantly

  • 8/2/2019 Online Report

    29/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 29

    linked with the provision of: lecture notes, discussion forums, on-line assessment and

    other tools including discussion forums and video summaries.

    Moreover, Becta ICT Research (n.d.) points out that disaffected and less confident

    students may be motivated by encouraging them to join in online discussions to share

    ideas, which ultimately helps to improve the quality of their work and to allows them to

    voice an opinion, whereas this might not happen in the traditional setting. A VLE

    encourages self-study and helps foster a higher level of learning, enabling students to

    engage in online discussions and improve their written communicative skills.

    Systems Analysis

    Before developing the system, the hardware and software requirements had to be

    analysed to ensure that there are sufficient resources to allow the production of the

    required software. After checking the University resources and available free trail

    versions of software downloaded from the Internet, it was decided that no additional

    hardware or software would need to be purchased.

    Correct planning and preparation is an essential first stage to any software-programming

    project. This becomes more complex with a larger project involving multiple teammembers. One of the initial activities is to look at the skills base of the people involved

    with the project; this is straightforward exercise with a single team member.

    The project then was decomposed, breaking the work into activities and identify tangible

    milestones (dates/targets) and deliverables associated with each of the activities. The

    main tasks of a software development project being as follows:

    Project planning & feasibility study;

    Systems analysis, requirements definition/gathering;

    Systems design describes desired features in detail, including screen layouts;

    business rules, process diagrams, etc.;

    Finalise design;

  • 8/2/2019 Online Report

    30/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 30

    Prototype/interface design;

    Development/coding;

    Testing; checks for errors, bugs and interoperability, user interface refinement;

    Maintenance throughout the rest of the software's life: changes, correction,

    additions etc.

    System Development Life Cycle Models

    After the initial project planning stage it was important to select a suitable design model,

    this then would allow the project plan to be revised to a higher degree of accuracy, and

    enable a more detailed schedule to be produced. Five suitable models as Catholic

    University of America (1999) suggest are introduced and compared below:

    Model Comparison

    Waterfall Model

    The waterfall lifecycle model (Fig. 1) is the most straightforward and controlled of the

    models considered. The software development is a series of ordered processes. Once a

    process is completed, the changes can only be made through a formal process. It is best

    suited for large projects where requirements are known, stable and understood. Its

    weaknesses include inflexibility for change, length of time before anything usable by the

    customer is produced and the implied requirement to complete every process correctly

    the first time.

    Incremental Model

    The incremental lifecycle model produces some results to the customer sooner. The

    initial processes of system requirements & feasibility, software requirements and generaldesign are performed in sequence. A partitioning into increments then occurs, where a

    number of different development efforts are identified. It is suitable for large projects

    with requirements that are known. It is the best choice when early release of some parts

    of the software is beneficial. When requirements are known and understood, but are not

  • 8/2/2019 Online Report

    31/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 31

    stable, it is a logical choice because later releases can incorporate changes that surface

    during the earlier development efforts

    Evolutionary Model

    The evolutionary lifecycle model applies in sequential aspects of the waterfall model, but

    adds the evolution or the discovery of requirements. The system requirements &

    feasibility and software requirements processes are not performed once for the entire

    project, but they are revised at the start of each cycle to incorporate the latest

    requirements. This is the preferable lifecycle model when requirements are not fully

    known. Benefits include the early delivery of some functions and the early testing of

    some assumptions before the entire system is built around them. The major weakness of

    this model is related to the inability to plan in detail at the outset of the project. Because

    the requirements are not fully known, inaccurate estimating and less than optimal

    architecture are possible. The predominately sequential nature of this lifecycle makes it

    not particularly rapid or cost efficient for complex systems.

    Spiral Model

    If the processes within the cycles of the spiral model were drawn in a cascading fashion

    like the waterfall, the standard processes from requirements & feasibility through

    operation & maintenance would fall in the same order and would be performed in

    sequence, just as in the waterfall model. While all projects have some risks, not all

    require extensive use of risk resolving techniques such as prototyping, simulation,

    benchmarking, administering user questionnaires, etc. The main benefit of the spiral

    model is in controlling project risks, preventing inadequate or unusable software from

    being built. The shortcomings of the spiral are that the additional processes are not as

    well defined as the standard processes, and there are not as many methods, techniques

    and tools to support them.

    It was decided to use a variation to the waterfall model (Fig. 2), the model is simple to

    understand and implement, it is a proven technique that allows relatively straightforward

    planning, and is still commonly used for software development. The variation called the

  • 8/2/2019 Online Report

    32/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 32

    Prototyping b model (due to it physical shape) is essentially a waterfall model that

    allows a prototype to be built in sequence with the actual system. This model shows some

    inherent flaws, as the results from the prototype do not feed into the system development.

    This was addressed and the prototype was built in close conjunction to the system.

    Needs

    Planning

    Analysis

    Design

    Implementation

    System

    Waterfall model

    Figure 1 Waterfall model

    Needs

    Planning

    Analysis

    Design

    Implementation

    Prototype

    Design

    Implementation

    System

    Prototyping b model

    Figure 2 Prototyping b model

  • 8/2/2019 Online Report

    33/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 33

    Requirements analysis

    Several methods were used to gather the requirements of the project

    Informal discussions with the target audience

    Observation of students using the current Web pages Comparison of existing VLEs.

    Student Profile

    The main users of this VLE will be post-compulsory education students that are both

    male and female and aged between 16 21 years and are studying either an Intermediate

    (level 2) GNVQ (General National Vocational Qualification) or AVCE (Advanced

    Vocational Certificate of Education, level 3) at Broxtowe College of Further Education.

    Other users will be students over 21 years and are studying towards a HNC or HND

    (Higher National Certificate/Diploma) in Computing, and to a lesser extent, part-time

    students that are studying a short course in the evening, with an age range between 20 to

    those that have retired. The majority of these students have previous computing

    experience and all have used the Internet in some capacity.

    There will also be users of this system that are visually impaired; ranging from one

    student that is completely without sight, those that are colour blind, those that may suffer

    from disabilities preventing them from reading or comprehending text properly (for

    example dyslexia), and others that may not be able to read small text (which affects a

    large segment of the population).

    It is important that all designers of interfaces for the Internet are aware about the

    importance of making websites accessible, aiding visually impaired users and the HTML

    guidelines to creating universally acceptable design solutions.

    There is commercially available software to help the visually impaired to use the Internet,

    but the success of these still depend upon good interface design concepts. Poor design is

    the biggest impediment for visually impaired users accessing and using websites.

  • 8/2/2019 Online Report

    34/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 34

    Browsers and tools developed for disabled people read the source code of the Web page

    and then render it providing speech or other (e.g. Braille) output. If the Web page is

    poorly constructed the alternate browser will encounter difficulties with interpretation

    and prevent the Web page from making sense to the user.

    Using standard HTML syntax helps to avoid important errors that may appear in the

    source code. Syntax errors prevent interpretation of the source code by alternate browsers

    and make it difficult to the user to take in the information. A validation service is

    available from the World Wide Web Consortium (W3C), which allows the HTML of the

    Web page to be externally checked by opening an image hyperlink that connects to the

    service. This will be placed on every page to ensure that the source code conforms to this

    standard.

    The use of tables requires extra care due to the way that table cells and rows are actually

    constructed in the source code; it is important that the source code interpreter extracts the

    information in logical order which makes grammatical sense. For example, it is

    preferable for text to be placed in vertical columns instead of a row, which may create

    confusion due to the source code first displaying the cells in a row before reading the

    second column cell.

    Visually impaired users are often unable to see images; they rely upon descriptions

    associated with them. Alternative text (ALT) tags for images and TITLE tag for other

    elements are essential to allow all elements of the page to be described.

    Graphics intensive Web interfaces are good for the able sighted Internet user but the

    visually impaired person has little or no ability to fully understand the message

    communicated by graphics, even when using an ALT tags. Using a reduced amount of

    graphics reduces download time for all users and is less distracting and stressful on the

    eye and makes the site more accessible for the visually impaired.

  • 8/2/2019 Online Report

    35/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 35

    Therefore, the VLE should not use an abundance of graphics and should ensure that

    hyperlinks are text based, rather than image based. Where images are required then

    alternative text should be provided to describe the image. The VLE will need to be

    validated using the software that Broxtowe College currently distributes to the visually

    impaired student (JAWS for Windows), and the usability tested with a visually impaired

    student.

    Student requirements

    Sclater and Howie (2003) state that theprimary requirement for all users of VLE system

    is that it is intuitive to operate and requires no formal training, although it is assumed that

    users will have a preliminary knowledge in the use of computers.The system will bedelivered through a standard Web browser and therefore users with special needs should

    find that data is as accessible as that on regular Web pages. If the subject is of a highly

    visual nature, it is not always appropriate to provide non-visual alternatives for learners

    with visual impairment. However, modern text recognition technologies and speech

    synthesis are available that are compatible with Web browsers.

    As a result, Sclater and Howie (2003) identify requirements that VLE users expect when

    using the CAA:

    Be able to view all available assessments;

    Be granted access to earlier results;

    Receive email reminders about times, dates and locations of assessments in

    advance;

    Be allowed to continue from the point where they left unfinished assessments;

    Points for each question should be clearly identified;

    All questions should be answered before completing the assessment;

    Results should be computed and displayed instantaneously;

    Grades should be allowed to be combined with marks from other assessments;

    Grades can be compared with group averages or ranked;

    Assessment questions, responses with correct answers can be printed out;

  • 8/2/2019 Online Report

    36/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 36

    Operating instructions are clear and easy accessible from all locations within the

    VLE;

    Time taken, and if applicable, times remaining are clearly visible;

    Students can revisit and adjust answers to questions before submission;

    Regular saving of responses throughout assessments rather that at the end of the

    test;

    Submissions which are late require a reason and automatically provides an email

    warning.

    System administrator requirements

    The VLE will be accessible via a typical free to download platform-independent Web

    browser software without the use of additional plug-ins (software that extends the

    capabilities of the Web browser). It also will be backwardly compatible so that learning

    material will be available for those with earlier versions of Web browsers (Dalziel &

    Gazzard, 1999). All assessments and personal data will only be made available to users

    who have the relevant access permissions, and these users will be assured that no

    unauthorised persons will be able to access personal details held on a server.Time-

    consuming and convoluted login procedures would discourage adoption of the system,therefore a username and password should be a sufficient level of security considering

    the sensitivity of personal details that will be stored on the server. A reliable network is

    crucial for the effective use of online assessment and a VLE. It would be appropriate to

    allow records to be updated continuously during the session rather than once at the end,

    this prevents assessment data being lost if the network fails, and would allow the student

    to continue from where the session ended.

    Sclater and Howie (2003) also state that systems administratorsexpect that new software

    should have the following requirements:

    Easy to install on the server, with a clear installation guide;

    Multi-platform and operate on both Windows and UNIX operating systems;

  • 8/2/2019 Online Report

    37/177

  • 8/2/2019 Online Report

    38/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 38

    Control the number of attempts taken at the assessment;

    Can inspect results of all assessments taken by a student;

    View mean and average assessment scores;

    Time taken to complete each assessment can be seen for both individuals and

    overall average for all students;

    Allow the results tables to be ordered in a variety of ways;

    Marks displayable as grades and percentages;

    All attempts at assessments can be viewed;

    Individual and overall group responses to questions can be viewed;

    Results can be viewed per group to identify weak areas;

    Grades can be computed over a series of assessments;

    Results of different groups can be compared;

    Group performance in different subject areas can be compared;

    Results can be exported to statistical analysis software (e.g. SPSS) or

    spreadsheets;

    Allow a hard copy of the results table to be printed.

    System requirements

    Ellaway, Dewhurst, and Cumming (2003) describe that the database is an essential tool

    for a complex software application. It is at the centre of the VLE and is the basis from

    which everything is built upon. Therefore, it is essential to carefully design the

    architecture, to ensure fast and efficient access, processing, data retrieval, and robust

    security. The database must be flexible to be modified to meet changing conditions and

    be capable to be integrated with other systems to communicate information, such as

    student records systems, and possibly with other VLEs. The database requires a highlevel of normalisation to facilitate this.

    In addition, Becta ICT Research (n.d.) suggest that most VLEs utilise Web-based

    architectures and are often accessed at the client level via a Web browser served over the

    local area network (LAN); this reduces the necessity for users to have high-bandwidth

  • 8/2/2019 Online Report

    39/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 39

    Internet connections. LAN infrastructures are often robust and reliable which is crucial

    for a VLE if the local implementation is to be successful. Some VLEs require specific

    versions of browsers to be used; this is restrictive as it may limit users wishing to access

    the VLE from home, although it is worth noting that the majority of Web users operate

    Internet Explorer. Some particular type of course material, especially multimedia may

    also require a plug-in to be downloaded from the software distributor or a media player to

    view the content.

    Becta ICT Research (n.d.) indicates that a VLE should include most of the following

    features:

    The VLE should be available to students from wherever they can obtain access to

    a computer connected to the Internet;

    Information about the syllabus, student and (non personal) staff information,

    course related notices, and associated documentation, together with clear

    descriptions to facilitate location and retrieval of information;

    An intuitive collection of synchronous and asynchronous communication tools,

    including threaded message discussion forums and virtual chat rooms;

    Group areas that include collaborative tools, which enable sharing of documents

    and notes; Designated areas for external Internet links, with include brief descriptions to

    prevent users from having to follow the links to discover their content;

    A variety of tools for assessing learning, ranging from automated multiple-choice

    questions to a virtual space for depositing exercises or assignments;

    A range of organisational tools for the student, including a calendar and a simple

    Web page generator to create a home page.

    From the comparison of existing VLEs the following technical requirements are

    recommended. This study was necessary to ensure that the developed system would be a

    feasible option for current students.

  • 8/2/2019 Online Report

    40/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 40

    Client side computers requirements

    Processor (CPU) from 233 MHZ with minimum 64 MB RAM (corresponding,

    analogous requirements for Apple Macintosh).

    Minimum 56kbps modem. Screen resolution of minimum 800x600 pixels.

    Operating system:

    o Windows 98, ME, 2000, XP, Linux, Unix.

    o Apple Macintosh OS 8 or later.

    Internet Explorer: from version 5.0 or Netscape: from version 7.01, Safari,

    Mozilla etc.

    Web server requirements

    Unix or Windows system.

    Suitable Web server.

    A SQL Database.

    Internet connection upload speed of 1Mbps.

    Substantial (preferably hardware) firewall.

    Server class computer a higher specification would allow a faster response to

    client requests.

    System specification

    Once all of the requirements have been gathered the formal specification may be

    constructed. This specification is moderately basic but encapsulates most of the user

    requirements. A test suite can then be designed at a later date to measure if the softwaremeets the specification and user requirements, thus producing a high quality solution.

    The final software should allow:

    The creation of a new user account;

    Secure login facility using a password and username;

  • 8/2/2019 Online Report

    41/177

    Computer Assisted Assessment within a Virtual Learning EnvironmentG6DDIS MSc Dissertation

    Author: Jonathan Marshall ID:1937840 41

    Access to learning resources;

    Online assessments;

    View previous and current assessment results;

    Asynchronous (email) communication with the tutor;

    Synchronous (discussion forum) with the tutor and other users;

    View and edit personal information;

    View timetable for tutors availability;

    Scalability to an estimated hundreds possibly thousands of accesses per week;


Recommended