V01 2014-04-28 Document compiled from Google Drive drafts and documents sent from CBS. (BHAM)
V02 2014-04-30 Internal review completed. (TALK)
V03 2014-05-01 Review comments addressed. (BHAM)
V04 2014-05-01 Review comments addressed. (CBS)
V05 2014-05-01 Screen shots updated (BHAM)
V06 2014-05-05 Final version – submitted to EC
Impressum
Full project title: Next Generation Teaching, Education and Learning for Life
Grant Agreement No: 285114
Workpackage Leader: Susan Bull, BHAM
Project Co-ordinator: Harald Mayer, JRS
Scientific Project Leader: Peter Reimann, MTO
Acknowledgement: The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 258114.
Disclaimer: This document does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of its content.
This document contains material, which is copyright of certain NEXT-TELL consortium parties and may not be reproduced or copied without permission. The information contained in this document is the proprietary confidential information of certain NEXT-TELL consortium parties and may not be disclosed except in accordance with the consortium agreement.
The commercial use of any information in this document may require a licence from the proprietor of that information.
Neither the NEXT-TELL consortium as a whole, nor a certain party of the NEXT-TELL consortium warrant that the information contained in this document is capable of use, nor that use of the information is free from risk, and does not accept any liability for loss or damage suffered by any person using the information.
D4.7 Student Model Tools Code Release and Documentation
1.1 Open Learner Model .................................................................................................................................. 1
2.1 Purpose of this Document ......................................................................................................................... 2
2.2 Scope of this Document ............................................................................................................................. 2
2.3 Status of this Document ............................................................................................................................. 2
2.4 Related Documents .................................................................................................................................... 2
3 OLM: The Next-TELL Open Learner Model .................................................................................................. 3
3.1 About ......................................................................................................................................................... 3
3.3 High Level Requirements ........................................................................................................................... 9 3.3.1 Visualisation of the Learner Model ............................................................................................................................................ 9 3.3.2 Information Content of the Learner Model ................................................................................................................................ 9 3.3.3 Updating the Learner Model ..................................................................................................................................................... 9
4 OLM: Functionality and Interface Design .................................................................................................. 11
4.1 Web Pages and Site Map ......................................................................................................................... 11
4.8 Customisation and Localisation ............................................................................................................... 22
4.9 User Manual ............................................................................................................................................. 23
D4.7 Student Model Tools Code Release and Documentation
5.6 Database - connectivity and design. Data storage structures.................................................................. 31
5.7 Session Variable Specification .................................................................................................................. 32 5.7.1 General .................................................................................................................................................................................... 32 5.7.2 Information Focus .................................................................................................................................................................... 33 5.7.3 Display ..................................................................................................................................................................................... 34 5.7.4 External Communication ......................................................................................................................................................... 34
5.8 API specification ....................................................................................................................................... 35
5.9 Authentication and Authorisation ........................................................................................................... 35
5.12 Connectivity with Google ......................................................................................................................... 36
6 OLM: Summary and Future Work ............................................................................................................. 37
Appendix 1: OLM: High Level Specifications ................................................................................................... 50
Visualising the Learner Model ............................................................................................................................ 50
D4.7 Student Model Tools Code Release and Documentation
Information Content of the Learner Model ....................................................................................................... 52
Updating the Learner Model .............................................................................................................................. 53
Appendix 2: OLM: Localisation Language Definition ....................................................................................... 55
Appendix 3: OLM: User Manuals .................................................................................................................... 72
2 Features and the Homepage ........................................................................................................................... 73
3 Open Learner Model ....................................................................................................................................... 75
5.2 Google Spreadsheet ..................................................................................................................................... 85
2 Features and the Homepage ........................................................................................................................... 90
3 Open Learner Model ....................................................................................................................................... 92
Group Database ............................................................................................................................................... 120
People and Roles .............................................................................................................................................. 136
10.7 RGFA Localization ................................................................................................................................... 141 10.7.1 Admin page ........................................................................................................................................................................... 141 10.7.2 Add a new language .............................................................................................................................................................. 142 10.7.3 Update translations ............................................................................................................................................................... 142 10.7.4 Activate / de-activate a language ......................................................................................................................................... 143 10.7.5 View RGFA in another language ............................................................................................................................................ 144
CoNeTo API Description ................................................................................................................................... 145
Appendix 8: RGFA and CoNeTo: User Manual for RGFA ............................................................................... 148
2. Suggestions for Teachers ............................................................................................................................. 148
3. Suggestions for Students .............................................................................................................................. 149
4. Creating a Repertory Grid Exercise using the RGFA Application .................................................................. 149
5. RGFA: User Manual ...................................................................................................................................... 150
This document contains a wealth of information about the design and implementation of the Next-TELL open learner model. Information is included about the final specification (Section 3), the interfaces and features (Section 4), its implementation and technical design (Section 5) and also a summary of its strengths and areas of improvement (Section 6). Several key appendices are attached to this report including user manuals for teacher and students (Appendix 3). Fundamentally, all relevant information is included in the report for those wishing to do further development work with the tool and to extend, tailor and refine its functionality. The source code may be downloaded as follows:
OLM Development Code (.zip) http://eeevle.bham.ac.uk/nexttell/openlearnermodelcode.zip
This document provides information about the design and implementation of the NEXT-TELL communication and negotiaton tool (CoNeTo). It includes description of the technical integration of CoNeTo with NEXT-TELL’s Repertory Grids for Formative Assessment (RGFA) and Open Learner Model (OLM) software applications. Appendix 6 provides the pedagogical scenario for CoNeTo & RGFA in terms of the nextGRID package. Appendix 7 contains deployment instructions and CoNeTo API description with Appendix 9 containing the user manual for CoNeTo.
This document provides information about the design and implementation of NEXT-TELL’s generic purpose knowledge diagnostic tool, Repertory Grids for Formative Assessment and describes the technical integration with other NEXT-TELL applications of ECAAD, OLM & CoNeTo. Appendix 6 provides a description of the pedagiogical scenario for nextGRID featuring RGFA. Appendix 7 provides deployment instructions and the user manual is provided in Apepndix 9.
This document provides updates on the technical implementation of the student modelling and negotiation tools, as at Month 44. Namely we consider the status of development of the OLM, CoNeTo, and RGFA. Updates to ProNIFA are reported in D2.7. Three tasks are reported here: T4.1 open learner model architecture, T4.2 Representing and visualising learner models, and T4.3 Communicating and negotiating learner models.
2.2 Scope of this Document
This document covers the following aspects of the work package 4, month 44 deliverable:
Learner model data sources
Learner model visualisation
Communication and negotiation tool
Repertory grid for formative assessment
2.3 Status of this Document
This is the final version of D4.7
2.4 Related Documents
Before reading this document it is recommended to be familiar with the following documents:
D4.1 Methods and Specifications for the Student Model V1
D4.2 Student Model Tools R1
D4.3 Methods and Specifications for the Student Model V2
D4.4 Student Model Tools R2
D4.5 Methods and Specifications for the Student Model V3
D4.6 Student Model Tools R3
Documents which complement this document include:
D2.7 ECAAD Tools Final Release
D3.7 Activity Capturing Tools Final Release
D4.7 Student Model Tools Code Release and Documentation
A learner model is a representation of the current state of a student’s learning. It may for example be a model of the learner’s current competencies or knowledge state and typically built from inferences about student understanding, updated dynamically, and may be used in a very specific context or activity, or over a period of time. Environments with an open learner model (OLM) allow the content of the underlying learner model to be visualised in a user-interpretable way for educationally relevant stakeholders (Bull and Kay, 2010). They can also be made available independent from any wider system, placing a greater emphasis on facilitating metacognitive processes (Bull and Kay, 2013). The Next-TELL Open Learner Model is exactly this, an independent open learner model (IOLM) designed specifically to visualise the underlying learner model for students and teachers using eight visualisation techniques (skill meters, tables, histograms, smiley faces, word clouds, radar plots, treemaps, and network diagrams). See Figure 1. The aims of visualising the learner model in such a way include raising learner awareness, prompting reflection on understanding and learning; acting as a starting point for planning; facilitating independent learning; encouraging collaborative interaction and problem-solving; and helping learners to take greater responsibility for their learning (Bull et al., 2013).
Figure 1: Next-TELL OLM Visualisations
D4.7 Student Model Tools Code Release and Documentation
In the remainder of this section we provide an up to date specification for the Next-TELL open learner model in terms of the SMILI Framework (Bull & Kay, 2007), and initial specifications from D4.1. In the remainder of this report we describe the implementation of the Next-TELL open learner model in terms of its functionality, interfaces and internal workings. Fundamentally this report describes the final release of the Next-TELL open learner model at Month 44 and gives details that may be used by those who wish to use the tool’s code for further development and future research. See D4.1 - D4.6 for more background information.
3.2 SMILI Specification
The Student Models that Invite the Learner In (SMILI) Framework allows OLM developers to describe their OLM in a consistent way to facilitate comparison and ensure that most key features are reported - it is not a recommendation of what should be included in an OLM (Bull & Kay, 2007). Building on the work of D4.1, D4.3, and D4.5 we describe here the specification of the Next-TELL open learner model in terms of the relevant categories of the SMILI Framework. The final implementation of the Next-TELL OLM is targeted at students and teachers.
KEY: ✓implemented T if teacher allows n/a not applicable (✓) can do if required
Table 1: SMILI – extent of the model accessible.
1. Extent of the model accessible Student Teacher
Complete
Partial
✓
✓
Competency
Knowledge level
Knowledge
Difficulties
Misconceptions
✓
(✓)
✓
(✓)
Learning issues
Social issues
Preferences
Other users’ LM (individual)
Other users’ LM (group)
✓
✓
✓
✓
✓
✓
✓
✓
Students can see all information in their learner models, and teachers can see all information in the learner models about their students that relates to their own teaching. This is complete access. This is updated from the initial specification which envisaged partial access for students.
The Next-TELL OLM models competencies (which can, instead, be configured by teachers as knowledge levels). Additional items of textual feedback/comments from students, peers and teachers can be collated alongside the learner model - these are categorised as strengths, guidance, areas of difficulty and suggestions. The main competency data is numerical and is transformed by the modelling process, whereas the textual information captured does not contribute to the modelling process, and is viewed unchanged. Thus, it is not strictly learner model data, but may help users to interpret the model inferences.
Learning issues, social issues, and preferences may be treated by the Next-TELL OLM in the same manner as competencies, and may be modelled in the same way as items configured in the competency framework. They
D4.7 Student Model Tools Code Release and Documentation
do not receive their own facet within the learner model, as the differentiation from the approach used to model competencies is minimal and user configured. The requirement permitting flexible definition of items to be modelled has facilitated this. Teachers may see students’ learner models on an individual or grouped basis. Students always see information from their own student model, but not those of other users.
Table 2: SMILI – presentation.
2. Presentation Student Teacher
Textual (i.e...)
Graphical (i.e...)
Other (i.e...)
✓
✓
✓
✓
Overview
Targeted details
All details
✓
✓
✓
✓
Support to use
Students and teachers have access to the same range of visualisations, which comprise both textual and graphical representations of the learner model (see Table 3 below). The range of visualisations may be customised by each user and all visualisations are able to display high level as well as more detailed breakdown of open learner model information, and the first six visualisations support comparison between different data sources.
Table 3: open learner model visualisations.
Visualisation Graphical Textual Student Teacher
Skill Meter
Table
Smiley Face
Word Cloud
Histogram
Radar Plot
Interactive Treemap
Network
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
The needs of the project have not required more sophisticated visualisation techniques, or for the model to be opened in forms other than textual/graphical (e.g. audio, multimedia, haptic). Users have been supported in their use of the OLM throughout the project, and this will continue until the end of August 2014. Training materials, user manuals and help sections exist to assist with teachers’ and students’ understanding and use of the tool.
Table 4: SMILI – similarity to the underlying representation.
3. Similarity to the underlying representation Student Teacher
Identical
Similar
Different
✓
✓
D4.7 Student Model Tools Code Release and Documentation
The visualisation of the learner model differs from the underlying representation in the database to enable the user to understand its meaning. The information may be accessed flexibly and so is combined from different sources at the point of access. This is to display information relevant to the stakeholder’s current informational needs, in an appropriate format and to an appropriate level of abstraction. The underlying representation is flexible enough to allow for future representations of its content tailored to different pedagogical requirements.
Table 5: SMILI – access to uncertainty.
4. Access to uncertainty Student Teacher
Complete
Partial
None
n/a
n/a
n/a
n/a
n/a
n/a
The OLM does not contain representations of uncertainty. However, it offers partial information about the possibility of uncertainty though the description of the modelling process. This states the extent to which items of evidence have contributed to the state of the model for any given element of visualisation, their weightings and the sources of information, from which the reliability of the information may be determined by the user. Certainty factors are not implemented and so this cannot be considered as access.
Table 6: SMILI – role of time.
5. Role of time Student Teacher
Previous
Current
Future
✓
✓
The system shows current information from the learner model only. This is an amendment from the initial specification that specified each stakeholder will have access to “past and predicted future states”. Other information (e.g. how the model has changed over time) is omitted to increase the focus of the tool, as this is key to its uptake and current role in the project.
Table 7: SMILI – access method.
6. Access method Student Teacher
Inspectable/view
Co-operative
Sys persuade user
User persuade sys
User add evidence
Negotiated
Editable
✓
✓T
✓
✓
✓
Students and teachers can inspect all learner model data that they have privilege to view. Teachers and other systems (via the API) may always add evidence. Student and their peers may add evidence if the teacher has enabled this feature. Furthermore, teachers may directly edit and delete specific pieces of evidence from the learner model.
D4.7 Student Model Tools Code Release and Documentation
Both the student and teacher will be able to initiate access to the OLM information at any point (self-initiated). Further to this, each time the learner model is updated, the ‘updates’ mechanism prompts the student or teacher that the learner model has updated, and provides a link to where this information may be viewed (system initiated). (I.e. this process is triggered by {automated systems, peers, teachers} for the student, and by {automated systems, students, students’ peers} for the teacher). The student or teacher cannot directly prompt each other to view the learner model content.
Table 9: SMILI – access to sources of input.
8. Access to sources of input Student Teacher
Complete
Partial
None
✓
✓
Aggregated
System
Student
Peer
Teacher
Other program
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Both teachers and students may see all underlying evidence that contributes to a student’s learner model that is contained in the Next-TELL OLM, and is therefore categorised as complete access. However, as information comes from a variety of sources, details of evidence from third party systems may not be present. Furthermore, the current implementation does not distinguish automated data from third party systems in the visualisations - these are combined in the visualisations as ‘automated data’. Therefore, while complete access is available, it must be specifically requested. The teacher has control to edit and disregard specific pieces of information from sources of input.
Evidence from each source is available to each stakeholder in both an aggregated form and its constituent parts. Students and teacher may also see the underlying evidence layer, and also a description of the modelling process that makes explicit the link between the evidence and the open learner model.
Table 10: SMILI – control over accessibility (to others).
9. Control over accessibility (to others) Student Teacher
Complete
Partial
None
✓
✓
D4.7 Student Model Tools Code Release and Documentation
9. Control over accessibility (to others) Student Teacher
System
Student
Peer
Teacher
Other program
n/a
n/a
The teacher may always see all content in their students’ learner models. In the present implementation, a teacher may not see student data that is associated with another teacher’s provision of learning, even if they share a student. A given teacher may only see aspects of the student’s learning that are relevant to their educational relationship with the student.
The specification initially allowed students to share learner model content with each other, if this practice was permitted by the teacher. This facility has become less relevant over the course of the project, and less coherent with the current project requirements and packaging approach being undertaken, and so this facility is omitted from the final release.
The Next-TELL OLM will always have access to all the student data it contains. Other Next-TELL tools, such as ProNIFA, will have access to a student’s information if the access Next-TELL security/authentication/authorisation requirements are met and the parameters are correctly synchronised with those specified in the configuration tool.
Table 11: SMILI – awareness of the effect of model on personalisation.
10. Awareness of the effect of model on personalisation Student Teacher
Complete
Partial
None
n/a
n/a
n/a
n/a
n/a
n/a
There is no change to this specification; the Next-TELL OLM does not itself contain an adaptive teaching component - the user’s awareness of the effect of the learner model on personalisation is not relevant in the traditional sense. However, the data held in the learner model may be connected to through the API. In this way the Next-TELL OLM can form a data source for other systems who wish to use this information for adaptation or personalisation purposes. For example ProNIFA could use the learner model’s competency data and could send it back to OpenSim for adaptation purposes, or for the teacher to adapt the course of interaction. This, however, is not part of the specific role of the OLM.
Table 12: SMILI – flexibility of access.
11. Flexibility of access Student Teacher
Complete
Partial
None
✓
✓
The OLM continues to support individual differences with regard to information access, for example through the use of isomorphic visualisations of the learner model content, and customisation of the visualisation set. Teachers and students both have the same privileges and methods available to them. This is an update from the initial specification that states students would only have partial access.
D4.7 Student Model Tools Code Release and Documentation
Can be used as central focus or additional support ✓ ✓
The OLM is a key tool within the Next-TELL project for each of the above stakeholders, and binds many aims of the project together. It would be best used in conjunction with multiple other Next-TELL tools or third party systems, but may also be used on its own.
Table 14: SMILI – evaluation (evidence).
I. Evaluation (evidence) Student Teacher
Deployment in real settings ✓ ✓
Students and teachers have been involved in use of the OLM in classrooms throughout the project. The OLM will be made available for continued to be deployed in real usage settings beyond the end of the project.
3.3 High Level Requirements
Further to the SMILI framework specification, throughout the project we have defined and adhered to a series of high level specifications that have influenced our design and implementation decisions. These are categorised as either concerning visualisation of the learner model, the information content of the learner model or the the process of updating the learner model. Appendix 1 (Section 0) contains a full specification of these and identifies the level of compliance and revisions that have taken place, further to those detailed in D4.5. These updates are namely:
3.3.1 Visualisation of the Learner Model
D4.1:1 Visualisations can break information down by data source, in addition to presenting an overview.
D4.1:2 Both the student and their teacher have access to all information that is in a student’s learner model. Visualisations can be configured, but all relevant data in the evidence layer contributes to these.
D4.1:4 Quick navigation (system remembers the context of use) is extended to the add evidence pages.
D4.1:6 The system shows current information from the learner model only. Other information (e.g. how the model has changed over time) is omitted to increase the focus of the tool, as this is key to its uptake and current role in the project. The initial rationale of this requirement is still supported by this amendment.
D4.1:11 Speed is increased by only one database connection being made per request. It is acknowledged that the active learner modelling approach could have impact on load times with large datasets.
3.3.2 Information Content of the Learner Model
D4.1:12 Everything in the database is available for inspection by both teacher and student.
D4.1:14 Information in the repository is unprocessed evidence and learner modelling happens at the point at which the model is opened to the student or teacher. The parameters given to the modelling algorithm at the time determine how course or fine grained the OLM visualisation is.
D4.1:15 One piece of information may update several components of the learner model. The same information may potentially be used to inform other modelling algorithms, as is supported by retaining a more ecological approach to learner modelling (McCalla, 2004).
3.3.3 Updating the Learner Model
D4.1:21 Information from key stakeholders can also be added through the API.
D4.7 Student Model Tools Code Release and Documentation
The final release of the Next-TELL open learner model is a web-based resource that can be accessed in any location by students and teachers. It supports a variety of platforms including desktop computers and tablets. Each user has a password protected user account. In this section we consider the functionality available to students and teachers, open learner model visualisations and the final interface design. Details about the implementation and inner workings of the system are covered in Section 5.
4.1 Web Pages and Site Map
Firstly, we summarise the available functions in the final release in terms of the web-pages available. The following ‘pseudo-site map’ describes the OLM in terms of it functioning as a website. Sections 4.2 - 4.7 describe each part in greater detail.
Header
Homepage o User Manual
Open Learner Model o Information Filters o 8 Visualisations o List of Evidence and Textual Information
Add Evidence, Guidance, Strengths, Suggestions, Difficulties o associated with an activity (teacher, student, peer assessments) o not associated with an activity (teacher, student, peer assessments) o for multiple students using a Google spreadsheet (teacher assessment)
Updates
Discussion
Preferences
Language
Log out
4.2 General
4.2.1 Header
The header bar (Figure 2) is present on every page in the tool and allows persistent navigation to each of the key screens of the web tool, together with a menu which gives access to the customisation facilities and the ability to log out. The number of updates and discussion items are also shown here. The “i” link in the menu opens up help sections in each page.
Icons are used to allow features to be quickly identified within the system. Icons are used consistently throughout to identify functions, visualisations and information types. They fall into the following five categories (Figure 3):
1. Information Categories
Groups
Students
Competencies
Activities
2. General Use
Open Filters
Close Filters
Refresh
Reset
Edit
3. Navigation
Home
Open Learner Model (Dashboard)
Add Evidence
Updates and Notifications
Discussion
Configure Preferences
Log Out
Information and Help
List of Evidence
Learner Modelling Process
Configure Information Sources
Tags
4. Visualisations
Skill Meters
Table
Smiley Faces
Histogram
Word Cloud
Radar Plot
Tree Map
Network
5. Languages
English
French
German
Norwegian
Figure 3: iconography.
4.3 Homepage
Figure 4: homepage (student).
D4.7 Student Model Tools Code Release and Documentation
The homepage is the entry page (Figure 4). It provides links to each of the main parts of the system, to the user manual page, and also displays a word cloud highlighting the concepts involved in the use of the tool. For the student, hyperlinks to the teacher’s course materials are available. The homepage may be returned to at any point
4.4 Open Learner Model
The open learner model page (Figure 5) is the same for both student and teacher. The visualisations are shown in the tabbed structure in the main part of the screen (see Sections 4.4.2 to 4.4.9). The information shown is broken down by groups, students, competencies and activities. The top part of the screen shows a summary of the information displayed (information sources, active filters, and active tags). The left hand part of the screen allows the student and teacher to narrow down the information presented by adding filters. (See Section 4.4.10) The filters section may be closed to free up more space for visualisation on the screen. Icons are used for triggering further functions on the screen (reset, refresh, list evidence, display learner model process) (see also Sections 4.4.11 and 4.4.12).
Figure 5: open learner model.
4.4.1 Visualisations: Introduction
The open learner model visualises the content of the learner model using 8 different visualisations: skill meter, table, smiley faces, histogram, word cloud, radar plot, treemap and network (see Sections 4.4.2 to 4.4.9). Each visualisation type shows information in the learner model per group, per student, per competency and per activity. The first 6 visualisations also have the ability to break the information down by its source (teacher, student, peer, or third party automated systems). Both teacher and student have access to the same visualisations. The visualisations present can be customised on the preferences page (Figure 28). The preferences page also allows users to assign colours to different categories of information for easy identification in the visualisations. The first four visualisations contain icons that link to pages displaying the list of evidence and learner model process for the specific part of the visualisation. (For all other visualisations there is a general link at the top of the page, as these do not easily lend themselves to the inclusion of additional icons in their visualisations.)
D4.7 Student Model Tools Code Release and Documentation
The proportion of colour in each bar is used to represent the extent to which the learner(s) are competent in this area (Figure 6). Structure is represented (where present) using indentation. When enabled, each skill meter can show information from different sources showing a group of coloured skill meters, in place of each single one (Figure 7).
4.4.3 Visualisations: Table
Figure 8: table visualisation.
Figure 9: table visualisation (sources).
For each competency (or other data field) a dot is placed in the column that best represents the strength of a competency, quantised into five categories from very weak to very strong (Figure 8). Structure is shown using indentation. When showing information from different sources each table item splits into the number of sources and colour is used to differentiate the sources (Figure 9).
4.4.4 Visualisations: Smiley Faces
Figure 10: smiley face visualisation.
Figure 11: smiley face visualisation (sources).
Smiley faces are represented on a scale of confused/little or no competence to happy/strong competence (Figure 10). There are five different faces. The level of indentation is used to represent the hierarchical structure of the information. When broken down into sources, a face is present to represent each data source (Figure 11). The colour identifies to which source the information belongs.
D4.7 Student Model Tools Code Release and Documentation
Each competency (or other data field) is quantised into one of five categories. Competencies that have stronger evidence are placed on the right hand side of the histogram, and competencies that have little evidence on the left. (Figure 12). If there is no input for a particular item in the histogram, this is omitted and placed in a list underneath. When information is to be broken down by source, a histogram is generated for each source, identified by colour (Figure 13).
4.4.6 Visualisations: Word Cloud
Figure 14: word cloud visualisation.
Figure 15: word cloud visualisation (sources).
D4.7 Student Model Tools Code Release and Documentation
The information is distributed between two word clouds. The cloud on the left shows stronger competencies, whilst the cloud on the right shows weaker ones (Figure 14). If no information is available, the item is placed in a list under the visualisation. The size of a word indicates the extent to which a competency is held or not held. When broken down by source a word cloud set is present for each source, and may be identified by the colour of the text in the background (Figure 15).
4.4.7 Visualisations: Radar Plot
Figure 16: radar plot visualisation.
Figure 17: radar plot visualisation (sources).
One competency (or other data field) is displayed per axis. The further away from the centre, the higher the competency. (Figure 16.) The order of competencies is the same as per the competency structure (clockwise). When broken down by source, data from different types are displayed together on the same axis (Figure 17).
4.4.8 Visualisations: Tree Map
Figure 18: treemap visualisation.
The size of the area is representative of the strength of the labelled competency. Left clicking on an area reveals the sub- competencies. Right clicking returns the tree to the super- competencies. The provided button allows this also for iPads and touchscreen devices. (See Figure 18.)
D4.7 Student Model Tools Code Release and Documentation
The size of the node and brightness of the colour green both represent the strength of the competency on the label. The lines between nodes show the relationships between sub- and super- competencies. All competencies are linked to the black node. Clicking on a node will hide its sub-competencies, improving readability. The nodes are “force-directed” and may be moved. (See Figure 19.)
4.4.10 Filters
Figure 20: filter options.
The information filters may be used to change the scope of the information displayed (e.g. for a specific student, group, competency, activity, information source, or evidence tag) – see Figure 20 for overview. Any combination of these filters may be applied (“break down by source” will override summary in the “information sources” section). The method supports both searching and browsing methods of interaction. One data item may be selected from each of the filter sections and the options available in the remaining filter sections automatically update, so that only possible options are displayed (e.g. if a teacher selects a group, then only
D4.7 Student Model Tools Code Release and Documentation
the competencies for that group are displayed). Any combination of information sources may be configured at any point. Selecting the option to break the information down by source will show the information categories separately in the first six visualisations, otherwise the data will be combined. A summary of which filters are currently active is shown in the top centre of the open learner model screen (see Figure 5).
4.4.11 Evidence List
Two facilities are provided for the user to gain more information about their learner model. The first is the evidence list. When the evidence icon is selected a modal dialogue is displayed with a list of all contributing evidence (Figure 21). The evidence is ordered by its age, with the newest items appearing first. The top part of the page summarises what evidence is displayed and a searchable table is used for the content. The table displays information that has contributed to the learner model, in addition to evidence collated alongside the model, including strengths, guidance, difficulties, and suggestions. This list of information may be downloaded and stored as a .pdf or .csv file, and there is an option to download this in an anonymised form. The teacher has permissions to edit the evidence.
Figure 21: evidence list.
4.4.12 Learner Model Process
The second facility is a description of the learner model process (Figure 22). Again, this is a modal dialogue which is opened by clicking the process icon in one of the visualisations, or at the top of the page. This starts at the summary of the information and allows the user to drill down through the modelling process to the evidence layer, using the “+” and “-” icons to open and close sections. This provides a link between the evidence and the open representation of the learner model and information about the learner model calculation.
D4.7 Student Model Tools Code Release and Documentation
Evidence to inform the modelling process may be added automatically using the API or manually using several interfaces with the open learner model software (Figure 23, Figure 24). When evidence is entered manually by a student, this is considered to be a self-assessment. If it is about another student it is considered a peer-assessment. When teachers enter information this appears as a teacher assessment. The teacher may do this for one student at a time using the embedded web-form (as used by students), or the teacher may use a Google Spreadsheet that the software generates to do this for several students together (Figure 25).
The area of the model to be updated is specified using the same filters as for the main open learner model screen (left of Figure 23). The add information screen automatically remembers the settings of these filters when switching between screens and the screen specifies which parameters should be selected in order to add information. When adding information this may be done with reference to a competency as part of a specific activity, or it may refer to a competency generally. Minimally the following filters need to be selected to add information (Table 15).
Table 15: ways of adding evidence manually.
Filter parameters to load add evidence screen Add evidence screen shows Model is updated
student, group, activity competencies associated with an activity
student, group competencies not associated with an activity
group, activity* students, competencies associated with an activity
* Teacher only, via specially generated Google Spreadsheet
When enough context is provided, the user has access to one of the following screens for adding information (Figure 24, Figure 25). The student or teacher may add information for any part of the form. A scale is provided for adding information that will be part of the learner model (the number of points on the scale, the labels, and the use of half scale points may all be set in the preferences page). Alongside the model additional information may be collated to help contextualise the information presented in the model. This may be a description of learner strengths, areas of guidance (teacher entered), areas of difficulty (student entered) or suggestions for the students peer (student entered as part of a peer assessment). Additionally the student or teacher may also tag the piece of evidence or add a URL to an associated artefact, further supporting the formative assessment description given. The Google Spreadsheet input is only available for the teacher. A copy of the spreadsheet is automatically saved to the teacher’s Google Drive for future reference.
Figure 24: add information screen (teacher).
D4.7 Student Model Tools Code Release and Documentation
Figure 25: add information screen (Google Spreadsheet).
4.6 Updates
The updates page (Figure 26) contains one entry for each competency, one entry for each activity, and one entry for each group. Each time new evidence is added to the learner model, or existing evidence is edited, the entry is moved to the top of the page and the changes are highlighted in red. Once the update has been viewed the entry will no longer appear in red, but will remain in its current position. Each entry is a hyperlink that will take the teacher or student to that specific place in the learner model where the content of the update/learner model entry may be viewed. The number of unread entries is displayed in the header bar. Information within the Updates are represented as Evidence, Students and Sources. When a new item is generated within the Updates tab important information is highlighted in red and is bold, for example Figure 26 shows “Learner Model: 1/3” This means that new evidence has been added increasing the count to the student’s learner models and increasing the total number to three.
Figure 26: updates page.
D4.7 Student Model Tools Code Release and Documentation
Groups of students and teachers may participate in discussion about specific competencies, encountered within the context of a specific group (Figure 27). The screen is split into four sections. The top part of the screen shows the specific area that the displayed discussion relates to and gives links to refresh and reset functionalities. The left hand portion of the screen is used to select discussions, and states the number of posts, and the number of unread posts in each case. There is one discussion for each competency for each group. The centre portion of the screen displays the discussion. Speech bubbles are used to present the messages and the colour of these is specific to each actor in the discussion (one colour for teachers, one colour for peers, one colour for students). The colours may be changed on the preference page and are the same as the different sources of information in the learner model. Posts from the user originate from the right and posts from others from the left. Discussion happens in real time and is persisted even when the window is closed - the discussion may also therefore happen asynchronously (users not all online at the same time). The right hand portion of the screen displays the learner model, for use alongside the discussion. Students can only see their own learner model, and the teacher may see the learner model of each of their students.
Figure 27: discussion screen.
4.8 Customisation and Localisation
All textual statements displayed on the screen are in the local language of the user. (See Appendix 2 (Section 0) for a definition of these items). The language can be changed using the menu on the top right of the webpages, without needing to go into the preferences page. Languages are identified by the flag of the respective country that the language originates from. The Next-TELL OLM is available in English (UK), German, Norwegian, and French.
D4.7 Student Model Tools Code Release and Documentation
In addition, the preferences page (Figure 28) allows the following items to be configured:
Which visualisations are shown
For adding evidence (teacher only): - number of points on the assessment scale - whether the scale uses numbers or the labels “Strong” and “Weak” on the axis - whether the scale allows half-point precision
Colours for the different information categories: teacher, student, peer, automated data.
Link to the teacher’s course materials (teacher only)
Colours for the different information sections: groups, students, competencies, activities, general.
4.9 User Manual
User manuals are available for the teacher interface, student interface and for the configuration tool (reported in D3.7). These are included as Appendix 3 (Section 0).
D4.7 Student Model Tools Code Release and Documentation
The following is a summary of the development environment and technical realisation of OLM software (Table 16). Two dependencies exist for the OLM software to run fully (Table 17). N.B. the two dependencies are released separately to the OLM development code (as detailed in Section 5.2)
Table 16: development setup
Development Environment Eclipse Helios Service Release 2
Development Platform Windows 7 Enterprise, Service Pack 1
Server Apache Tomcat V7.0.14
Database Apache Derby V10.8.1.2
Front End Languages HTML, JavaScript/JQuery, CSS, JSP
Supported Browsers Internet Explorer 11, Mozilla Firefox 27.0.1, Google Chrome 33.0.1750.154, Safari 5.1.7, Opera 20.0
Back End Languages Java 1.6.0.25 (Java Classes and Java Servlets)
Database Connectivity JDBC
Table 17: dependencies
Authentication (CAS*)
Provided by JRS. The mapping is specified in the web.xml. The software automatically checks with the CAS server each time a page is loaded to confirm the credentials of the user. CAS handles the authentication process. As this is a customised version of CAS, the setup needs to be as specified in D3.7.
Configuration Tool The configuration tool is required to update the specification tables for the Next-TELL OLM (e.g. which students are in which activity, definition of groups, definition of competencies etc.) See D3.7 for more information.
*http://www.jasig.org/cas
5.2 Downloadable Code Sources and Documentation
The OLM development code and the accompanying JavaDocs documentation may be downloaded or accessed at the following URLs.
OLM Development Code (.zip) http://eeevle.bham.ac.uk/nexttell/openlearnermodelcode.zip
The high level architecture for the Next-TELL OLM is shown in Figure 29. The system has two informational dependencies: context information provided by the configuration tool; and preference-based information from the user (bottom left). This information is then used both in the learner modelling process and in the routines that are used to update the evidence layer of the system (centre of figure). Evidence for the learner model can
come from multiple sources; principally evidence either comes through the API from another system, or it is entered manually (top left). The front end functions (i.e. the pages the user may see) are shown to the right of Figure 29. Information about the evidence and updates to the learner model are taken directly from the evidence layer. Open learner model information is the result of the learner modelling process, which is influenced by evidence, contexts and preferences set in the OLM filter (i.e. what should currently be displayed). OLM visualisations are also used in the discussion feature, which has its own database to store discussion related information.
Figure 29: information flow through the OLM software.
5.4 Core Libraries and Directories
To realise the architecture outlined in Section 5.3 the following resource libraries/directories are implemented (Table 18). For each of the key areas of the architecture the table outlines the JSP pages, Java Servlet packages and Java packages that are key to the implementation. For a full specification of available functions, please refer to the online Java Documentation at http://eeevle.bham.ac.uk/nexttell/javadocs
Table 18: core libraries and directories for key facilities of the system
JSP Page web>gui>read pages for displaying the learner model, the evidence layer and the modelling process layer.
web>gui>read>evidence pages for displaying and editing evidence
Add Evidence
Servlet nexttell.servlets.evidence.* functions for adding evidence (non-API)
nexttell.servlets.spreadsheets.* functions for utilising spreadsheets in the add evidence process
nexttell.servlets.api.addevidence.* functions for adding evidence using the API
JSP Page web>gui>add web pages for adding evidence
Updates and Notifications
Java nexttell.notifications.* methods for displaying updates
JSP Page web>gui>notifications updates pages
Discussion
Java nexttell.chat.* methods supporting the discussion facility
JSP Page web>gui>chat pages for the discussion facility
API
Servlet nexttell.servlets.api.* all API methods (including those to query and update information originating from the configuration tool)
nexttell.servlets.api.addevidence.* API methods for adding evidence
nexttell.servlets.api.readevidence.* API methods for reading evidence
5.5 Key Algorithms
The JavaDocs and commented code that form part of this software release for the Next-TELL OLM contain a wealth of description for the algorithms, methods and classes realised in the system. Included here are several key algorithms in use, such as those used for learner modelling, data searches, identifying information for the filter mechanism, and for the notifications mechanism. JavaDocs are available online at http://eeevle.bham.ac.uk/nexttell/javadocs
5.5.1 Opening the Learner Model
Information that is ultimately visualised in the open learner model exists in several layers of the system (Figure 30). Evidence is persisted in the database layer of the system in denormalised tables (see Sections 5.5.5 and 5.6). Information is then accessed through the connection layer, which implements JDBC. For a given information request the data layer will then contain all the relevant pieces of data that are to contribute to the learner model representation (these can be accessed as per Section 4.4.11). The modelling layer takes these pieces of evidence and uses them to create a model of the learner’s current competencies, taking into account the age of the information, in addition to its relationship to other entities in the modelling process (competencies, students, activities, groups etc.) The modelling process is also opened up for students and
teachers to inspect (Section 4.4.12). Finally the results of the modelling process are passed to visualisation layer where they are represented in user interpretable forms such as skill meters, word clouds, network diagrams and radar plots (Sections 4.4.2 to 4.4.9).
Figure 30: data for open learner model visualisation
The following algorithm is the method used to open the learner model. Evidence from the underlying database is retrieved, the context defined, and the learner modelling algorithm executed to result in a list of data items that can be passed to the visualisations.
1. Define the context of the information search. The data structure KLContext is used to store this information. Minimally, components of the following are (optionally) specified:
information sources (teacher, student, peer, automated)
group
student
activity
competency
teacher
tag
2. The raw data is retrieved from the database using methods from the class KLSourceSearch. The data is in for form of KLStructure, which contains the full context for each data item.
3. How the information will be combined/aggregated is specified, together with which elements should be in the list of entries (i.e. is this a list of activities, groups, competencies or students, and what items should be present in the list). Methods .setBreakdownStructure and .setBreakdownType from the class KLSourceSearch are utilised.
4. The learner modelling algorithm is called (see Section 5.5.2). This takes the KLContext from (1), with the additional context specified in (3), and the KLStructure data from (2). The routine outputs the model in the form of a further KLStructure.
5.5.2 Learner Modelling Algorithm
The Next-TELL learner modelling process is competency-based and executes over the raw data (individual inferences) at the point at which the learner model is required to be opened. This allows the OLM to take a more ecological approach (McCalla, 2004) and flexibly visualise different combinations of the underlying data. The learner model may be calculated for any combination of students, groups, competencies, or activities, from any custom combination of the raw data. The following algorithm is applied:
D4.7 Student Model Tools Code Release and Documentation
1. Retrieve all relevant items of evidence, according to the sub-set of inferences to be modelled. Identify how the visualisation is to break down the information (e.g. list of students, groups, competencies or activities). Split the evidence items by student, and then by competency.
2. For each sub-set of information (split by student, and then by competency) order the evidence newest to old- est. Determine influence factor for each item of evidence by applying the following formula iteratively over each dataset item. This provides a basic weighting for each item of evidence, with the most recent item receiving a greater influence weighting. b = basic influence (initially 1.0) and d = basic depreciation factor (default 0.2).
bi = bi-1 (1.0 – di) (1)
3. Once each data item has its basic depreciation factor (b), the level of influence (i) the item has in the learner model is further influenced by parameters that the teacher has control over, according to the following calculation. f = teacher specified depreciation factor, affecting level of influence of new information; c = level of influence that the competency has; a = level of influence the activity has, with which the item of evidence is associated.
i = b (0.5 + f/2) (0.5 + c/2) (0.5 + a/2) (2)
4. For each sub-set of information, the levels of influence (i) are normalised so that they total 1.0.
5. For each sub-set of information, the level of influence (i) and each value (v) are multiplied together and then summed, where v>=0.0 and v<=1.0.
Σ iv (3)
6. For each sub-set of data, the values (V) for competencies (C) and their sub-competencies (S) are combined recursively according to their structure, using the following formula, where D indicates the presence of data for the competency or sub-competency:
Vi = {
0.5VC + 0.5VS
VC
VS
0.0
if DS /\ DC
if ¬DS /\ DC (4)
if DS /\ ¬DC
if ¬DS /\ ¬DC
7. The value for each student (see point 1) is taken as the root node in the respective competency structure. The value for the item in the list (e.g. group, competency or activity – see point 1) is determined by taking an average of all students for whom there is data.
5.5.3 Filter Mechanism Algorithm
This algorithm is used to determine the contents of a list based where parameters are specified. For example if the session attribute scope_activity is set, specifying that a particular activity is the current focus, then the list will only contain elements that have association with that activity. The lists may be of varying types e.g. competencies, activities, groups, students etc.
1. Retrieve all possible items for the list that the user has permission to see
2. For each of the following types of information execute states 3-5:
subject,
unit of work,
activity,
competency,
teacher,
group,
D4.7 Student Model Tools Code Release and Documentation
3. If there is a filter parameter set, then perform a further search to determine all possible items that should be visible.
4. If the item is in both the master list from (1) and parameter list from (3), then keep it in the master list, else remove non-relevant items from the master list.
5. The list to be presented comprises the items remaining in the master list.
6. (Competency list only) For each item in the list from (5), check the list to see if its parent item in the competency structure is present: if it is not, then add it. For each item do this until the root node is reached.
5.5.4 Notifications/Updates Algorithm
1. get raw data, ordered from newest to oldest
2. from the data define the list items: one for activity, one for each competency, one for each group that is present in the dataset. Retain the order of last updated to oldest.
3. for each item determine the number of updated items for each of the categories, and the total number of items for each category:
learner model information
textual information (collated alongside the learner model, e.g. strength, guidance, suggestion, difficulty)
The denormalisation routine is used to retrieve relevant attributes from a variety of different tables to add into the main evidence table. The process of denormalisation duplicates the places that this information is available in the database, but increases the speed at which the information is accessed upon retrieval, as no database table joins are required. The denormalisation process also provides the full context of each piece of evidence.
1. Start with a few basic pieces of information:
is the piece of evidence for the learner model, or text to be collated alongside the learner model?
ID of the class/group
ID of the student the evidence relates to (anonymised OLM user id)
ID of the ‘activitytemplate’ entry (or the competency ID if it not associated with a group)
name of the evidence source
the values for the piece of evidence (if for the learner model this is a number between 0.0 and 1.0, if it is to be collated alongside the model then this is a textual statement which is either a strength or an item of guidance.)
any tags or artefacts associated with the piece of evidence
2. Based on the type of evidence, decide which table to place the information in: search_knowledgelevelraw for the learner model, or search_textualraw for other.
3. Determine the next ID number for the piece of evidence, and specify the current time.
4. Retrieve the credentials of the student the information relates to
5. Retrieve the credentials relating to the evidence source
6. Retrieve the credentials relating to the person who contributed the information
D4.7 Student Model Tools Code Release and Documentation
7. Retrieve the information relating to the activity and the competency the evidence is associated with.
8. Retrieve the credentials of the teacher associated with the activity, and also the subject and unit of work (where relevant)
9. Retrieve the credentials relating to the class/group
10. Retrieve the full definition of the competency (including the path, describing which other competencies are the ‘super’ competencies in the structure)
11. Add in any artefacts and tags required. Specify the information as ‘approved’.
5.6 Database - connectivity and design. Data storage structures.
The class nexttell.databasefunctions.DBComms contains methods communicating with the Apache Derby database using JDBC. The class accepts SQL, and can return sets of data or execute statements are required. A series of extra methods are provided to abbreviate the amount of SQL that needs to be sent to the class. Connectivity to the database is encapsulated in the Java class nexttell.databasefuctions.DatabaseSearch which provides many of the methods required for communication with the OLM database, which comprises of the structure which is fully specified in Appendix 4 (Section 0). The following is a summary of the database tables and their purpose (Table 19).
Table 19: database tables and their function
Table Name Description Type Sync. with Config. Tool
ACTIVITY Information about activities Informational Yes
ACTIVITYCLASS Specifies the activities assigned to each class/group
Relational Yes
ACTIVITYTEMPLATE Specifies the competencies assigned to each activity
Relational Yes
CASID_USERID Maps the user’s CAS ID to the anonymised OLM user ID
Informational Yes
CHAT Discussion entries Informational No
CHATREAD Specifies which discussion entries have seen by their potential recipients
Informational No
CLASSGROUP Information about class/groups Informational Yes
COLOURDEFINITION Defines the colour preferences for each user Informational No
COMPETENCIES Information about competencies Informational Yes
COURSEMATERIALS Information about links to teachers’ course materials
Informational No
EVIDENCESOURCE Information about data sources Informational No
GENERALPREFERENCES Preferences of teachers and students Informational No
LANGUAGE Definition of all phrases and words displayed on the interface in each supported language
Informational No
NEXTAVAILABLEID Next available ID numbers for key fields in Informational No
D4.7 Student Model Tools Code Release and Documentation
Table Name Description Type Sync. with Config. Tool
database tables
SEARCH_KNOWLEDGELEVELRAW Evidence for the learner modelling algorithm. This is a denormalised table and contains duplicate data from many of the other tables categorised as “informational”
Informational No
SEARCH_TEXTUALRAW Evidence that is collated alongside the learner model (strengths, guidance, difficulties, suggestions). This is a denormalised table and contains duplicate data from many of the other tables categorised as “informational”
Informational No
SESSION A record of each login by each user Informational No
STUDENT Information about users. e.g. usertype. Informational Yes
STUDENTACTIVITY Specifies the students linked to each activity Relational Yes
STUDENTCLASS Specifies the students linked to each class/group
Relational Yes
STUDENTSUBJECT Specifies the students linked to each subject Relational Yes
STUDENTTEACHER Specifies the students linked to each teacher Relational Yes
STUDENTUNIT Specifies the units of work linked to each student
Relational Yes
SUBJECT Information about subjects Informational Yes
TEACHERCOMPETENCY Specifies the competencies linked to each teacher
Relational Yes
UNIT Information about units of work Informational Yes
UNITCLASS Specifies the units of work linked to each class/group
Relational Yes
VISUALISATIONPREFERENCES Which visualisations users wish to include in their open learner model
Informational No
5.7 Session Variable Specification
Session variables are used to store temporary information about the user’s interaction that is used in the rendering of pages and executing algorithms. These are split into four categories: general, information focus, display and external communication. Some attributes must be set in order for the system to function, whilst others are optional. These are specified in the following sections. In all cases when a variable is does not contain a value it is set to null.
5.7.1 General
These variables identity of the user and their high level preferences e.g. language, role. (Table 20.)
D4.7 Student Model Tools Code Release and Documentation
authenticated The user is authenticated, and the required session variables are set. If this is null no access will be granted to any page in the OLM and the login script will be executed
Mandatory
studid Numerical ID of the user Mandatory
username CAS ID of the user Mandatory
logintype Role of the user, i.e. student or teacher Mandatory
sessionnumber Total number of times the user has logged in Mandatory
mastergroup Group the user belongs to i.e. school Mandatory
language Language preference (English, Norsk, Deutch, Francais) Mandatory
pagetype Title description of the main page current displayed Optional
errormessage Message returned from a routine where there has been an error Optional
5.7.2 Information Focus
The following variables are used to specify which filters are currently set. These are used to determine what information is displayed in the learner model and are used to initialise the learner modelling routine each time an information request is made. Ultimately they determine what is to be displayed on the screen for the user. For the teacher the attribute scope_teacher is permanently set to their numerical user ID, and for the student the attribute scope_student is permanently set to their numerical user ID. This ensures they do not see information of other users. (Table 21.)
Table 21: session variables relating to informational focus
Variable Name Usage Status
scope_teacher Numerical user ID of the teacher Mandatory
scope_class ID of the specified class group Optional
scope_student Numerical user id of the student Mandatory
scope_activity ID of the specified activity Optional
scope_unit ID of the specified unit of work Optional
scope_subject ID of the specified subject Optional
scope_competency ID of the specified competency Optional
scope_peer Numerical user ID of the specified peer (add evidence only) Optional
scope_tag Specific tag that evidence is to be filtered by Optional
showinfofromteacher Include information from teacher assessments (manual entry) Optional
showinfofromstudent Include information from self-assessments (manual entry) Optional
showinfofromautomated Include information from external systems (through API) Optional
showinfofrompeer Include information from peer assessments (manual entry) Optional
showinfofrommerge Combine information from all sources into one value Optional
D4.7 Student Model Tools Code Release and Documentation
chat_classid ID of the class group for the currently selected discussion Optional
chat_competencyid ID of the competency for the currently selected disucssion Optional
5.7.3 Display
The following session variables relate to the OLM screen specifically, and also to the add evidence screen, in the case of the filters. Each screen contains a large volume of information and in the interests of conserving screen real estate these variables specify whether certain sections are collapsed/closed. (Table 22.)
Table 22: session variables relating to whether sections are open/closed in the OLM screen
Variable Name Usage Status
filterstate Filter bar on the left hand side of the screen is shown Optional
open_f1 Information sources section of the filter is shown Optional
open_f2 Tags section of the filter is shown Optional
open_f3 Groups section of the filter is shown Optional
open_f4 Students section of the filter is shown Optional
open_f5 Competencies section of the filter is shown Optional
open_f6 Activities section of the filter is shown Optional
open_c1 Groups and students section of the OLM visualisation is shown Optional
open_c2 Competencies section of the OLM visualisation is shown Optional
open_c3 Activities section of the OLM visualisation is shown Optional
open_sk Skill meter visualisation is the active visualisation Optional
open_ta Table visualisation is the active visualisation Optional
open_sm Smiley Faces visualisation is the active visualisation Optional
open_hi Histogram visualisation is the active visualisation Optional
open_wo Word cloud visualisation is the active visualisation Optional
open_ra Radar plot visualisation is the active visualisation Optional
open_tr Treemap visualisation is the active visualisation Optional
open_bu Network visualisation is the active visualisation Optional
5.7.4 External Communication
Communication with some other systems requires credentials to be stored to maintain the connection. For the Google Spreadsheet facility the following are required (Table 23).
Table 23: Google document specific session variables
Variable Name Usage Status
showgooglespreadsheet Google spreadsheet is to be shown Optional
D4.7 Student Model Tools Code Release and Documentation
credentials Credentials returned from Google’s OAuth process Optional
credentials_text Credentials reutured from OAuth stored as text Optional
5.8 API specification
An application programming interface (API) is implemented to allow other systems to communicate with the OLM. The API may be used to access configuration information, update configuration information, add evidence to the learner model, read the output of the learner modelling process etc. Each request to the API is required to give the CAS user id of the person requesting and a shared secret, before the information request is satisfied. API functions will check that the specified user has authorisation to access the requested method and will also perform error checking to see that all parameters are specified and are of a valid combination. For a full specification of API functions, please refer to Appendix 5 (Section 1).
5.9 Authentication and Authorisation
The authentication process is handled by a third party tool, CAS, which is described in D3.7. Each user has a unique password-protected account. CAS protects all webpages that start with the path nexttell-cas/gui; No OLM webpages may be accessed without CAS credentials being verified first, and the OLM has no access to password information. The API is outside of the area of authentication and is protected by a shared secret. The following attributes are returned from the authentication service:
varchar() username user’s CAS ID (not anonymous)
varchar() lastName user’s surname
varchar() givenName user’s forename
varchar() role user type, i.e. student or teacher
varchar() group the group that the user belongs to, i.e. the name of the school
The attribute “username” is converted to numerical value, which is then used as a key in the OLM database; this anonymises the information stored about the user.
Each user is either a teacher or a student, which customises specific features available to them. In summary:
Students: Students may only see data related to their own learning and their own learner model.
Peers: Students are permitted to contribute information to other students’ models by completing a peer-assessment. One student may not see the data of another student.
Teachers: Teachers may only see information for students that they currently teach, and may only view student information for activities they create. Teachers may not see information in other teachers’ user accounts, and cannot see data for students they do not teach, and for activities students are involved in that they have not created.
Software using the API: May access information about any specific user, provided that the user’s CAS credentials and the shared secret are passed as parameters along with the request.
5.10 Localisation
At the time of pageload all required textual statements that are to be displayed on the interface are retrieved from the “Language” table in the database. Each statement is referred to by number, and the language is specified as the session attribute “language”. A full definition of these is given in Appendix 2 (Section 0). Language statements were distributed to project partners for translation and then refined through following workflows within the OLM to check the translations were appropriate in context.
D4.7 Student Model Tools Code Release and Documentation
The configuration tool is used to provide the context and base information required for the OLM to run. Primarily this includes the definition of competencies, groups of students, and of activities. The configuration tool specifies these entities along with the relationships between them, in addition to structural relationships.
Figure 31: synchronisation with configuration tool.
The configuration tool and the open learner model have different databases (Figure 31). When information is changed in the configuration tool, an update is automatically pushed to the open learner model. The API function ConfigurationToolUpdate is used to execute SQL to update the open learner model database accordingly. If the configuration tool is offline, the OLM will continue to function, but no updates to the configuration will occur. If the OLM has been offline it will pull data from the configuration tool upon restart, to ensure its tables are up to date. Users may not log into the OLM whilst it is synchronising. The location of the OLM and configuration tool are specified as parameters in each piece of software. The tables that are synchronised are specified in Table 19 (Section 5.6).
5.12 Connectivity with Google
The OLM software uses Google Spreadsheets to allow multiple learner models to be updated at once by the teacher and for a record of the data entry to be stored on the teacher’s Google Drive. To do this the OLM authenticates with the teacher’s Google account and stores the Google credentials in the session. Google’s OAuth authentication process is followed (Figure 32). Each time the spreadsheet facility is used a spreadsheet is uploaded using a .xlsx template and the uploaded spreadsheet is embedded in the teacher’s OLM screen. When input has occurred a button is clicked to trigger the system to read the spreadsheet and use its content to update the learner model accordingly.
Figure 32: Google OAuth process.
D4.7 Student Model Tools Code Release and Documentation
In this section we consider the final implementation of the Next-TELL OLM in terms of some “quality software” attributes (Section 6.1) and a strengths/weakness analysis (Section 6.2). We highlight areas of the implementation worthy of mention in terms of these two analyses and propose a few areas for future development (Section 6.3).
6.1 Quality Software
We comment on the implementation of the Next-TELL OLM in terms of interoperability, scalability, reliability, performance and usability, as defined by Klerkx et al., 2010.
6.1.1 Interoperability
“The ability for two or more systems to cooperate at runtime.”
The API can both accept evidence data and also provide external representations of the learner model to other systems. It implements a flexible specification for configuration and may be easily extended.
There has been established integration within the lifetime of the project/tool development with other learner model systems (OLMlets (Bull et al., 2010)), other inferencing systems (ProNIFA (Kickmeier-Rust et al., 2013)), other data systems (Mahara (Giorgini and Reimann, 2013)), and can combine data with information tendered manually through web forms, or read from specially generated Google Docs.
6.1.2 Scalability
“The support large numbers of components and interactions”
The system has been used successfully in classrooms of size 40+.
The software has an extendable database structure.
Single database connection established for each pageload, to be economical with processing resources.
The adoption of an ecological ‘active’ learner modelling approach, increases the flexibility of the learner modelling procedure, however this has potential for increased visualisation load time for larger datasets, as compared to a learner modelling process that implements a cache.
6.1.3 Reliability
“The amount of time a system is up and running correctly (uptime and responsiveness)”
The server structure used during the project is reliable. As it is an open source project that is commercially supported, updates and bug fixes are implemented and new, more efficient versions made available across time.
Uptime has been maintained continuously from month 11 of the project onwards, with the exception of periods of scheduled maintenance and updates.
6.1.4 Performance
“The response time of the system for specific events (not human performance or system delivery time that results from local computing resource setup)”
All pages load within an acceptable tolerance of several seconds on the current setup.
The basic page loads first and all non-essential page items and visualisations are loaded post page-load.
D4.7 Student Model Tools Code Release and Documentation
Much of the implemented functionality is asynchronous and communicates with the server through JavaScript (AJAX). This means that pages are not required to be reloaded when new information needs to be displayed (reduces traffic, demands on the server and produces a smoother experience for the user).
Important tables (e.g. evidence for the modelling process) are denormalised to increase speed of database queries and decrease page load time.
Limitation: the performance of the package is dependant on the processing capabilities of the host server as also client machines (particularly with reference to rendering visualisations).
If users select large datasets, or request large amounts of data be present on the screen, processing time increases. For more interactive visualisations this load is placed on the client computer, rather than the server.
6.1.5 Usability
“The ease of use, the ease of training”
The number of webpages presented to the user are kept to a minimum for ease of navigation. Where appropriate, functionalities are combined into one place, where they are conceptually similar. AJAX functionality allows some pages to run more in the style of a desktop application.
The software uses affordances, feedback throughout, to notify the user that actions have taken place.
Summaries used throughout at top of the screen. This tells the user where they are and what information is being displayed.
Persistent navigation is available to the user in the form of a header bar.
Iconography is used to easily identify functions and concepts within the system.
Screens will adapt to changes in browser size dynamically and ensure that content is displayable. Resolution is designed from 1024x768 upwards, and will display comfortably on smaller monitors.
Further to the quality software commentary of Section 6.1, we highlight some areas of the software in terms of strengths, weaknesses, opportunities and threats to provide a summary of the final code release and implementation (Table 24).
Table 24: SWOT summary of final OLM software release.
Strengths
Multiple visualisations.
Multiple data sources.
Extendable API for connecting third party software.
Evidence and inferences can be added dynamically from third party sources.
Flexible deployment/usage: web service to be accessed from multiple locations including home and school, and in a variety of browsers on desktops, laptops or tablets.
Intuitive user interface.
Can be packaged up for local use - can run on a school intranet, so that data stays local, if privacy policies require.
Is linked to a configuration tool to allow set-up by teachers.
Is available for both students and teachers.
Weaknesses
Concept of OLM needs introduction to teachers in order for them to make full use of the tool.
OLM algorithm executes at pageload time. This may not scale for very large evidence sets.
Requires access to CAS to run.
Does not currently implement SSL.
More connected data sources would strengthen this tool’s usage.
Teachers cannot share class groups and activities in the current implementation.
Complexity of registration process (non-OLM).
D4.7 Student Model Tools Code Release and Documentation
As discussed in D4.6, NEXT-TELL’s RGFA is used for detecting students’ personal constructs external to the OLM. Next-TELL’s CoNeTo then provides a facility for negotiating the outcomes of these modelling processes. In this report, we provide an overview integration with other NEXT-TELL tools, design science model that informed the design, development and evaluation of the tools, illustrations of teaching analytics visualizations, key findings and reflections. Pedagogical scenario for nextGRID, software code release and installation instructions, and user manuals are included as appendices to this report.
7.1 Design Science Model
RGFA & CoNeTo are teaching analytics applications that are designed, developed, and evaluated based on the “Triadic Model of Teaching Analytics (TMTA) (Vatrapu, Teplovs, Fujita, & Bull, 2011). Teaching analytics research (Vatrapu, Reimann, Halb, & Bull, 2013) focuses on providing both computational and methodological support for teachers in real-time and in-situ classroom settings. At its core, the TMTA model calls for collaborative knowledge building between teachers, analysts and researchers. Each has a complementary role in the teaching analytics setting. Eliciting criteria for Teaching Analytics involves a collocated collaborative triad of a Teaching Expert (TE), a Visual Analytics Expert (VAE), and a Design-Based Research Expert (DBRE) analyzing, interpreting and acting upon real-time data being generated by students’ learning activities by using a range of visual analytics tools. TMTA involves a close collaboration between the TE, VAE, and the DBRE. It includes teaching practitioners in the design process and invites them to contribute significantly to the innovation of the visual analytics tools. This allows these learning analytics tools to address pedagogical issues as they arise and evolve in real classrooms (Vatrapu et al., 2011).
Figure 33: Triadic Model of Teaching Analytics (TMTA)
7.2 RGFA:Repertory Grids for Formative Assessment1
Repertory Grid Technique (hereafter RGT) is a method for eliciting personal constructs of individuals about elements belonging to a topic of study. RGT is based on the seminal contribution of psychologist George Kelly (1963, 1992), Personal Construct Theory, and subsequent theoretical and methodological developments (cf. Adams-Webber, 2006; Fransella, Bell, & Bannister, 2003). We implemented the triadic sorting method in RGFA
(Vatrapu, Reimann, & Hussain, 2012) which consists of the participants being presented sets of three elements each. For a given set of three elements (e.g, Windows, OSX, Unix), the participant is prompted to select the element (e.g., Unix) that is different from the other two (Windows, OSX) and to state how it is different as the “opposite construct” (e.g., “command line interface”). Then, the participant has to state how the two remaining elements in the triad are similar to each other as the “similarity construct” (e.g., GUI). The rest of the elements (other operating systems, in our example) are then rated on a Likert-item scale ranging from the Opposite Construct (1) to the Similarity Construct (5). The participants repeat this process until all the triads of elements are sorted into different and similar and the elements for that comparison are rated. The outcome of this exercise is the Repertory Grid (RG) consisting of rows with triads, columns consisting of elements with the first column being the Opposite Construct and the last column being the Similarity Construct, and the cell values consisting of the ratings given for elements. Based on the RG, one can qualitatively appraise learners’ “mental models”—what they see as ‘going together’, and on what dimensions—and/or apply clustering methods or dimension reduction methods to derive quantitative measures of learners’ knowledge structures. We conducted three cycles of iterative research and development of RGT in real classroom settings followed eye-tracking studies in the laboratory setting (Vatrapu, Reimann, et al., 2012; Vatrapu, Reimann, Hussain, & Kocherla, 2013).
7.3 CoNeTo: Communication and Negotiation Tool2
CoNeTo (Vatrapu, Tanveer, & Hussain, 2012) is a graphical argumentation tool was developed based on a review of the state-of-the art in argumentation research in Computer Supported Collaborative Learning (CSCL) and other related fields [1, 3, 4] together with the outcomes from the Participatory Design Workshop with the teachers/students in the Master of ICT and Learning programme, design explorations with teachers participating in the NEXT-TELL Project.
Figure 34: Printscreen of CoNeTo
Figure 34 presents a screenshot of CoNeTo. CoNeTo has a dual interaction space of “Graphical Argumentation Editor” (Task Space) and “Artifact-Centered Discussion” (Coordination Space). CoNeTo builds on the current state-of-the-art in CSCL argumentation systems in supporting an ontology of two types of entities (Activity and Knowledge) and three types of relationships between these entities (“For”, “Against”, and “Unknown”).
Artifact-centered discourse is supported by a linear threaded discussion associated with a given entity with support for a global view of persistent discussions.
7.4 Integration between RGFA, CoNeTo, and OLM
4
)
1
)
2
)
3
)
ECAAD RGFA OLM CoNeTo
Figure 35: Integration of ECAAD, RGFA, OLM and CoNeTo. Arrows denote communication between tools. The dotted-arrows represent future development
Figure 35 depicts the integration between ECAAD, RGFA, OLM and CoNeTo. More specifically, teacher uses ECAAD/Excel to design a RGFA exercise (activity) for knowledge diagnostic purposes. This information is used as input in RGFA. In ECAAD/Excel/RGFA a teacher specifies elements, creates triads and assigns competencies for each triad. Competencies in this case are conceptions or constructs that the teacher has in mind when creating a particular triad. A simple Boolean operator between multiple construct words with weights will provide the probability for that knowledge item/competency. The second step in Figure 35 illustrates the integration between RGFA and OLM. After the teacher has created the ECAAD/RGFA exercise, students will answer and the OLM will be automatically updated based on their constructs and ratings. Since RGFA can be used to design open-ended exercises, not all “correct” constructs can be anticipated by the teacher at the design time. OLM communicates with CoNeTo (Figure 35, step 3) where each student uses CoNeTo to “negotiate” the OLM of their RGFA grid with the teacher. For example, the student might have come up with a different construct than what the teacher has intended at design time and the OLM shows that they do not have that competency. This can result in the teacher adding new competencies at the global level for that triad and/or updating the OLM for the individual student. In addition, CoNeTo communicates with RGFA as shown in Figure 35, step 4. Technical details of this integration have already been presented in D4.6 as RGFA API and CoNeTo API.
As already described in D4.6, in order for a RGFA exercise to be created in OLM, CoNeTo exposes an API method CreateRGFAExercise that accepts two parameters, namely a casUserId and a json representing the RGFA exercise as a serialized text. Once the CoNeToAPI method receives the parameters, it de-serializes the json string and loads it into an instance of an RGFAObj. The OLM API is then used to created corresponding entities in the OLM based on the following mapping.
1. An Activity node is created for the main exercise, using the name of the exercise.
2. For each of the Triads within the exercise, a competency node is created under the previously created
activity.
Once the RGFA exercise has successfully been created in the OLM, it is then visible under the list of activities of the corresponding teacher whose casUserId was used to create these nodes in the OLM. The relevant user can then open the newly created activity in CoNeTo in order to visualize the underlying RGFA exercise represented by the OLM.
Once the exercise is loaded into CoNeTo, the details of each triad can be viewed by clicking on a particular node representing the corresponding triad. A teacher can update the knowledgelevel of the selected node by using a slider that represents the current value. Clicking on the update button confirms and sends the changes back to OLM. Consequently, the exercise is reloaded to represent the updated OLM model.
D4.7 Student Model Tools Code Release and Documentation
Moreover, clicking a node will also load the details of the particular triad that it represents, for instance the list of triad elements it contains and the answers for that particular triad, given by the selected student.
The individual nodes (triads) can be negotiated/ discussed by using the chat functionality provided in CoNeTo. The individual conversation pertaining to each node is then persisted in the RGFA database and gets updated each time a message is sent by clicking the send button.
7.5 Analytics in RGFA
Adopting the TMTA discussed above, design workshops were conducted that involved both pre-service and in-service teachers, a visual analytics expert, and a design based researcher from the learning sciences and a design science researcher from the information systems. The primary design objective was to co-create visualizations that are both meaningful and actionable to teachers. Figure 36 below presents teacher-created visualizations.
Figure 36: Teacher-Created Visualizations for RGFA
Such teacher-created visualizations were taken into account in the design, development and evaluation of teaching analytics visualizations (Figure 37) and dashboards (Figure 38 and Figure 39).
Figure 37. Teaching Analytics Support (Self vs. Class Comparison; Collective Response Time for the Class; and Opposite Element Selection Frequency Indicator) (taken from Vatrapu, Reimann, Hussain, et al., 2013)
D4.7 Student Model Tools Code Release and Documentation
Figure 38. Constructs and Elements Ratings Analysis. The teacher clicks over “Social” to compare triads between two groups of students. Word frequency is set to 10 and all triads are shown (taken from Pantazos,
Vatrapu, & Hussain)
Figure 39. Word Analysis. The teacher clicks over a word to view details from constructs elections for all students (taken from Pantazos et al.)
D4.7 Student Model Tools Code Release and Documentation
With regard to the pedagogical method of repertory grids for formative assessment, prior findings show that in designing RGFA exercises, teachers should pay particular attention to the previous domain knowledge of students (Vatrapu, Reimann, et al., 2012). Further, an ideal repertory grid exercise would involve 6-10 elements and 5-6 triads with each element appearing at least once and in different positions of the triad when a particular element features more than once across the different triads. The repertory grid exercise could be designed for individual students or as a computer supported collaborative learning (CSCL) exercise involving a small group of students(Vatrapu, Reimann, et al., 2012). Based on design workshops with teachers (Figure 34) and empirical findings from the laboratory study(Vatrapu, Reimann, et al., 2012), we implemented visual analytics for the time taken for construct elicitation and elements ratings (Figure 35) in order to provide another dimension for pedagogical decision-making. With regard to the teaching analytics dashboard (Figure 36), as reported in (Pantazos et al., 2013):
“visualization techniques and design principles from Information Visualization and Visual Analytics to allow teachers and students to obtain an overview of the data and drill-down into details. The word cloud representation of constructs provides an overview of the most used terms during construct elicitations. Details on-demand views present information regarding the element ratings and constructs. Filtering among users supports comparison and allows teachers to perform a formative assessment, and distinguish students’ knowledge based on elements ratings and constructs. Students can compare their element ratings and constructs with other students. Color and shape encoding were used in order to improve readability”.
With regard to the teaching analytics workshops in general and RGFA and CoNeTo evaluations in particular, the key lessons learnt were: (a) the need to further focus the workshop on teachers’ current pedagogical and analytical practices, (b) expand the scope to include real-time teaching scenarios in the actual classroom, (c) explicitly emphasize the use of visual analytics for technology enhanced formative assessment, (d) address the training deficiencies in both teacher education as well as professional development programs related to the psychological concepts related to competency frameworks and both declarative knowledge and procedural skills with respect to data literacy, and (e) address concerns about the privacy and security concerns with big educational data as well as the concerns about the radical transformation of the professional practices of teachers with teaching analytics and learning analytics. We will continue to explore these issues in future teaching analytics workshops.
D4.7 Student Model Tools Code Release and Documentation
[Adams-Webber, 2006] Adams-Webber, J. (2006). Reviews of A manual for repertory grid technique. Journal of Constructivist Psychology, 19(4), 351-353. doi: 10.1080/13854040600689133
[Bull and Kay, 2007] Bull, S. & Kay, J. (2007). Student Models that Invite the Learner In: The SMILI Open Learner Modelling Framework, International Journal of Artificial Intelligence in Education 17(2), 89-120.
[Bull and Kay, 2010] Bull, S. & Kay, J. (2010). Open Learner Models, in R. Nkambou, J. Bordeau & R. Miziguchi (eds), Advances in Intelligent Tutoring Systems, Springer, 318-338.
[Bull and Kay, 2013] Bull, S. & Kay, J. (2013). Open Learner Models as Drivers for Metacognitive Processes, in R. Azevedo, V. Aleven (eds), International Handbook on Metacognition and Learning Technologies, Springer.
[Bull et al., 2010] Bull, S. Jackson, T. & Lancaster, M. (2010). Students' Interest in their Misconceptions in First Year Electrical Circuits and Mathematics Courses, International Journal of Electrical Engineering Education 47(3), 307-318.
[Bull et al., 2013] Bull, S., Johnson, M.D., Alotaibi, M., Byrne, W. & Cierniak, G. (2013). Visualising Multiple Data Sources in an Independent Open Learner Model, in H.C. Lane & K. Yacef (eds), Artificial Intelligence in Education, Springer-Verlag, Berlin Heidelberg.
[Fransella et al., 2003] Fransella, F., Bell, R., & Bannister, D. (2003). A Manual for Repertory Grid Technique (2 ed.): Wiley.
[Giorgini and Reimann, 2013] F. Giorgini, P. Reimann: Computer-enhanced learning diagnostics for 21st Century classroom. ATEE Winter Conference, “Learning & Teaching with Media & Technology”, Genoa, Italy, March 2013. http://www.ateegenoa2013.sdf.unige.it/
[Kelly, 1963] Kelly, G. A. (1963). A theory of personality: W. W. Norton & Company.
[Kelly, 1992] Kelly, G. A. (1992). The Psychology of Personal Constructs: Volume Two: Clinical Diagnosis and Psychotherapy (New ed.): Routledge.
[Kickmeier-Rust and Albert, 2013] M. D. Kickmeier-Rust, D. Albert: Learning Analytics to Support the Use of Virtual Worlds in the Classroom. SouthCHI2013, Workshop on Data Mining and Data Visualization, Maribor, July 2013. http://southchi.org/
[Klerkx et al., 2010] J. Klerkx, B. Vandeputte, G. Parra, J.L. Santos, F. Van Assche, E. Duval: (2010) How to Share and Reuse Learning Resources: The ARIADNE Experience. Sustaining TEL: From Innovation to Learning and Practice Lecture Notes in Computer Science Volume 6383, 2010, pp 183-196.
[McCalla et al., 2000] McCalla, G., Vassileva, J., Greer, J. & Bull, S. (2000). Active Learner Modelling, in G. Gauthier, C. Frasson & K. VanLehn (eds), Intelligent Tutoring Systems, Springer-Verlag, Berlin Heidelberg, 53-62.
[McCalla, 2004] McCalla, G. (2004). The Ecological Approach to the Design of ELearning Environments: Purpose-Based Capture and Use of Information about Learners, Journal of Interactive Media in Education, Vol. 7.
[Pantazos et al, 2013] Pantazos, K., Vatrapu, R., & Hussain, A. Visualizing Repertory Grid Data for Formative Assessment. Paper presented at the The 2nd International Workshop on Teaching Analytics (IWTA) 2013.
[Vatrapu et al., 2011] Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011). Towards Visual Analytics for Teachers' Dynamic Diagnostic Pedagogical Decision-Making. Paper presented at the 1st International Conference on Learning Analytics & Knowledge (LAK 2011), Banff, Canada.
[Vatrapu et al., 2012a] Vatrapu, R., Reimann, P., & Hussain, A. (2012). Towards Teaching Analytics: Repertory Grids for Formative Assessment. Paper presented at the International Conference of the Learning Sciences (ICLS) 2012, Sydney, Australia.
[Vatrapu et al., 2012b] Vatrapu, R., Tanveer, U., & Hussain, A. (2012). Towards teaching analytics: communication and negotiation tool (CoNeTo). Paper presented at the Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, Copenhagen, Denmark.
[Vatrapu et al., 2013a] Vatrapu, R., Reimann, P., Halb, W., & Bull, S. (2013). Second International Workshop on Teaching Analytics. Paper presented at the Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium.
[Vatrapu et al., 2013b] Vatrapu, R., Reimann, P., Hussain, A., & Kocherla, K. (2013). Towards Teaching Analytics : Repertory Grids for Formative Assessment (RGFA). CSCL 2013 Conference Proceedings. Volume 2 - Short Papers, Panels, Posters, Demos & Community Events, International Society of the Learning Sciences, 422-426.
D4.7 Student Model Tools Code Release and Documentation
EFL 'English as a Foreign Language'; EFL refers to learning English in a non-English-speaking region, such as studying English in an Asian or Latin American nation. Typically, EFL is learned as part of a student's school curriculum or for career purposes if working for an international corporation.
ENA Epistemic Network Analysis
ESL English as a Second Language; refers to learning English in the target language environment
HCI Human Computer Interaction
ICT Information and Communication Technology
IT Information Technology
LEPP Longitudinal Evaluation of Performance in Psychology (2nd generation e-portfolio)
NEXT-TELL Next Generation Teaching, Education and Learning for Life
OLM Open Learner Model
D4.7 Student Model Tools Code Release and Documentation
PADI The PADI project aims to provide a practical, theory-based approach to developing quality assessments of science inquiry by combining developments in cognitive psychology and research on science inquiry with advances in measurement theory and technology.
RA Requirement Analysis
RDS Researcher-led Design Study
SRI Stanford Research Institute
STEM The Science, Technology, Engineering, and Mathematics (STEM) fields are collectively considered core technological underpinnings of an advanced society, according to both the National Research Council and the National Science Foundation
TDS Teacher-led Design Study
TEL Technology Enhanced Learning
TESL Teaching English as Second Language
TISL Teachers Inquiry into Students Learning
Acknowledgement: The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 258114.
D4.7 Student Model Tools Code Release and Documentation
There requirements refer to the process of opening the learner model.
Table 25: high level requirements: visualising the learner model.
Number Formulation (D4.1)
Rationale (D4.1) Compliance (final)
D4.1:1 Open learner model presentations (visualisation) should be appropriate for the experience and skill level of each stakeholder.
Each stakeholder is intended to use the information in a different way and will be of a different ability/background. All must be able to use the OLM for their needs.
(Same as D4.5)
Each stakeholder has the same range of visualisations available to them, these are of different levels of complexity.
(Extended)
Visualisations can be configured to display information in a simple manner, or to break down the information into its different soruces.
D4.1:2 The teacher is ultimately in control of when learner model information is released to students. Access needs to be reconciled with issues of trust and consistency.
Information should be available as is pedagogically appropriate at a given time. The partial availability of information must not detract from the usability and aims of the system.
(Revised)
The student and their teacher both have access to all information in a specific student’s learner model. Visualisation methods can be configured, but the information available in the evidence layer is the same.
D4.1:3 Information will encompass a wide range of subjects, modules, topics and concepts. These will need to be displayed hierarchically, in an appropriate form for each stakeholder.
The scope of interest for information will vary between different stakeholder groups and between different activities. The open learner model may be used for different purposes at different times. The hierarchical structure of information will ensure clarity is retained.
(Same as D4.5)
Information includes subjects, units of work, activities, competencies, groups and students, presented together for comparison, and available individually.
Information is displayed hierarchically where the view type supports this.
D4.1:4 Quick navigation to parts of the model recently inspected would be beneficial.
Stakeholders may wish to access the same part of the model in quick succession (e.g. during a learning activity).
(Same as D4.5)
The OLM remembers the last model inspection made by the user, and starts from this point when the page is returned to. Adding a list of recent searches could improve this functionality.
(Extended)
This context is also extended to the add
D4.7 Student Model Tools Code Release and Documentation
D4.1:5 The system must be able to justify claims it makes about the student, i.e. it must present detailed evidence, as is appropriate.
This reconciles issues of trust and clarity, and permits substantiated general claims to be made about students. Detailed information may be optionally inspected, as is appropriate to a task.
(Same as D4.5)
It is possible to drill down to the evidence layer, and also for a description of the modelling process to be inspected.
D4.1:6 In addition to current model information, the system should show historical information and be able to make predictions about future knowledge levels.
This helps facilitate planning, promotes reflection and supports various aspects of metacognition such as student self-assessment.
(Comment - D4.5)
Predictions and availability of historical information is not present in the OLM. There is potential for this to be included at a later date.
(Revision)
Current information is shown. This is to support planning and promote reflection in alignment with the rationale of the requirement. Additional information about how the model has changed reduces the focus of the tool, which is key to its uptake and its current application within the project.
D4.1:7 The OLM interface must support different granularities of inspection.
Dependent upon the short term goals of the stakeholder, sometimes summary information is required, sometimes more detail is helpful.
(Same as D4.5)
The OLM presents high level overviews, and information broken down into canonical components is available. A filter mechanism exists to tailor the scope of information shown.
D4.1:8 Avoid information overload, but allow sufficient detail to be inspected, if required.
Information overload can cause confusion and demotivation on the part of the stakeholder. Detailed information needs to be available if required for a specific task.
(Same as D4.5)
High level abstraction OLM visualisations are available. Information can be drilled down into to get at the evidence layer, together with information about the modelling process. A filter mechanism is available to tailor results.
D4.1:9 OLM presentations must account for individual differences (particularly amongst students, parents and teachers).
Each stakeholder group is not homogeneous. This is particularly the case with students and parents. Each group will contain a range of backgrounds, skills and experiences in addition to different approaches to learning and preferences.
(Same as D4.5)
OLM views of different complexity are implemented. Each has potential to show different aspects of the same underlying information, for example some represent structure, whilst others categorise information, and others collate it. The parent stakeholder group does not form part of the release - this was omitted in year 3.
D4.1:10 Visualisations must be intuitive to use.
The open learner model should not confuse its users. Good usability is important for sustained usage.
(Same as D4.5)
Implemented visualisations follow standard metaphors, and are presented with supporting information, such as keys.
(Extended)
Colours and icons are used consistently throughout.
D4.7 Student Model Tools Code Release and Documentation
D4.1:11 The visualisation mechanism should be quick to load in the web browser.
Good usability is important to retain users’ engagement with the system. Slow to load information will increase frustration.
(Same as D4.5)
All features in the OLM load within several seconds. Data transfer is minimal. Algorithms execute on the server side wherever possible, and cached information is retained to enable quick display of information. Some database information is denormalised to reduce query times.
(Extended)
Database connections are made only once per request, enabling quick access to the data layer. All non essential page load items are executed after the page is loaded.
(Comment)
An active learner modelling approach is used to increase the flexibility of access - i.e. the learner modelling happens at the time of this information request. This could impact on performance with large data sets.
Information Content of the Learner Model
Requirements listed here are in reference to the content of the learner model
Table 26: high level requirements: learner model information content.
Number Formulation (D4.1) Rationale (D4.1) Compliance (final)
D4.1:12 Information should only be modelled if it is intended to be used.
For reasons of ethics, rights of access to information, software efficiency and data storage, data should not be captured without purpose.
(Revised)
Everything captured is available for inspection by both the teacher and student. All information is used.
D4.1:13 The format in which the learner model is kept must be flexible and able to encompass new information formats
The learner model tools may be used beyond the scope of the project and may adapt to any changes in specification that might occur. Issues of extensibility, flexibility and future proofing are important.
(Same as D4.5)
The OLM can cope with numerical data relating to student competencies. Textual information is collated alongside the model, although it is not transformed by the modelling process. The API interface allows information of different formats to be patched into the system.
D4.1:14 Finer grained model information should be kept, which may then be abstracted to the level to which it is to be externalised.
The system is able to give justification of its inferences, without presenting information overload. Detail may be inspected in contexts in which it is useful.
(Revised)
Information stored in the repository is the raw evidence, which is the finest grain size permitted. The modelling process executes at the point of the information request (active learner modelling), and the granularity of model information visualised is determined by the parameters of the
D4.7 Student Model Tools Code Release and Documentation
D4.1:15 Single pieces of information may update multiple parts/facets of the model.
Different inferences can be made about a single piece of evidence.
(Revised)
A single piece of evidence may update multiple competencies and trigger updates to activities, groups, and a student. The model contains only a limited number of facets, but with the addition of extra access methods to the data layer, the same information can be used to inform different model and representations of its content. This functionality is supported by retaining a more ecological approach to learner modelling.
D4.1:16 Information from different parts of the model may be combined to create secondary inferences.
Some inferences may result from the triangulation of more fine grained metrics. (e.g. time on task, competency and confidence to infer affective motivational states)
(Same a D4.5)
Numerical information is combined at different levels of abstraction, according to the modelling process, and textual information is collated.
D4.1:17 The format/language of the learner model must be flexible enough to represent qualitative and quantitative information.
The learner model is required to hold information about competencies in addition to epistemic beliefs and 21st Century skills.
(Same as D4.5)
Numerical information is modelled and textual information is collated. Because of the introduction of RGFA, the originally envisaged epistemic information now takes a lesser role in the project. (Teachers now transform RGFA outcomes.)
Updating the Learner Model
The following are formulated to inform the way in which the learner model updates should take place.
Table 27: high level requirements: updating the learner model
Number Formulation (D4.1) Rationale (from D4.1) Compliance (final)
D4.1:18 The teacher should have control over configuration settings of the learner model (e.g. information weightings).
The same information contributing to the learner model may be more or less relevant, depending upon a current pedagogical strategy.
(Same as D4.5)
The teacher has control over configuration settings including weightings, and may enable and disable features such as peer-assessment or student self-assessment.
D4.1:19 The learner model needs to comprise current information that is weighted accordingly.
The learner model is a model of the current state of the learner and so recent evidence is considered to be more important. Different information sources provide different qualities of information; the data should not be distorted through addition of something with lesser
(Same as D4.5)
The content of the learner model is always current. Weighting mechanisms define the influence information has, and recent information is counted with greater influence, where multiple pieces of
D4.7 Student Model Tools Code Release and Documentation
relevance. Weightings may also depend on pedagogical aspects of the teacher’s chosen strategy.
information exist.
D4.1:20 Teachers should be able to override inferences, in specific cases.
Teachers should be able to edit the learner model. They perform an administrator role in the learning process.
(Same as D4.5)
Teachers are able to edit and delete pieces of evidence.
D4.1:21 Key stakeholders should be able to add information that the system does not currently have, or cannot infer.
Classroom deployed technology cannot automatically infer many aspects of the human affective state that humans can readily interpret. The NEXT-TELL system will also rely on accounts and evidence for activities that take place away from a computerised environment.
(Same as D4.5)
Manual information input is possible for students, teachers and peers,
(Extended)
In addition, information of this type can be automatically transferred from other systems through the API.
D4.1:22 The learner model / modelling process must be able to contain/deal with conflicting and incomplete information.
Learners will demonstrate conflicting understanding. Different learners may contribute different types of information depending upon activities specified, their particular class group and their engagement outside of the classroom environment.
(Same as D4.5)
The learner model can deal with conflicting inferences, and works with the information it has available.
D4.1:23 The learner model must be able to store information and viewpoints from key stakeholders separately.
Open learner models for different stakeholders will have different access privileges. Information from different sources may have different weightings, in different contexts. Particularly with epistemic information, it is important to be able to discern one stakeholder’s account from another.
(Same as D4.5)
Information and evidence is flagged in the OLM database as originating from a specific stakeholder. This information is used when data is presented to the user.
D4.1:24 Model updates should be instantaneous.
The model information should always be current. To maintain engagement and usability of the NEXT-TELL system, information needs to be available immediately when it is requested.
(Same as D4.5)
Updates to the learner model propagate through the modelling process with immediate effect.
(Extended)
The active learner modelling approach ensures that modelling happens with all relevant items of data that should inform the modelling process.
D4.7 Student Model Tools Code Release and Documentation
The following is a definition for all language terms used in the Next-TELL OLM in the four languages supported (Table 28). Please see Section 5.10. for more information about the localisation of the tool.
Table 28: localisation of the Next-TELL OLM: all phrases and language items.
ID English German Norwegian French
1 Home Home Hjem Accueil
2 Open Learner Model Open Learner Model Open Learner Model Open Learner Model
3 OLM OLM OLM OLM
4 Add Evidence Evidenz hinzufügen Legg til resultat Ajouter des Preuves
107 add unit Einheit hinzufügen Legg til enhet ajouter une unité
108 add activity Aktivität hinzufügen legg til aktivitet ajouter une activité
109 edit unit weights Einheitengewichtung verändern
Endre vekting av enheter
changer les poids d'unité
110 Subject moved up one position
Fach eine Position nach oben
Emne flyttet opp en posisjon
Le sujet a été déplacé d'un cran
111 Enter name for unit of work
Namen für Arbeitseinheit angeben
Gi navn til arbeidsenhet
Donner un nom à l'unité d'apprentissage
112 Add New Unit of Work
Neue Arbeitseinheit hinzufügen
Legg til ny arbeidsenhet
Ajouter une nouvelle unité d'apprentissage
113 Create Erstellen Skap Créer
114 Cancel Abbrechen Avbryt Annuler
115 Unit moved up one position
Einheit eine Position nach oben
Enhet flyttet opp en posisjon
L'unité a été déplacée d'un cran
116 OK OK OK OK
117 Google Spreadsheets Google Spreadsheets Google Spreadsheets Feuilles de calcul Google
118 This feature uses Google Drive and will require you to log in using your Google account.
Diese Funktion benötigt Google Drive. Sie müssen sich mit Ihrem Google-Account einloggen.
Denne funksjonaliteten bruker Google Drive, og krever at du logger deg på din Google konto
Cette fonctionalité utilise le Google Drive et demande que vous vous connectiez à votre compte Google.
119 Don't show this message again
Diese Meldung nicht mehr anzeigen
Ikke vis denne meldingen igjen
Ne pas montrer ce message à l'avenir
120 new notification Neue Nachricht ny melding Nouvelle notification
121 show calculation Zeige Berechnung vis utregning Montrer les détails du calcul
122 hide calculation Berechnung ausblenden skjul utregning Cacher les détails du calcul
123 This page shows feedback about learners' understanding.<br><ul><li>Select <i>filter</i> criteria to tailor your search (e.g. group, student or activity) </li><li>Select a <i>presentation</i> method to view the
Diese Seite gibt Rückmeldung über das Verständnis der Schüler.<br><ul><li>Wählen Sie <i>Filterkriterien</i> aus, um Ihre Suche anzupassen (z.B. Gruppe, Schüler oder Aktivität). </li><li>Wählen Sie eine <i>Darstellungsvariante<
Denne siden viser tilbakemeldinger på elevers forståelse. <br><ul><li>Velg <i>filter</i> kriterier
for å skreddersy ditt søk (e.g. gruppe, elev eller aktivitet)
</li><li>Velg en <i>presentasjons</i>
metode for å se
Cette page montre les détails du niveau de compréhension et d'apprentissage des élèves. Veuillez <br><ul><li>sélectionner les <i> filtres</i> nécessaires pour affiner votre recherche (e.g. groupes, élèves ou activités) </li><li>Selectionn
D4.7 Student Model Tools Code Release and Documentation
information (e.g. skill meter)</li><li>Click the "<i>e</i>" icons to get evidence and guidance information</li></ul>
/i> (z.B. Histogramme) aus, um die Informationen anzuzeigen. </li><li>Klicken Sie auf die <i>e-Symbole</i> um Rückmeldung und Erklärungen zu erhalten.</li></ul>
informasjonen (e.g. kompetansehistogram)</li><li>Klikk "<i>e</i>" ikonene for å få resultat og veiledningsinformasjon
</li></ul>
er une méthode de <i>visualisation</i> des informations (e.g. Histogramme)</li><li>Cliquer sur une icone "<i>e</i>" pour afficher les preuves et les informations de conseils d'améliorations</li></ul>
124 Combining Learner Model Information, and the Learner Model Calculation
Lernermodell-Informationen mit der Lernermodell-Berechnung kombinieren
Kombinerer læringsmodellinformasjon og læringsmodellberegning
Agglomération des informations du modèle, et calculs des éléments du modèle
125 selected ausgewählte utvalgte sélectionné
126 students Schüler elever élèves
127 students' peers Mitschüler elevers medelever camarades de classes
128 No visualisations are currently selected
Im Moent sind keine Darstellungen ausgewählt
Ingen visualiseringer er valgt for øyeblikket
Aucune préférence de visualisation choisie
129 Please select some visualisations using the <a href="../../../../../../<%=admin.appname%>/gui/preferences.jsp">preferences page</a>
Bitte wählen Sie Ihre Darstellungsvarianten unter <a href="../../../../../../<%=admin.appname%>/gui/preferences.jsp">Oersönliche Einstellungen</a>
Velg noen visualiseringer ved hjelp av <a href="../../../../../../<%=admin.appname%>/gui/preferences.jsp">innstillingssiden</a>
Veuillez sélectionner une ou plusieurs visualisations en utilisant la <a href="../../../../../../<%=admin.appname%>/gui/preferences.jsp">page de préférences</a>
131 nothing to display nichts anzuzeigen Ingenting å vise rien à afficher
132 up one level EIne Ebene nach oben opp ett nivå aller un niveau plus haut
300 Squares Kästchen Firkanter Par carrés
133 Strips Streifen Striper Par bandes
134 Slices Stücke Stykker Par quartiers
135 This page contains the latest user manual for the tool.
Diese Seite beinhaltet das aktuellste Benutzerhandbuch für dieses Werkzeug
Denne siden inneholder den siste brukermanualen for verktøyet
Cette page contient la dernière version du manuel d'utilisateur
136 You are up to date. Sie sind auf dem neusten Stand
Du er oppdatert Les informations sont actualisées.
137 No notifications to display.
Keine Benachrichtigungen vorhanden.
Ingen meldinger å vise Aucune notification à afficher
138 Today Heute I dag Aujourd'hui
D4.7 Student Model Tools Code Release and Documentation
Open learner Models (OLMs) show learners’ current understanding or competencies. The visualisation of the learner model dynamically updates, to present the current competencies or knowledge state of the learner. Learner models are built up over time and information may be gathered from a variety of sources. Information (inferences about student understanding) may come from:
Information is available at different levels of abstraction: competencies, individual students, groups, activities, units of work, or subjects. The information in the learner model is available for inspection by teachers, learners, and their peers. Several types of visualisation are provided to support inspection of the learner model (Figure 1).
User accounts for the CAS system may be registered at:
Upon login teachers are directed towards the homepage (Figure 2). This page provides navigation to each of the main features available. The homepage can be returned to at any point by clicking the “home” button in the header bar, which is always present at the top of the screen.
Using the buttons to the left of the page, or the links in the header bar, teachers can access the main features of the tool:
Open Learner Model: students’ open learner models, visualised as skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency treemaps and competency networks. The open learner model page displays information on all competencies involved in current teaching, with the option to narrow the information down to specific students, groups, activities, and sub-competencies. This page also contains links to a description of the modelling process, lists of supporting evidence, and textual statements that describe student strengths and areas of guidance. Information in the learner model can come from multiple sources, including student self-assessments, peer-assessments, and other Next-TELL tools. (See Sections 3 and 4)
Add evidence using a web form: select a student and add learner model information, strengths and guidance (See Section 5.1). Alternatively select a group and an activity to use a spreadsheet method of uploading learner model information. (See Section 5.2)
View OLM updates: view a statement of updates to the learner model made by students and their peers, and also updates from automated data sources. (Section 6)
Participate in discussion: participate in discussion with groups of students about information presented in their open learner model and competencies in their learning. (Section 7)
Hovering over the username will display a dropdown menu. Using this menu teachers may specify preferences, and specify a desired language:
Preferences: select one or several visualisations to use in the open learner model. (Section 8)
Configuration: be redirected to the configuration tool to define groups of students, lists of competencies, activities that students should complete, and additional options about the sources of learner model information. (See the Configuration Tool user manual.)
The link at the bottom of the homepage offer an opportunity to learn more about the system:
User Manual: an online version of the tool’s documentation, for easy reference.
2.1 Icons
Icons are used to allow features to be quickly identified within the system. Icons are used consistently throughout to identify functions, visualisations and information types. They fall into the following five categories (Figure 3Figure 3):
D4.7 Student Model Tools Code Release and Documentation
The open learner model (OLM) displays the current state of student competencies taking into account all the formative assessment evidence in the database. The learner model updates its presentation each time new inferences about knowledge or competencies are available.
Figure 4: OLM browser
Initially all information about all students’ current learning is presented. In order to narrow down the scope of information presented on the screen, filters can be added using the button as illustrated in Figure 4. The filter may be configured to show information for a specific group and/or student and/or activity and/or competency and/or information source and/or tag. Each time an item in the filter is changed, the page content updates. The OLM content may also be updated using the refresh button and filters may be set to their default state using the reset button.
D4.7 Student Model Tools Code Release and Documentation
The lower part of the page shows the open learner model. Alternative visualisations are available as tabs, and these may be turned on or off using the preferences page (See Section 8). The visualisations show subtly different aspects of the information, visualised as skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency treemaps and competency networks. All visualisations are available to view by clicking on the name or icon of the visualisation, which is then displayed in the area below the labels.
3.1 Filter
The filter mechanism (left of Figure 5) is used to narrow down the information presented. The filters that are applied are summarised in the top-centre of the screen above the visualisation labels. The criteria chosen in the filters can be removed all at once by clicking the reset button in the right corner of this section, or in each individual filter using the reset buttons present in each filter tab (opposite the tab label).
Figure 5: View the OLM using Filters
Several types of filters are available:
Information Sources: This allows the rage of data sources to be specified: teacher assessments, student self-assessments, peer assessments, or data from other pieces of software (automated sources). Additionally an option to break down the visualisations by the source is also available; by setting this to ON the single visualisations will be broken down into four (one for each data source represented by the colour adjacent to the label for that source).
Tags: When evidence is submitted to the learner model, it may be optionally tagged. Selecting a tag from the list will build the learner model from all information that matches the tag.
Groups, Students, Competencies, and Activities: The last four filters allow specific areas of data to be filtered out. Clicking on one of the filters will show a list of all the possible options. Clicking on one of the options applies the criterium to the filter. Only one criterium can be shown at a time when using the filter.
D4.7 Student Model Tools Code Release and Documentation
The open learner model part of the OLM browser is visualised in up to eight different ways: skill meters, table, smiley faces, histogram, word cloud, radar plots, competency treemap, and competency network. Visualisations are presented in a tabbed structure (Figure 6). Clicking on the name of the visualisation loads the new tab. The range of visualisations available may be customised using the preferences page (See section 8). The information in the tab automatically updates when the filter is amended.
Skill Meters Table
Smiley Faces Histogram
Word Cloud Radar Plot
Tree Map Network
Figure 6: OLM list of visualisation links
Each visualisation section shows a set of student competencies in terms of groups and students, competencies, and activities.
3.2.1 Skill Meter Visualisation
Skill meters show the level of competency (Figure 7). Blue is inferred to be level of confirmed competency or understanding, whilst grey indicates lack of competency or understanding. White shows that there is insufficient evidence in the learner model. The larger the section of blue, the stronger the understanding. The scale is continuous. The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the skill meter loads the evidence page (Section 4.1) and clicking on the gears icon to the left of the skill meter loads the description of the modelling process (Section 4.2).
Figure 7: OLM skill meter visualisation
D4.7 Student Model Tools Code Release and Documentation
The table representation (Figure 8) shows student understanding or competency on a 5-point scale of “very weak” to “very strong”. A circle is placed in the category in which the data item falls. The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the label loads the evidence page (Section 4.1) and clicking on the gears icon to the left of the label loads the description of the modelling process (Section 4.2).
Figure 8: OLM table visualisation
3.2.3 Smiley Faces Visualisation
The Smiley Faces representation (Figure 9) shows student competency using a emotion-based metaphor; the larger the smile, the stronger the understanding, the more confused the face, the weaker the understanding. The 5-point scale used is the same as the table visualisation (see above section). The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the smiley face loads the evidence page (Section 4.1) and clicking on the gears icon to the left of the smiley face loads the description of the modelling process (Section 4.2).
D4.7 Student Model Tools Code Release and Documentation
The histogram visualisation gives an overall shape to the distribution of information (Figure 10). The horizontal placing of an item shows the strength of understanding or competency. The left hand side of the scale is weak and the right hand side of the scale is strong. The vertical placement of items does not represent anything. Clicking the list icon in the histogram item loads the evidence page (Section 4.1) and clicking on the gears icon loads the description of the modelling process (Section 4.2).
Figure 10: OLM histogram visualisation
D4.7 Student Model Tools Code Release and Documentation
Two word clouds are presented for each type of information as well as an item exclusion list below. The darker section shows stronger items (the larger the text, the better understood the item is) and the lighter section shows weaker items (the larger the text, the weaker the understanding or competency of the given item). Clicking on the word cloud item loads the evidence page (Section 4.1).
Figure 11: OLM word cloud visualisation
3.2.6 Radar Plot Visualisation
The Radar Plot visualisation (Figure 12), each item is represented by an axis. The further away from the centre the plotted points are, the stronger the competency. The number of axis is dependent on the number of items to be represented. By moving the mouse over one of the points, the representation for this particular source of information is highlighted and comes to the foreground.
Figure 12: OLM radar plot visualisation
D4.7 Student Model Tools Code Release and Documentation
The tree map (Figure 13) is a presentation of all the competencies currently involved in student learning. Clicking on a box in the competency tree loads the sub-competencies associated with it. Hovering over an item on the treemap shows the value of the learner model. The size of the box represents the extent to which the competency is held. To return to the previous level click the “up one level” button, or right click anywhere on the treemap. The “squares”, “strips” and “slices” change the way in which the boxes tessellate.
Figure 13: OLM competency treemap visualisation
3.2.8 Competency Network Visualisation
The competency network (Figure 14) is a presentation of all the competencies currently involved in student learning. Clicking on a bubble in the competency network shows/hides the sub-competencies associated with it. The size and color of the bubble represents the extent to which the competency is held (the bigger and greener the bubble is, the higher level of learning has been achieved). The relationship between competencies and sub-competencies is show by lines between the bubbles. The bubbles can be moved using a “drag and drop” movement in order to see better the structure when things are clustered.
D4.7 Student Model Tools Code Release and Documentation
The OLM visualisations as presented in Section 3.2 show an overview of level of competency or understanding. Two features are available to gain more understanding of the way the levels were calculated from the different sources of information: viewing the evidence list (Section 4.1) and description of the modelling process / how evidence is combined (Section 4.2).
4.1 View Evidence List
Clicking the list icons within the visualisations displays a popup that allows the teacher to drill down into the database to list all the items of evidence behind the particular part of the open learner model. Collated alongside the learner model evidence are textual statements about student strengths, guidance and difficulties, with reference to student competencies. The text is not transformed by the modelling process (as the numerical pieces of evidence are). Evidence may originate from a variety of sources, including teacher, student and peer assessment, and external data sources. The summary box at the top of the page shows the information that is presented, pressing the OK button located at the bottom of the popup window will close the Evidence and Guidance window. The most recent information is at the top. The table is further searchable using a text box that allows any phrase to be entered, these are located at the top of each column.
D4.7 Student Model Tools Code Release and Documentation
Figure 15: OLM browser evidence and guidance layer
The information is presented in a tabular form. The date, student name, activity and competency are shown, to contextualise the piece of information. The information is colour coded and is categorised as:
Teacher entered assessment value (contributes to learner model)
Student entered self-assessment value (contributes to learner model)
Peer entered assessment value (contributes to learner model)
Automated source entered assessment value (contributes to learner model)
Student strength (teacher entered) (collated alongside the learner model)
Student strength (student entered) (collated alongside the learner model)
Student strength (peer entered) (collated alongside the learner model)
Student strength (proposed by an external source) (collated alongside the learner model)
Student guidance (teacher entered) (collated alongside the learner model)
Student difficulty (student entered) (collated alongside the learner model)
Suggestion for the student (peer entered) collated alongside the learner model)
Student guidance (proposed by an external source) (collated alongside the learner model)
4.2 Description of the Learner Model Calculation
The description of the modelling process may be accessed through the gears icon located for each visualisation (see Section 3.2). A dialogue box opens up on top of the visualisation page that displays a visual of how the learner model value was calculated for a specific item. Clicking the green “+” buttons expands the next level down of information. The red “-“ buttons may be used to collapse a section. This screen provides a visual link between the learner model value and the underlying data.
D4.7 Student Model Tools Code Release and Documentation
Starting with the value presented, the modelling process may be drilled down into to see the components of the calculation and how the competencies are linked to evidence that is captured over time. For example in Figure 16, the competency level of the C21 Skills: Meetings (Next-TELL)id:259 is calculated as the mean of the values of all the data sources in this group (which amounts here to 4 students and 1 peer).
Evidence may originate from a variety of sources, including teacher-, student- and peer- assessment, and external data sources. The evidence section shows a list of all information sources entered for this competency, with the most recent information is at the bottom of each evidence section. Evidence is weighted according to time; more recent evidence is given a greater weighting.
Figure 16: learner model calculation visualisation
5 Add Evidence
Evidence may be added to the learner model using the webform, a Google spreadsheet, or by another piece of technology using the OLM API.
5.1 Webform
Assessments of student competency may be submitted on an individual basis for each activity. Student competency may be assessed multiple times for each activity, with the evidence being modelled over time.
1. Locate the activity and then the student, using the tree section at the top of the page. Click the “add info” button next to the name of the student for whom you wish to add information.
D4.7 Student Model Tools Code Release and Documentation
2. A list of all the competencies associated with the activity is presented. For each competency the teacher may formatively assess the student using the scale of stars, may give feedback on their strengths as text, and may enter guidance information for the student, also as text. All fields are optional.
3. A tag may be associated with the data entry. 4. Learning based artefacts may be optionally added. This may be, for example, a Google Document or
an e-portfolio entry. The URL for the artefact should be entered which is then automatically loaded. 5. The “submit information to the learner model” button should be clicked when all data is entered.
Figure 17: add evidence using the web form
5.2 Google Spreadsheet
Assessments of student competency may be submitted for each activity. The spreadsheet allows information to be entered for all students within the activity at the same time. A copy of the data that is entered is saved to the teacher’s Google Drive account for their reference. Student understanding or competency may be assessed multiple times for each activity.
D4.7 Student Model Tools Code Release and Documentation
Figure 18: add evidence using a Google spreadsheet
1. Locate the activity using the tree section at the top of the page. Click the “add info” button next to the name of the activity. The OLM will request access to your Google Drive account, so that it may record the spreadsheet and embed it within the lower section of the page.
2. A list of all students and all competencies in the activity is presented in a tablular form. For each competency the teacher may rate the student’s competency on a scale of 1 to 10, may feed back on the student’s strengths as text, and may enter guidance information for the student, also as text. A tag may be associated with the data entry, and a URL linking to a learning based artefact may be entered. All fields are optional.
3. The “submit information to the learner model” button should be clicked when all data is entered.
6 Updates The number of unread updates is always present in the menu bar at the top of the screen (Figure 19).
Figure 19: notifications in the header bar
Clicking the Updates tab shows all unread notifications as a list (Figure 20). Updates are automatically marked as read upon viewing the page. Once updates are marked as read, they will no longer appear highlighted in red.
D4.7 Student Model Tools Code Release and Documentation
Teachers may participate in group discussion with students about competencies within current learning provision and information presented in the open learner model. The discussion feature may be selected from the menu bar, or home page.
From the list of discussions on the left of the screen, select a discussion. This will display all the competencies associated with the discussion on the right side of the screen. Additionally the current discussion will appear in the section in the middle of the page. The content of the discussion will automatically update when a new post is made (no need to refresh the page). To submit a post to the discussion, type the message in the box at the very bottom of the page and click the “post” button or press the enter key on your keyboard. The discussion tool may be used in real time, with multiple parties contributing at once, or may be used across a large time period, with participants contributing at different points in time. The content of the discussion remains even when the page is closed. When content is contributed at a different point in time from when the teacher was logged on, a red speech bubble will appear next to the discussion with a number of unread messages out of the total.
D4.7 Student Model Tools Code Release and Documentation
The eight visualisation methods detailed in Section 3.2 show subtly different aspects of the information (skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency tree maps and competency networks). These visualisations may be turned on or off using the preferences page, accessed on the top right of the menu bar (Figure 22). If the box is ticked, the visualisation is active. Additionally the colour for the data sources may be set here allowing for users to ease of use.
Figure 22: OLM visualisation preferences page
D4.7 Student Model Tools Code Release and Documentation
Open learner models (OLMs) show learners’ current understanding or competencies. The visualisation of the learner model adaptively updates, to present the current competencies or knowledge state of the learner. Learner models are built up over time and information may be gathered from a variety of sources. Information (inferences about student understanding) may come from:
Information is available at different levels of abstraction: competencies, individual students, groups, activities, units of work, or subjects. The information in the learner model is available for inspection by teachers, learners, and their peers. Several types of visualisation are provided to support inspection of the learner model (Figure 1).
D4.7 Student Model Tools Code Release and Documentation
User accounts for the CAS system may be registered at http://sandbox.next-tell.eu/register/
Figure 1: OLM visualisations (summary)
2 Features and the Homepage
Upon login students are directed towards the homepage (Figure 2). This page provides navigation to each of the main features available. The homepage can be returned to at any point by clicking the “home” button in the header bar, which is always present at the top of the screen.
Using the buttons to the left of the page, or the links in the header bar, students can access the main features of the tool:
Open Learner Model: the student’s open learner model is visualised as skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency treemaps and competency networks. The open learner model page displays information on all current competencies, with the option to narrow the information down to specific, groups, activities, and sub-competencies. This page also contains links to a description of the modelling process, lists of supporting evidence, and textual statements that describe student strengths and areas of guidance. Information in the learner model can come from multiple sources, including student self-assessments, peer-assessments, and other Next-TELL tools. (See Sections 3 and 4.)
Browse peers’ learner models: students may allow each other to see information in parts of their learner model, and anonymously or with their name. This page allows students to define who can see information in their model, and also inspect information in other students’ models. (See Section 5)
Make a self-assessment using a web form: Students may select an activity and give a self-assessment of their understanding or competencies contained within it. (See Section Error! Reference source not found.)
Make a peer-assessment using a web form: Student may select an activity and give a peer-assessment of their peer’s understanding or competencies contained within it. (See Section Error! Reference source not found.)
View OLM notifications: view notifications of updates to the learner model made by teachers and peers, and also updates from automated data sources. (Section 7)
Participate in discussion: participate in discussion with other students and teachers about information presented in their open learner model and competencies in learning. (Section 8)
Using the menu on the top right of the screen students may specify preferences:
Preferences: select one or several visualisations to use in the open learner model. (Section 9)
2.1 Icons
Icons are used to allow features to be quickly identified within the system. Icons are used consistently throughout to identify functions, visualisations and information types. They fall into the following five categories (Figure 3Figure 3):
D4.7 Student Model Tools Code Release and Documentation
3 Open Learner Model The open learner model (OLM) displays the current state of student competencies taking into account all the formative assessment evidence in the database. The learner model updates its presentation each time new inferences about knowledge or competencies are available.
Figure 4: OLM browser
Initially all information about all students’ current learning is presented. In order to narrow down the scope of information presented on the screen, filters can be added using the button as illustrated in Figure 4. The filter may be configured to show information for a specific group and/or student and/or activity and/or competency and/or information source and/or tag. Each time an item in the filter is changed, the page content updates. The OLM content may also be updated using the refresh button and filters may be set to their default state using the reset button.
D4.7 Student Model Tools Code Release and Documentation
The lower part of the page shows the open learner model. Alternative visualisations are available as tabs, and these may be turned on or off using the preferences page (See Section 8). The visualisations show subtly different aspects of the information, visualised as skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency treemaps and competency networks. All visualisations are available to view by clicking on the name or icon of the visualisation, which is then displayed in the area below the labels.
3.1 Filter
The filter mechanism (left of Figure 5) is used to narrow down the information presented. The filters that are applied are summarised in the top-centre of the screen above the visualisation labels. The criteria chosen in the filters can be removed all at once by clicking the reset button in the right corner of this section, or in each individual filter using the reset buttons present in each filter tab (opposite the tab label).
Figure 5: View the OLM using Filters
Several types of filters are available:
Information Sources: This allows the rage of data sources to be specified: teacher assessments, student self-assessments, peer assessments, or data from other pieces of software (automated sources). Additionally an option to break down the visualisations by the source is also available; by setting this to ON the single visualisations will be broken down into four (one for each data source represented by the colour adjacent to the label for that source).
Tags: When evidence is submitted to the learner model, it may be optionally tagged. Selecting a tag from the list will build the learner model from all information that matches the tag.
Groups, Students, Competencies, and Activities: The last four filters allow specific areas of data to be filtered out. Clicking on one of the filters will show a list of all the possible options. Clicking on one of the options applies the criterium to the filter. Only one criterium can be shown at a time when using the filter.
D4.7 Student Model Tools Code Release and Documentation
The open learner model part of the OLM browser is visualised in up to eight different ways: skill meters, table, smiley faces, histogram, word cloud, radar plots, competency treemap, and competency network. Visualisations are presented in a tabbed structure (Figure 6). Clicking on the name of the visualisation loads the new tab. The range of visualisations available may be customised using the preferences page (See section 8). The information in the tab automatically updates when the filter is amended.
Skill Meters Table
Smiley Faces Histogram
Word Cloud Radar Plot
Tree Map Network
Figure 6: OLM list of visualisation links
Each visualisation section shows a set of student competencies in terms of groups and students, competencies, and activities.
3.2.1 Skill Meter Visualisation
Skill meters show the level of competency (Figure 7). Blue is inferred to be level of confirmed competency or understanding, whilst grey indicates lack of competency or understanding. White shows that there is insufficient evidence in the learner model. The larger the section of blue, the stronger the understanding. The scale is continuous. The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the skill meter loads the evidence page (Section 4.1) and clicking on the gears icon to the left of the skill meter loads the description of the modelling process (Section 4.2).
Figure 7: OLM skill meter visualisation
3.2.2 Table Visualisation
The table representation (Figure 8) shows student understanding or competency on a 5-point scale of “very weak” to “very strong”. A circle is placed in the category in which the data item falls. The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the label loads the
D4.7 Student Model Tools Code Release and Documentation
evidence page (Section 4.1) and clicking on the gears icon to the left of the label loads the description of the modelling process (Section 4.2).
Figure 8: OLM table visualisation
3.2.3 Smiley Faces Visualisation
The Smiley Faces representation (Figure 9) shows student competency using a emotion-based metaphor; the larger the smile, the stronger the understanding, the more confused the face, the weaker the understanding. The 5-point scale used is the same as the table visualisation (see above section). The indentation reflects the structure of the content (e.g. sub-competencies). Clicking the list icon to the left of the smiley face loads the evidence page (Section 4.1) and clicking on the gears icon to the left of the smiley face loads the description of the modelling process (Section 4.2).
Figure 9: OLM smiley faces visualisation
D4.7 Student Model Tools Code Release and Documentation
The histogram visualisation gives an overall shape to the distribution of information (Figure 10). The horizontal placing of an item shows the strength of understanding or competency. The left hand side of the scale is weak and the right hand side of the scale is strong. The vertical placement of items does not represent anything. Clicking the list icon in the histogram item loads the evidence page (Section 4.1) and clicking on the gears icon loads the description of the modelling process (Section 4.2).
Figure 10: OLM histogram visualisation
3.2.5 Word Cloud Visualisation
Two word clouds are presented for each type of information as well as an item exclusion list below. The darker section shows stronger items (the larger the text, the better understood the item is) and the lighter section shows weaker items (the larger the text, the weaker the understanding or competency of the given item). Clicking on the word cloud item loads the evidence page (Section 4.1).
D4.7 Student Model Tools Code Release and Documentation
The Radar Plot visualisation (Figure 12), each item is represented by an axis. The further away from the centre the plotted points are, the stronger the competency. The number of axis is dependent on the number of items to be represented. By moving the mouse over one of the points, the representation for this particular source of information is highlighted and comes to the foreground.
Figure 12: OLM radar plot visualisation
D4.7 Student Model Tools Code Release and Documentation
The tree map (Figure 13) is a presentation of all the competencies currently involved in student learning. Clicking on a box in the competency tree loads the sub-competencies associated with it. Hovering over an item on the treemap shows the value of the learner model. The size of the box represents the extent to which the competency is held. To return to the previous level click the “up one level” button, or right click anywhere on the treemap. The “squares”, “strips” and “slices” change the way in which the boxes tessellate.
Figure 13: OLM competency treemap visualisation
3.2.8 Competency Network Visualisation
The competency network (Figure 14) is a presentation of all the competencies currently involved in student learning. Clicking on a bubble in the competency network shows/hides the sub-competencies associated with it. The size and color of the bubble represents the extent to which the competency is held (the bigger and greener the bubble is, the higher level of learning has been achieved). The relationship between competencies and sub-competencies is show by lines between the bubbles. The bubbles can be moved using a “drag and drop” movement in order to see better the structure when things are clustered.
D4.7 Student Model Tools Code Release and Documentation
The OLM visualisations as presented in Section 3.2 show an overview of level of competency or understanding. Two features are available to gain more understanding of the way the levels were calculated from the different sources of information: viewing the evidence list (Section 4.1) and description of the modelling process / how evidence is combined (Section 4.2).
4.1 View Evidence List
Clicking the list icons within the visualisations displays a popup that allows the teacher to drill down into the database to list all the items of evidence behind the particular part of the open learner model. Collated alongside the learner model evidence are textual statements about student strengths, guidance and difficulties, with reference to student competencies. The text is not transformed by the modelling process (as the numerical pieces of evidence are). Evidence may originate from a variety of sources, including teacher, student and peer assessment, and external data sources. The summary box at the top of the page shows the information that is presented, pressing the OK button located at the bottom of the popup window will close the Evidence and Guidance window. The most recent information is at the top. The table is further searchable using a text box that allows any phrase to be entered, these are located at the top of each column.
D4.7 Student Model Tools Code Release and Documentation
Figure 15: OLM browser evidence and guidance layer
The information is presented in a tabular form. The date, student name, activity and competency are shown, to contextualise the piece of information. The information is colour coded and is categorised as:
Teacher entered assessment value (contributes to learner model)
Student entered self-assessment value (contributes to learner model)
Peer entered assessment value (contributes to learner model)
Automated source entered assessment value (contributes to learner model)
Student strength (teacher entered) (collated alongside the learner model)
Student strength (student entered) (collated alongside the learner model)
Student strength (peer entered) (collated alongside the learner model)
Student strength (proposed by an external source) (collated alongside the learner model)
Student guidance (teacher entered) (collated alongside the learner model)
Student difficulty (student entered) (collated alongside the learner model)
Suggestion for the student (peer entered) collated alongside the learner model)
Student guidance (proposed by an external source) (collated alongside the learner model)
4.2 Description of the Learner Model Calculation
The description of the modelling process may be accessed through the gears icon located for each visualisation (see Section 3.2). A dialogue box opens up on top of the visualisation page that displays a visual of how the learner model value was calculated for a specific item. Clicking the green “+” buttons expands the next level down of information. The red “-“ buttons may be used to collapse a section. This screen provides a visual link between the learner model value and the underlying data.
D4.7 Student Model Tools Code Release and Documentation
Starting with the value presented, the modelling process may be drilled down into to see the components of the calculation and how the compencies are linked to evidence that is captured over time.
Evidence may originate from a variety of sources, including teacher-, student- and peer- assessment, and external data sources. The evidence section shows a list of all information sources entered for this competency, with the most recent information is at the bottom of each evidence section. Evidence is weighted according to time; more recent evidence is given a greater weighting.
Figure 16: learner model calculation visualisation
5 Add Evidence
Evidence may be added to the learner model using the webform, in addition to other pieces of technology using the OLM API.
5.1 Self-Assessment
Students may submit self-assessments about their understanding within an activity, or their level of competency, if permitted to do so by the teacher. Student competency may be assessed multiple times for each activity, with the evidence being modelled over time.
1. Locate the activity and then the student, using the tree section at the top of the page. Click the “add info” button next to the name of the student for whom you wish to add information.
2. A list of all the competencies associated with the activity is presented. For each competency the teacher may formatively assess the student using the scale of stars, may give feedback on their
D4.7 Student Model Tools Code Release and Documentation
strengths as text, and may enter guidance information for the student, also as text. All fields are optional.
3. A tag may be associated with the data entry. 4. Learning based artefacts may be optionally added. This may be, for example, a Google Document or
an e-portfolio entry. The URL for the artefact should be entered which is then automatically loaded. 5. The “submit information to the learner model” button should be clicked when all data is entered.
Figure 17: add evidence using the web form
5.2 Peer-Assessment
Students may submit self-assessments about their understanding within an activity, if permitted to do so by the teacher.
1. The student is required to locate the activity using the tree section at the top of the page. Activities will only appear here if peer- assessment for that activity is permitted. The student should then select a peer and click the “add info” button next to the peer’s name.
2. A list of all the competencies associated with the activity is presented. For each competency the student may add a formative peer-assessment using the scale of stars, may give feedback on their strengths as text, and may make suggestions, also as text. All fields are optional.
3. The student may associate a tag with the data entry. 4. The student may optionally include a learning based artefact that supports their peer assessment. This
may be, for example, a Google Document or an e-portfolio entry. The URL for the artefact should be entered, which is then automatically loaded.
5. The student should click the “submit information to the learner model” button when they are happy with their peer-assessment.
D4.7 Student Model Tools Code Release and Documentation
Figure 18: add evidence using a the peer assessment form
6 Updates The number of unread updates is always present in the menu bar at the top of the screen (Figure 19).
Figure 19: notifications in the header bar
Clicking the Updates tab shows all unread notifications as a list (Figure 20). Updates are automatically marked as read upon viewing the page. Once updates are marked as read, they will no longer appear highlighted in red.
D4.7 Student Model Tools Code Release and Documentation
Students may participate in group discussion with students about competencies within their current learning and information presented in the open learner model. The discussion feature may be selected from the menu bar, or home page.
From the list of discussions on the left of the screen, select a discussion. This will display all the competencies associated with the discussion on the right side of the screen. Additionally the current discussion will appear in the section in the middle of the page. The content of the discussion will automatically update when a new post is made (no need to refresh the page). To submit a post to the discussion, type the message in the box at the very bottom of the page and click the “post” button or press the enter key on your keyboard. The discussion tool may be used in real time, with multiple parties contributing at once, or may be used across a large time period, with participants contributing at different points in time. The content of the discussion remains even when the page is closed. When content is contributed at a different point in time from when the teacher was logged on, a red speech bubble will appear next to the discussion with a number of unread message out of the total.
D4.7 Student Model Tools Code Release and Documentation
The eight visualisation methods detailed in Section 3.2 show subtly different aspects of the information (skill meters, tables, smiley faces, histograms, word clouds, radar plots, competency tree maps and competency networks). These visualisations may be turned on or off using the preferences page, accessed on the top right of the menu bar (Figure 22). If the box is ticked, the visualisation is active. Additionally the colour for the data sources may be set here allowing for users to ease of use.
D4.7 Student Model Tools Code Release and Documentation
The Next-TELL OLM database consists of the following tables. Table 29 specifies the key attributes that are used to link the database tables. Table 30 to Table 57 specify the field names, variable types and purpose of each field in each table. For further information about the purpose of each table, please refer to Section 5.6 of the main report.
Table 29: key fields used to link the database tables.
ActivityID Integer Identification number for the activity
ActivityName VarChar Name of the activity
TeacherID Integer Identificiation number for the teacher
ActivityInfluence Double Weighting that specifies the activity’s relative influence to other activities, as part of the modelling process. (Value between 0.0 and 10.0)
UnitID Integer Identification number of the unit of work
SubmissionAllowed VarChar Whether student self-assessments are permitted for this activity (always set to 1)
AllowView Integer Whether peer model viewing is allowed for this activity (not used in final release)
D4.7 Student Model Tools Code Release and Documentation
EvidenceSourceID Integer Identification number of the type of external source (through API)
EvidenceSourceClientID Integer Identification number of the individiual source connection (through API)
ContributionFromAnyone Integer Not used.
PeerSubmissionAllowed Integer Students may submit peer assessments for this activity (always set to 1)
Position Integer The sequence that activities appear in when ordered in a list.
Table 31: database table “ActivityClass”.
Field Data Type Description
ActivityID Integer Identification number for the activity
ClassID Integer Identification number for the class group
Table 32: database table “ActivityTemplate”.
Field Data Type
Description
ActivityID Integer Identification number for the activity
SourceInfluence Double Weighting that specifies the competency’s relative influence to other competencies, as part of the modelling process (Value between 0.0 and 10.0)
Weighting Double Depreciation factor that specified the level of influence that new information has, as part of the modelling process
ActivityTemplateID Integer Identification number of this activitytemplate entry
CompetenceID Integer Identification number of the competency
EvidenceSourceClientID Integer Identification number of the individual source connection (through API)
EvidenceSourceID Integer Identification number of the type of external source (through API)
Position Integer The sequence that activitytemplate items appear in when ordered in a list
Table 33: database table “CASID_UserID”.
Field Data Type Description
CASID VarChar CAS ID of the user (as specified during authentication and used as key in API)
UserID Integer Identification number of the user (teacher or student)
Table 34: database table “Chat”.
Field Data Type Description
MessageID Integer Identification number of the discussion message
StudID Integer Identification number of the user who posted the message (teacher or student)
ClassID Integer Identification number of the classgroup (to identifiy the discussion scope)
CompetencyID Integer Identification number of the competency (to identify the discussion scope)
Message Long VarChar Discussion item (message)
D4.7 Student Model Tools Code Release and Documentation
Time TimeStamp Time at which the discussion item was posted
KLCorr Double State of the learner model for the particular competency, for the student, in the group
Table 35: database table “ChatRead”.
Field Data Type
Description
StudID Integer Identification number of the user who is reading the message (teacher or student)
ClassID Integer Identification number of the classgroup (to identify the discussion scope)
CompetencyID Integer Identification number of the competency (to identify the discussion scope)
LastCount Integer Number of read items for this classgroup/competency/user combination last time the discussion was loaded (used to determine if the discussion is updated)
Table 36: database table “ClassGroup”.
Field Data Type Description
ClassID Integer Identification number of the classgroup
ClassName VarChar Name of the classgroup
TeacherID Integer Identification number for the teacher who created the classgroup
Position Integer The sequence in which classgroups appear when ordered in a list
Table 37: database table “ColourDefinition”.
Field Data Type Description
StudID Integer Identification number of the user (either student or teacher)
Val_Student VarChar Hexidecimal colour used to represent information from students
Val_Teacher VarChar Hexidecimal colour used to represent information from teachers
Val_Peer VarChar Hexidecimal colour used to represent information from peers
Val_Automated VarChar Hexidecimal colour used to represent information from automated sources
Str_Student VarChar Hexidecimal colour used to represent information from groups
Str_Teacher VarChar Hexidecimal colour used to represent information from competencies
Str_Peer VarChar Hexidecimal colour used to represent information from activities
Str_Automated VarChar Hexidecimal colour used to represent information of other types
GUI_Student VarChar Hexidecimal colour used to represent information from aggregated sources
GUI_Teacher VarChar Not used
GUI_Peer VarChar Not used
GUI_Automated VarChar Not used
Table 38: database table “Competencies”.
D4.7 Student Model Tools Code Release and Documentation
UnitID Integer Identification number of the unit of work
UnitName VarChar Name of the unit of work
CompetencyID Integer Identification number of the competency
CompetencyPath VarChar Name of the competency’s super-competency (recursive to the root node)
CompetencyName VarChar Name of the competency
UnitInfluence Double Weighting that specifies the unit of work’s relative influence to other units of work, as part of the modelling process (Value between 0.0 and 10.0)
ActivityInfluence Double Weighting that specifies the activity’s relative influence to other activities, as part of the modelling process (Value between 0.0 and 10.0)
CompetencyInfluence Double Weighting that specifies the competency’s relative influence to other competencies, as part of the modelling process (Value between 0.0 and 10.0)
Depreciation Double Depreciation factor that specified the level of influence that new information has, as part of the modelling process
Artefact VarChar URL that links to a supporting artefact for the piece of evidence
Tags VarChar Any tags associated with the piece of evidence (delimited by space)
KLCorr Double The extent to which the competency is held (value between 0.0 and 1.0)
KLProb Double The extent to which the competency is not held (not misconception) (value between 0.0 and 1.0)
KLMisc Double The extent to which the competency demonstrates a misconception (value between 0.0 and 1.0)
Approved Integer Whether the piece of information should contribute to the modelling process
TeacherRead Integer Whether the teacher has viewed the notification that part of the model has been updated as a result of this piece of evidence
StudentRead Integer Whether the student has viewed the notification that part of the model has been updated as a result of this piece of evidence
Table 45: database table “Search_TextualRaw”.
Field Data Type Description
RawID Integer Identification number of the evidence entry
Time TimeStamp Time of the evidence entry
StudID Integer Idenfitication number of the student
Forename VarChar Forename of the student
Surname VarChar Surname of the student
EvidenceSourceID Integer Identification number of the evidence source
EvidenceSourceName VarChar Name of the evidence source
ContributorID Integer Identification number of the user who added the information (teacher,
D4.7 Student Model Tools Code Release and Documentation
ContributorForename VarChar Forename of the user who added the information
ContributorSurname VarChar Surname of the user who added the information
ContributorType VarChar Type of user who added the information (student, teacher, peer etc.)
TeacherID Integer Identification number of the teacher who created the group that information is associate with.
TeacherForename VarChar Forename of the teacher who created the group the information is assocaited with
TeacherSurname VarChar Surname of the teacher who created the group the information is associated with
ClassID integer Identification number of the class group
ClassName VarChar Name of the class group
ActivityID Integer Identification number of the activity
ActivityName VarChar Name of the activity
ActivityTemplateID Integer Identification number of the activity template (specifies competency, activity relationship)
SubjectID Integer Identification number of the subject
SubjectName VarChar Name of the subject
UnitID Integer Identification number of the unit of work
UnitName VarChar Name of the unit of work
CompetencyID Integer Identification number of the competency
CompetencyPath VarChar Name of the competency’s super-competency (recursive to the root node)
CompetencyName VarChar Name of the competency
UnitInfluence Double Weighting that specifies the unit of work’s relative influence to other units of work, as part of the modelling process (Value between 0.0 and 10.0)
ActivityInfluence Double Weighting that specifies the activity’s relative influence to other activities, as part of the modelling process (Value between 0.0 and 10.0)
CompetencyInfluence Double Weighting that specifies the competency’s relative influence to other competencies, as part of the modelling process (Value between 0.0 and 10.0)
Depreciation Double Depreciation factor that specified the level of influence that new information has, as part of the modelling process
Artefact VarChar URL that links to a supporting artefact for the piece of evidence
Tags VarChar Any tags associated with the piece of evidence (delimited by space)
StudID Integer Identification number for the student
SubjectID Integer Identification number for the subject
Table 51: database table “StudentTeacher”.
Field Data Type Description
StudID Integer Identification number for the student
TeacherID Integer Identification number for the teacher
Table 52: database table “StudentUnit”.
Field Data Type Description
StudID Integer Identification number for the student
UnitID Integer Identification number for the unit of work
Table 53: database table “Subject”.
Field Data Type Description
SubjectID Integer Identification number for the subject
SubjectName VarChar Name of the subject
TeacherID Integer Identification number of the teacher
Position Integer The sequence that subjects appear in when ordered in a list.
Table 54: database table “TeacherCompetency”.
Field Data Type Description
CompetencyID Integer Identification number of the competency
TeacherID Integer Identification number of the teacher
Table 55: database table “Unit”.
Field Data Type
Description
UnitID Integer Identification number of the unit of work
UnitName VarChar Name of the unit of work
SubjectID Integer Identification number of the subejct
TeacherID Integer Identification number of the teacher
Influence Double Relative influence that the unit has when combined with other units of work, as part of the modelling process (value between 0.0 and 10.0)
Position Integer The sequence that units of work appear in when ordered in a list.
Table 56: database table “UnitClass”.
Field Data Type Description
UnitID Integer Identification number of the unit of work
D4.7 Student Model Tools Code Release and Documentation
All methods are not protected by the CAS system. In its place the shared secret (“n!e@x3t^t8e2l!l”) and CAS userid must be specified with each request.
To access API functions, prefix the method name with http://eeevle.bham.ac.uk/nexttell-cas/api/
All functions return XML as their response type. The XML tag will be the name of the data element type. E.g.
<competencyParent>
<competencyId>330</competencyId>
<competencyName>Kommunikasjon</competencyName>
</competencyParent>
Informational messages will be returned with the tag <message> E.g.
<message>ERROR: Please enter a name for the competency.</message>
This document is split up into the following sections:
Competency Database (context definition)
Student/Group Database (context definition)
Curriculum Database (context definition)
Activity Creation (collection node)
Add evidence
Read evidence
The following methods contribute to the configuration and setup of the software that hosts the OLM.
Competency Database
This is the database of what we want to model. API methods are as per Table 58.
competencyId (int) competencyParentId is the ID of the competency that the new competency should be listed under. For the root node (“all competencies”) competencyParentId is 0.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/SetCompetencyName?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&competencyId=592&newCompetencyName=api_test
<message>ok</message>
Delete DeleteCompetency competencyId Ok http://eeevle.bham.ac.uk/nexttell-cas/api/DeleteCompetency?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&competencyId=592
<message>ok</message>
Read GetCompetencyTreeForUser List of competencies for user. See example to the right.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/SetClassGroupName?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&classGroupId=95&newClassGroupName=api_test_2 <message>ok</message>
Delete DeleteClassGroup classGroupId Ok http://eeevle.bham.ac.uk/nexttell-cas/api/DeleteClassGroup?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&classGroupId=95 <message>ok</message>
Read GetStudentList List of all students in the database with Ids and name. See example to the right.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/AddStudentToClassGroup?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&studentid=1000&classgroupId=95 <message>ok</message>
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/RemoveStudentFromClassGroup?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&classgroupId=95&studentid=1000 <message>ok</message>
Curriculum Database (Excluding Activities)
This is the database how we structure the delivery of learning, for the purpose of contextualising activities. API methods are as per Table 60.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/SetSubjectName?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&subjectId=73&newSubjectName=api_test_2 <message>ok</message>
Delete DeleteSubject subjectId Ok http://eeevle.bham.ac.uk/nexttell-cas/api/DeleteSubject?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&subjectId=73 <message>ok</message>
activityId Activity influence is the relative influence the activity will have when combined with other activities in the same unit. By default set this to 5.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/SetActivityName?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&activityId=174&newActivityName=changed_activity_name
<message>ok</message>
Delete DeleteActivity activityId Ok http://eeevle.bham.ac.uk/nexttell-cas/api/DeleteActivity?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&activityId=174
<message>ok</message>
Read GetSubjectUnitAndActivityList Nested list of the curriculum database. See example to the right.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/AddCompetencyToActivity?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&competencyId=259&activityId=129 <message>ok</message>
Read GetCompetenciesForActivity activityId List of http://eeevle.bham.ac.uk/nexttell-
Ok http://localhost:8080/nexttell-cas/api/RemoveCompetencyFromActivity?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&activityId=129&competencyId=259 <message>ok</message>
Read GetClassesForActivity activityId List of class groups. See example to the right.
Ok http://eeevle.bham.ac.uk/nexttell-cas/api/RemoveClassGroupFromActivity?sharedsecret=n!e@x3t^t8e2l!l&casid=matt_teacher&activityId=129&classgroupId=72 <message>ok</message>
See below for truncated sample output (Figure 42):
The data from the database that is given for each node is:
id <activityid>, <competencyid>
name <activityname>, <competencyname>
knowledge level <knowledgelevel> a number between 0 (weak understanding) and 1 (strong understanding) summarising the modelled learner understanding of the given area.
visualisation <visualisation> the URL to where the visualisation of this information is available.
narrative <narrative> a block of HTML that describes what has taken place in the modelling process for the specified model value to be calculated. (This is work in progress.)
strengths <strengths> additional pieces of textual feedback that are tendered to the software by the a stakeholder in the learning process (at the moment the teacher). This data is not manipulated by the modelling process, it is just collated.
feedforward / guidance <feedforward> additional pieces of textual feedforward information that are tendered to the software by a stakeholder in the learning process (at the moment the teacher). This data is not manipulated by the modelling process, it is just collated.
artefacts <artefacts> the learning based artefacts that are associated with the inferences used to update the learner model.
Read GetOLMCompetencyTreeForUser Nested list of the competency database. See example to the right.
To support teachers’ formative assessment activities with regard to knowing and analysing their students’ personal conceptions about a particular topic. The methods and software are intended to help teachers who have been inspired by their class or colleagues; who want to explore what and how their students things about a particular topic; who want to share their exercise with colleagues in an easy and accessible way; and who want to know how to work with interactive visualizations and dashboards of their students’ learning data.
A Chemistry teacher, responsible for preparing a Year 8 class for their final year of secondary school study would like diagnose, identify and correct her students’ misconceptions about the elements of the periodic table . She also would like to use the exercise to adapt her teaching. This provides the kickoff for her repertory grid exercise. She uses the nextGrid method and software to plan, communicate, execute, and analyze the formative assessment activity as a repertory grid exercise.
She uses the RGFA software to design a formative assessment activity based on her pre-existing assumptions about her learners. These include a notion that Year 8 students do not have a holistic view of the elements of the periodic table in terms of their physical and chemical properties.Using the nextGrid method, she begins to plan her formative assessment activity as a repertory grid exercise. First, she identifies the key elements of the periodic table for the repertory grid exercise. She then creates triads of three elements each based on her pre-existing assumptions about her students’ knowledge and common misconceptions. She creates general instructions for the exercise and specific instruction for each triad of elements. She then creates a name for the competency/conception for each of the triad and identifies four keywords that would serve as evidence. She then shares the exercise with students in the class.
D4.7 Student Model Tools Code Release and Documentation
She introduces the repertory grid exercise in the class and allocates 30 minutes of class time for students to complete it individually. She provides a quick de-brief of the exercise results to the classroom using the web analytics of the RGFA tool with regard to the time taken by students for personal construct elicitation and elements rating phases of the exercise. She asks her students to reflect on their individual results and use CoNeTo to discuss their results with her during the week. She then uses the teaching analytics dashboard and web analytics of RGFA to diagnose and identify students’ misconceptions and conceptions. She uses this information to structure the discussions with the students in the CoNeTo tool. As a result of the nextGrid activity, she changes supplement her planned teaching activity with additional materials related to correcting the misconceptions. These findings are recorded and shared with colleagues using the nextGrid tool and can be used during the next school year of in other classrooms.
Trends
Teachers learn a new formative assessment method
Teachers learn to formulate their own competencies
Teachers learn a knowledge diagnostics method
Teachers work with interactive visualizations
Possible Approaches to Teaching and Assessment
nextGrid can be employed in
Assessing domain-generic as well domain-specific competencies
Applying new ICTs in classroom practice
Co-operative and collaborative learning
Peer and self-assessment of learners
Environment
F2F or Online or Blended administration
Individual or Group Exercise
Laptops, Tablets and/or Smartphones
People and Roles
Teachers: design a new repertory grid exercise or reuse an existing one, classroom/home orchestration of the exercise, data analysis, student discussions, and share exercises with their peers
Students: complete repertory grid exercise, reflection, discussion and create their own repertory grid exercises
Activities
Plan knowledge diagnostics activity
Create repertory grid exercise
Orchestrate the exercise in the classroom/home
Analyse the results using the teaching analytics dashboards
Discuss students’ results individually or collectively
Share practice with peers by publishing the repertory grid exercise in the public repository
D4.7 Student Model Tools Code Release and Documentation
This document provides the specification for web service that can be used to create new exercise for Repertory Grids for Formative Assessment (RGFA).
The API provides different methods that allow creation of RGFA exercise in multiple steps and a particular sequence has to be followed in order to complete exercise generation from remote applications.
This document is split up into the following sections:
Accessibility
Authentication
Methods and short description
Method Specifications
Code Examples
10.2.2 Accessibility
The web service can be accessed from the following URL.
http://cssl.cbs.dk/software/rgfaAPI
10.2.3 Authentication
API utilized the Central Authentication Service (CAS) in order to authenticate users and in case of valid credentials access to the service has been provided.
The credentials have to be included in the header of the SOAP request. Following syntax explain the method of assigning Authentication header to the service.
Following code assumes that reference to the service is already created and that reference is named RGFAWS.
RGFAWS.AuthKey authKey = new AuthKey();
authKey.Username = <CAS Username>;
authKey.Password = <CAS Password>;
10.3 Methods and short description : CreateExercise
CreateExercise
This method facilitates initialization of the creation process of a new exercise. The method accepts basic exercise information like name and labels related to the exercise and return the ID of the newly created exercise. The ID returned by the method should be used in the following steps for creating RGFA exercise.
10.4 Methods and short description : CreateElement
10.4.1 CreateElement
This method is used to add/create elements for the exercise that is generated through CreateExercise method. The method accepts the element details along with the exercise ID.
10.4.2 CreateElement : Method signature
Method name: CreateElement
IN params OUT params Comments
int ExerciseID Int (ElementID) ID gained through CreateExercise method.
String ElementName
String SortOrder
String Type [TEXT|Image|Video]
String URL In case of image and/or video
10.5 Methods and short description : CreateTriad
10.5.1 CreateTriad
This method is used to add/create Triads for the exercise that is instantiated through CreateExercise method at first and then elements are generated through CreateElement method. The method accepts the element details along with the exercise ID.
10.5.2 CreateTriad : Method signature
Method name: CreateTriad
IN params OUT params Comments
int ExerciseID Int (TriadID) ID gained through CreateExercise method.
int ElementID1 First element of triad, Element generated through CreateElement method.
int ElementID2
int ElementID3
D4.7 Student Model Tools Code Release and Documentation
10.6 Methods and short description : GetGridExercise
10.6.1 GetGridExercise
This method is a service method that accepts the ExerciseId and returns complete definition of the exercise. This method will provide the current state of the exercise.
10.6.2 GetGridExercise : Method signature
Method name: GetGridExercise
IN params OUT params Comments
int ExerciseID Object GridDef
Public class GridDef {
Public int ExerciseID;
Public string ExerciseName;
Public string Topic;
Public string ElementNameSingular;
Public string ElementNamePlural;
Public string ElementAspectSingular;
Public string ElementAspectPlural;
Public string Description;
Public string TeacherReflect;
Public List<Element> Elements;
Public List<Triad> Triads;
}
10.7 RGFA Localization
To support localisation, RGFA provides a customisable environment where users can add a language, activate it, and perform a thorough translation from English to their native language. Below, we present several print screens of RGFA and describe the process.
10.7.1 Admin page
A teacher can log into RGFA Localization Admin Page (Figure 1) and access the admin page. The teacher
The purpose of RGFA localisation admin page is to enable teachers from different countries use the repertory grid technique. Therefore, a teacher logs into and adds a new language. As shown in Figure 2, the teacher is adding the German language.
Figure 2: add a new language. The teacher is adding German
10.7.3 Update translations
After adding a new language the teacher clicks on edit button. A table with three columns is shown (Figure 3). The first column represents the keys, references in the code that are used together with the localisation code (e.g. GB, ger) to locate the correct translation. The second column contains the text in English, which is read-
only. The third column contains editable text fields. Each field can be changed by a teacher, and the result is shown in RGFA website. For example, a teacher applies the localisation settings and students from Great Britain will see “all replies” as text. Whilst another teacher from Germany can localise the RGFA website so students from Germany can view a German expression (i.e. Alle Antworten). Following the same pattern, teachers translate RGFA website. Rather than supporting a limited number of languages, RGFA aims at providing a universal environment for teachers tailored to students’ needs.
Figure 43: the teacher is updating the content from the default one to German.
10.7.4 Activate / de-activate a language
This functionality allows teachers to enable or disable a specific language. For example in Figure, a teacher is adding a new language and updating the translations. At this stage the language is de-activated, so that others
D4.7 Student Model Tools Code Release and Documentation
cannot view the frontend in this specific language. This aims at minimising the level of confusion. Once the translations are ready the teacher makes the German language active, which it can be used from students but also by other teachers. Figure 4 shows a print screen from RGFA and German is shown in the top-right area (the language bar).
Figure 3: the teacher can activate or deactivate a language. In this case, the teacher is activating German.
Figure 4: the teacher has added and activated German, which is shown at the language bar.
10.7.5 View RGFA in another language
After activating a language, teachers and students can localize RGFA. In our case, we have translated RGFA to German, activated the language, and click on German link on the language bar. As a result, teachers and students view it in German. Figure 5 shows two print screens. The top area shows RGFA using its default language (i.e. English), and the bottom area of this figure shows RGFA in German.
D4.7 Student Model Tools Code Release and Documentation
Figure 5. RGFA in English (top). RGFA in German (bottom).
CoNeTo API Description
In order for a RGFA exercise to be created in OLM, CoNeTo exposes an API method CreateRGFAExercise that accepts two parameters, namely a casUserId and a json representing the RGFA exercise as a serialized text. Once the CoNeToAPI method receives the parameters, it de-serializes the json string and loads it into an instance of an RGFAObj. The OLM API is then used to created corresponding entities in the OLM based on the following mapping.
1. An Activity node is created for the main Exercise, using the name of the exercise. 2. For each of the Triads within the Exercise, a Competency node is created under the previously created
Activity.
Once the RGFA exercise has successfully been created in the OLM, it is then visible under the list of activities of the corresponding teacher whose casUserId was used to create these nodes in the OLM. The relevant user can then open the newly created Activity in CoNeTo in order to visualize the underlying RGFA exercise represented by the OLM.
D4.7 Student Model Tools Code Release and Documentation
Once the exercise is loaded into CoNeTo, the details of each Triad can be viewed by clicking on a particular node representing the corresponding Triad. A teacher can update the knowledgelevel of the selected node by using a slider that represents the current value. Clicking on the update button persists the changes back to OLM and the exercise is reloaded to represent the updated OLM model.
Moreover, clicking a node will also load the details of the particular triad that it represents, for instance the list of triad elements it contains and the answers for that particular triad, given by the selected student.
The individual nodes (triads) can be negotiated/discussed by using the chat functionality provided in CoNeTo. The individual conversation pertaining to each node is then persisted in the RGFA database and gets updated each time a message is sent by clicking the Send button.
The format of the JSON object that is used to create a RGFA exercise in OLM is as follows.
{
"exercise": {
"id": "1",
"name": "Exercise 1",
"triads": [{
"chat": null,
"constructs": [{
"keywords": "keyword 1, keyword 2, keyword 3",
"name": "Similarity",
"weight": "0.5"
},
{
"keywords": "keyword 4, keyword 5, keyword 6",
"name": "Opposite",
"weight": "0.5"
}],
"elements": [{
"id": "1",
"name": "Element 1"
},
{
"id": "2",
"name": "Element 2"
},
{
"id": "3",
"name": "Element 3"
}],
"id": "1",
"name": "Triad 1"
}]
}
}
The following code snippet shows the calls made to the CoNeTo API in order to create the exercise in OLM.
[WebMethod]
public int CreateRGFAExercise(string casUserId, string json) {
var rgfaObj = RGFAObj.ParseJson(json);
if (rgfaObj == null)
return -1;
D4.7 Student Model Tools Code Release and Documentation
Repertory Grid Technique is a method for eliciting personal constructs of individuals about elements belonging to the topic of study. Within the NEXT-TELL project, for the purposes of formative assessment, we have decided to start researching RGT with an implementation of the widely adopted method of triadic sorting of elements for personal construct elicitation and subsequent five-point scale rating of the rest of the elements. Briefly put, the triadic sorting method consists of the participants being presented sets of three elements each. For a given set of three elements, the participant is prompted to select the element that is different from the other two and to state how it is different as the “opposite construct”. Then, the participant is to state how the two remaining elements in the triad are similar to each other as the “similarity construct”. The rest of the elements are then rated on a Likert-item scale ranging from the Opposite Construct (1) to the Similarity Construct (5). The participants repeat this process until all the triads of elements are sorted into different and similar and the elements for that comparison are rated. The outcome of this exercise is the Repertory Grid (RG) consisting of rows consisting of triads, columns consisting of elements with the first column being the Opposite Construct and the last column being the Similarity Construct, and the cell values consisting of the ratings given for elements.
2. Suggestions for Teachers
In designing repertory grid exercises, teachers should pay particular attention to the previous domain knowledge of students and to what extent the elicited constructs are grounded in the personal lived experience of the students compared to the domain knowledge.
The repertory grid exercise could be designed for individual students or as a computer supported collaborative learning (CSSL) exercise involving a small group of students. Apart from the classroom usage scenario, another usage scenario for teachers is to employ the repertory grid exercise as lightweight appraisal method for informal learning tasks. We will continue to research this usage scenario in future work with teachers participating in the NEXT-TELL project.
RGFA can be used in a pre-test and post-test paradigm to understand students’ knowledge of concepts and aspects of a topic/subject before and after a particular curriculum module has been taught.
An ideal repertory grid exercise would involve 6-10 elements and 5-6 triads with each element appearing at least once and in different positions of the triad when a particular element features more than once across the different triads.
Each triad should be associated with a competency. Competencies can be specifically created by the teacher for that RGFA exercise or an existing competency can be used from the OLM.
The teachers should provide four keywords each for the opposite construct and the similarity constructs for each triad and its competency. When the student’s personal construct text contains one or more of these keywords, it serves as evidence for that particular competency for that particular triad and the OLM is updated accordingly.
Post repertory grid exercise tasks could include asking the individual students or groups to reflect on their own repertory grids, inspects the repertory grids of their peers or domain experts, and/or inspect the visualizations of the repertory grids for the entire class.
D4.7 Student Model Tools Code Release and Documentation
An additional implication from the classroom exercises and the eye-tracking laboratory studies is that teachers could also learn about students’ current understanding based on the time take for construct elicitation and element rating.
With regard to formative assessment, teachers can use the Teaching Analytics Dashboards to inspect the individual or collective constructs and discern students’ level of domain knowledge.
Teachers can use the built-in RGFA Dashboards to scrutinize the elements ratings to discern students’ ability to distinguish between the different concepts.
3. Suggestions for Students
Repertory grid exercises on topics might not be familiar to students either from prior formal learning settings or from personal experience. That said, a well-designed repertory grid exercise on the familiar and lived practice would allow students to externalize their implicitly held constructs. Students should then be motivated and guided to reflect on their intuitions and connect their personal constructs to domain concepts.
Students should also be able to co-design repertory grid exercises with peers and teachers. Co-designing a repertory grid exercise would require students to select the topic, the elements, and the number, content and order of triads. This in itself could be pedagogically effective.
Students could be given the option of sharing their repertory grids with their classmates and within their social networks. Students should be able to interact with their visualizations of their individual repertory grids and those of their peers and the classroom level repertory grid.
Students should be able to upload their repertory grid exercises to their e-portfolios and integrate them with their open learner models.
Students can be asked to use the CoNeTo tool to compare and contrast their RGFA exercise responses to those of the teacher. The student can discuss and negotiate the OLM’s representation of their competencies for that particular RGFA exercise.
4. Creating a Repertory Grid Exercise using the RGFA Application
The instructions below are for the English version but the steps are identical other language versions.
The following figures present the teachers’ workflow with RGFA3.
1. Visit
a. English: http://cssl.cbs.dk/software/rgfa/Login.aspx?Countrycode=GB
b. German: http://cssl.cbs.dk/software/rgfa/Login.aspx?Countrycode=DE
c. Norwegian: http://cssl.cbs.dk/software/rgfa/Login.aspx?Countrycode=NO
d. French: http://cssl.cbs.dk/software/rgfa/Login.aspx?Countrycode=FR
e. Danish: http://cssl.cbs.dk/software/rgfa/Login.aspx?Countrycode=DK
2. Select Teacher” from the drop down box. (Figure 1)
3. Use “demo” as the userid and “nextell” as the password (Figure 2)
3. In case the user is a teacher, the user needs to select a student from the dropdown list presented in the right most panel called Dashboard. In this list, the names for all the teacher’s students are displayed.
4. Once the student has been selected, the user needs to select an activity under the dropdown list for students. This dropdown shows all the relevant activities for the selected student name.
5. Once the student and the activity have been selected, click on the Load OLM button, which would then load the activity in the main canvas as a diagram.
D4.7 Student Model Tools Code Release and Documentation
Once the diagram is loaded, the 2 panels named, Teacher Panel and Student Panel are displayed, each of which represent the loaded information for the exercise from the teacher’s perspective and the student’s perspective respectively.
6. Selecting a node by clicking it, updates both the Teacher’s and Student’s panels with the information loaded for the selected node as well as any chat between the teacher and the student associated with the selected node in the Chat panel.
7. In order to start discussion on particular node or continue discussion for a node, select the node and then write a message is the text box, followed by pressing the Send button.
D4.7 Student Model Tools Code Release and Documentation
8. In case of the teacher, the score for a particular node can be modified by using the slider that allows a value to be chosen between 0 and 1. This sends the updated value to OLM which then calculates the new value based on the latest one and the updated value is then loaded and displayed in the panel. For a student, this panel and the corresponding text box is readonly.
