TheIntelligentSchool$$ $ $ TheIntelligentSchool$ · TheIntelligentSchool$$ $ $ $ iii$...

The Intelligent School

i


By

Paula Marcelle

In Partial Fulfillment of the requirements for the course

EDID 6512: Design Project

Group Facilitator’s Name: Dr. Camille Dixon-‐Deane

Project Advisor: Dr. Julaine Fowlin

Date Submitted: July 21st 2017


ii

Executive Summary

This design project report examines the application of knowledge management systems in schools for academic and administrative staff. A review of theories and models of knowledge management system revealed that its application is in infancy in school systems. From a few existing web-‐based examples, along with the Haney’s model the design of the Intelligent School app was executed. From user experience observation, and survey results the app was found to be moderately successfully scoring 69 on the System Usability Scale (SUS) scale.


iii

Table of Contents

Executive Summary ii

Table of Contents iii

Introduction 1

Identification and Justification of Need 2

Brief Description of the Target Group 3

Literature Review 4

Existing Knowledge Management Examples 10

Proposed Designs 14

Evaluation of the Design 23 Methodology 24 Methods and Procedures 24 Validity and Reliability 25 Ethics 26

Results and Analysis of Results 26

Alternative Considerations 29

Discussion 30

Conclusion 31

References 33

Appendix A – Task Analysis 37

Appendix B Questionnaire 38

Appendix C – Post Usability Interview Schedule 39


1

Introduction

Teachers are faced with increasing demands, as they are more accountable for

student outcomes than in the past. As such it is important for teachers to keep learning

throughout their careers (Darling-‐Hammond, 2006). The potential for teachers learning

collectively as a group has gained attention in the literature (Honingh and Hooge, 2014),

and organizational hierarchies in general are being put aside as knowledge work calls

for more collaboration. Thus, there is a need for the deliberate and systematic

coordination of an institution’s people, technology, processes and organizational

structure in order to add value through re-‐use and innovation. This coordination can be

achieved through creating, sharing, applying knowledge and lessons learned into the

institution’s memory in order to foster continued organizational learning. This

encapsulates a good definition of knowledge management.

This report outlines the need for and the implementation of the Intelligent

School Research Design Project. The need for the use of knowledge management

systems intervention at this private secondary school is outlined. This report identifies

the specific needs of one department and justifies that need. The target audience is

described. The theoretical foundation for knowledge management, communities of

practice, and knowledge management models are discussed in the literature review

section. Examples of similar existing knowledge management interventions are

illustrated, and then the proposed design for the Intelligent School is detailed. Each

design will be evaluated using research methods and a discussion of the justification of

the chosen design presented. Finally, the findings will be discussed and conclusions

drawn.


2

Identification and Justification of Need

Knowledge Management (KM) is about developing, sharing and applying

knowledge within an organization to gain and sustain a competitive advantage

(Peterson and Poulfelt, 2002). The Intelligent School has as part of its vision “the

creation of liberal environment, grounded in teamwork, a commitment to academic

excellence, the all-‐round development of the individual, and the encouragement of

leadership and self-‐expression at all levels of the school system”. Knapp et al., 2003 had

conceptualized learning agendas that occur throughout schools in accordance to three

levels: student learning, teacher learning and organizational learning. Through extant

data analysis, the knowledge management specific strategic pillars of the school, which

forms the basis to which the annual school development plan is created lists: academic

excellence and technology leadership. There is no implicit knowledge database that

resides within the organization itself to draw knowledge from seamlessly.

Informal interviews have revealed that every year during mid-‐term break, all

members of staff are required to participate in mandatory professional development

sessions. Individual consultants and external firms lead these sessions for the

organization. This is despite Little et al, 2002 notion that there is a rediscovery that

employees have skills and knowledge that are not captured by organizations.

Observations have revealed that the relevance of the professional development to the

specific job role is not taken into account. Additionally, the already developed skill set

level, interest, past training; education and experience are not taken into account when

selecting participants for each session. Thus, professional development is adhoc, with a

one-‐size-‐fits-‐all model, but is highly touted by senior managers at the institution.

Although there is professional development, there is no knowledge management system

in place. Thus gaps exist between the vision of creating leadership and self-‐expression


3

at all levels of the system, and harnessing the power of technology as a technology

leadership school for organizational learning despite the availability of adequate

financial resources.

Thus, there is a need for a technology-‐based knowledge management system in

the school to develop, share and apply knowledge within the school, with a teacher and

organizational learning agenda. The gap in the types of professional development

available to staff, the relevance to their specific function, the ease of availability of

resources online and in mobile friendly version, the ease of updating professional

development files, the tracking of professional development attendance and re-‐

certification deadlines, the input of in-‐house staff to share knowledge with staff are

some gaps that can be addressed with a knowledge management system.

Brief Description of the Target Group

The Intelligent School is a quasi-‐government secondary school whereby the state

funds the tuition for students, but the school board pays teachers and ancillary staff and

is entirely responsible for the operation and maintenance of the plant.

The students enrolled are from form one (7th Grade) through six (12th Grade)

and the student population is close to 2000 students. Within the campus of the

Intelligent School, although registered as one institution with the Ministry of Education,

there practically exist three schools due to three independent management teams. The

two lower schools (forms 1-‐5) have the majority of students and are unisex, and there is

independent management in both gendered school. The upper school (form 6) consists

of 25 teachers and 250 students. There are 250 teachers and 75 ancillary staff members.

Due to the relatively large teacher population, and time constraints of this design

project, a proposed design will be in the form of a pilot design. Since the designer is


4

familiar with the upper schoolteacher body, the population of the sixth form

department was chosen as the sample frame. The staff at the sixth from department

spans all subject areas. Teachers are required to have schemes of work; lessons plans

and give regular assessments and feedback to students. Most staff members are

contracted without tenure and there is a 12% annual turn over rate in the sixth form

department. The staff is comprised of mainly young and mid-‐career teachers who have

a wealth of implicit and tacit knowledge due to experience, continuous professional

development and the isolated cases of attendance at conferences and symposiums.

Teachers are also encouraged to integrate technology into the teaching and learning

process, thus individual teachers may have a large repository of class material in

softcopy. Furthermore, the school has education for sustainable development as a pillar;

thus, an electronic repository can minimize the use of paper and storage space.

Literature Review

There are some main theories and models, which drive the research design

process: knowledge management, human performance technology, communities of

practice, connectivism, model building, spiral theory, and user experience design.

What role does Human Performance Technology play in the Instructional Design

Process?

“Increasingly, practitioners in the field of instructional design and human

performance technology are asked to orchestrate organizational learning and

knowledge management to meet perceived needs for sustained, continuous knowledge

flow” (Fowlin and Cennamo, 2017, p. 1). Stolovitch and Keeps’, (1992) definition of


5

Human Performance Technology (HPT) as the application of what is known about

human and organizational behaviour to enhance accomplishments, economically, and

effectively, in ways that are valued within the work setting. The HPT is a field of

endeavour, which seeks to bring about changes to a system in such, a way that the

system is improved in terms of achievements its values. Since there are increased

demands on schools for mainly the improvement of student outcomes, HPT processes

can be applied. The model is illustrated in Figure 1 and begins with a comparison of the

current and desired levels of individual and organizational performance to identify

performance gaps. A cause analysis follows to determine what impact the work

environment and the people have on performance. The appropriate interventions are

designed and developed based on the gaps and evaluations are done at each stage.

Specific the intervention stage is the performance support, which may be instructional

and non-‐instructional. The focus of a knowledge management intervention is non-‐

instructional in nature and the context is depicted in Figure 3.

Figure 1: HPT model. Source: Fundamentals of Performance Technology by D.M. Van Tiem, J.L Moseley, and J.C. Dessinger, published by ISPI in 2000

Deborah Marcelle� 7/22/17 9:04 PM


Deleted: Figure 1

Deleted: Figure 2


6

Figure 3: Knowledge Management nested in non-training intervention categorized as e-learning. Source: Rosenberg (2003). Building Management Solutions. Presented at the Training Director's forum, Phoenix Arizona, June, 2003

Why should educators be involved in Communities of Practice?

Communities of practice are focused on a domain of knowledge, and over time

accumulate expertise in this domain. They develop their shared practice by interacting

around problems, solutions and insights and building a common store of knowledge,

Wenger, 2001. This theory is also related and similar to Functional Context Theory

(Carnevale et. al 1990), which is about making learning relevant to the experiences of

learners and their work context and Situated Learning (Lave and Wenger, 1991) which

postulates that knowledge need to presented in authentic contexts and learning

requires social interactions. The social interaction aspect of communities of practice

suggests that organizational culture plays a great part in communities’ and how they

operate (Coakes, and Clarke, 2006). They also suggest that enthusiasm amongst

Deborah Marcelle� 7/22/17 9:04 PMDeleted: 2


7

community members from seeing the benefit to their own self-‐knowledge and

development as a part to pay in the success of the community.

One challenge when building communities mentioned by McDermmot (1999) is

the design of human and information systems to help the community members think

together and interact. On the same issue Kling and Courtright (2003) espoused that ICT-‐

supported strategies of Communities of Practice development are better than ICT-‐led

strategies. To add to this perspective Giles Grant of BFNI in KMOnline (2004) suggests,

“IT should only be an enabler for sharing and collaboration. It isn’t the community; the

community is the people.” Thus, from a review of the literature on community of

practice, it can be inferred that since teachers share the same domain of knowledge;

either by content knowledge or pedagogy they can be organized into communities of

practice. Additionally, they should be initially socialized into this community before

there is any IT-‐enabled knowledge repository intervention.

What is the role of knowledge management life cycles in a knowledge management

design project?

A review of the literature on knowledge management life cycles reveals that they

can be classified as simple, complex or static in nature. A simple cyclic representation

was done by Salisbury (2008) and is depicted in Figure 5. This model comprises three

phases: create, preserve, and disseminate and is perceived to be most suitable for

problem solving. A more complex cyclic 7-‐stage, Haney’s model (2006) illustrated in

Error! Reference source not found. in comparison has a more important phase of

maintaining which is described as the updating, changing adding to, deleting, or

replacing activities. This reflects the dynamic nature of knowledge as knowledge is

growing exponentially (Siemens, 2014).



Deleted: Figure 3

Deleted: Figure 4


8

Figure 5: Salisbury (2008) knowledge life cycle

In the Haney’s model, the create, or learn and identify stage involves the

acquisition of knowledge through learning, creating or purchasing based on what is

identified as useful to parts of the firm. Following which is the organizing and codify

stage, which provides context and background for the knowledge and then converting

the content to formats that can be easily accessible to all individuals. The storage phase

involves saving the database in a form that would prevent accidental deletion, or

sabotage, yet, be easily retrievable by users. Distribution is the stage that follows

involves decision on whether the knowledge will be actively or passively distributed

based on push and pull technology. The use phase refers to how the knowledge will be

applied to the user in project or operations of the firm. Finally, the maintain phase

addresses the nature of knowledge in an organization, thus content should be updated

or replaced to reflect change, and at the same time a knowledge management system

itself needs to be monitored and maintained.



9

Figure 7: Haney (2006) 7-stage knowledge life cycle

Ward and Aurum (2004) also presented a 7-‐ stage model, and the interpretation

though not visually depicted in the literature is shown in Figure 8. These authors

present the stages as listed knowledge management activities rather than a cycle,

though it can be inferred from the descriptions that the activities are meant to be

cyclical. In comparison to Haney’s model there is an additional stage of acquiring but no

specific mention of maintaining similar to the Salisbury model. However, maintenance

seemed to be captured under the adaption activity where there is modification of

knowledge collected from previous experience, and the organization phase where

knowledge is stored and maintained for future reference by others. With respect to a

school knowledge management system, teachers, rather, will not create all the files as

some for example Ministry of Education publications would have to be acquired. Thus, a

mixture of the Ward and Aurum’s (2004) and Haney’s (2006) life cycles may be most

suitable for a school’s knowledge management system.



Deleted: 4

Deleted: Figure 5


10

Figure 8: Ward and Aurum Knowledge activities

Existing Knowledge Management Examples

Reviewing existing knowledge management system strategies in schools is

important for features and issues that can be used in a proposed design for the

Intelligent School.

The Physics Educator’s Network of Trinidad and Tobago (PENTT)

www.pentt.weebly.com is a free open community web page that provides syllabus and

internal examination documents, directory of teacher, blogs, notices, resources and

upcoming events for teachers. The page is specific to a subject area and a specific

examination board.

creation

acquistion

identihication

adaptation organization

distribution

application



11

Figure 10: Screenshot from the Physics Educators network of Trinidad and Tobago

The Caribbean Educators’ Network http://caribbeaneducatorsnetwork.com/ is a

regional community of practice. Free registration is needed to access most services.

There are optional groups that one can join mainly by subject, level or education issue

interest. There are discussion boards, a member listing, job boards a general blog

posting area and there is an RSS feed that one can subscribe.



12

Figure 11: Screenshot from Caribbean Educator's network landing page

Betterlesson available at https://betterlesson.com/home is a free registration

based site that allows teachers to share lesson plans, presentations, and assessments.

The knowledge management system is purposed for the US education system, whereby

the content is arranged by standards and grade level. This site offers a robust search

engine mechanism whereby a teacher can just enter a topic, resource type and grade

level and all results will be shown and displayed similar to a search engine.


13

Figure 12: Screenshot of Betterlesson interface for a searched topic

MOODLE although its mainly used as a learning management system is a

learning content management system. Thus, an organization can re-‐purpose its use as a

knowledge management system for teachers. An examples of its use as a knowledge

management system is provided at http://training.tle.courses.open.uwi.edu/

In this example training material can be accessed at any time, thus it may be suitable for

on-‐demand training. A practical example at the high school level is a resource room

where education issues or topics can be organized in blocks.

Figure 13: MOODLE being used as knowledge management system

Edmodo https://www.edmodo.com/home is learning management system that

can be used for knowledge management among teachers. Edmodo allows for the



14

creation of groups by interest. Within each group teachers are allowed to upload files to

the library, post resources to the group. The main panel shows posts in reverse

chronological order, however users can use the in-‐built search bar to search for content.

Figure 14: Screenshot of an EDMODO teacher group

Proposed Designs

A mobile application and a complementary sensitization session are proposed as

the knowledge management design solution. This design project report focuses on the

design of the mobile application for The Intelligent School. The app is available at

https://app.igenapps.com/2324628#index

The Intelligent School’s mobile knowledge management system was designed

based on the Haney’s (2006) model: create or learn, identify, organize and codify, store,

distribute, use and maintain. The design integration of these phases will be discussed in

this section along with screenshots and links as examples of its functionality in this



15

section of the report. Additionally, the Boehm’s (1988) spiral model of progressive

iterative design, which comprise of design, build, and access as depicted in Figure 15

below was used to make several iterations towards the final product. The iterations

were used and the user experience report suggestions are encapsulated within the

alternative considerations section of the report.

Figure 15: Simplified Spiral Model of Software Development. Source: Introduction to UX. University of Michigan https://tinyurl.com/y6vkv94q

Create

The first phase of the knowledge management cycle involves acquiring

knowledge. The Intelligent School already has a large repository of documents in soft

and hard copy, but they are stored in various places. Thus, a campaign to digitize the

hard copies and collect all the digital folders on the server is a first step. All the files

were uploaded to Google Drive cloud services to ensure that if there were any changes

or updates to be made, all members of staff would be able to view the most recent

version. From the Google Drive, the source data will display the last date and time all

the files were updated. This version update is also connected to the maintain phase.



Deleted: Figure 11

Deleted: 11


16

The acquisition of tacit knowledge can prove to be challenging, thus, employees

will be encouraged to write blogs and create video presentations on any topics, where

their expertise lies. Since a review of the literature suggests that it is difficult to explain

complex processes that usually remain tacit, a discussion forum will be attached to each

blog, thereby allowing further questions for clarifications to be asked and answered. In

order to motivate staff to the culture of sharing, a badge system will be implemented

based on the number and quality of blog post. This design ensures that tacit knowledge

is transferred and a sense of community is maintained. Since the school already uses

social media, by the user’s discretion content can be pushed from the social media pages

as well.

A mobile application prototype shell was built at this stage. Several open source

images were chosen using the spiral model of software development to the end product

icon.

The final icon design represents a community of practice, as this is one of the

main theories that underpin the Intelligent School’s knowledge management solution is

shown in Figure 17. The image shows different colours of persons holding hands

forming a circle that represents diversity in community, but through sharing the

connection is maintained.

Figure 17: Icon used for The Intelligent School



Deleted: Figure 12

Deleted: 12


17

Organizing and Codifying

In order to make the knowledge useable to all members of staff, the files needed

to be organized. Haney (2006) suggests that organization involves an explanation of

context. Thus, for the app store and the introductory page of the app a brief overview of

the context and use of the app is given in the form of rhetorical questions and

statements which appeals to the need for the use of the app and a summary of the

functionalities. This descriptive page is highlighted in the screenshot in Figure 19

below.

Figure 19: Welcome page of Intelligent School App

Codifying involves changing the content into forms that are readable by the app.

Additionally, since most computer users access information through their desktop or

laptops, iOS or Android devices, the codifying process should ensure that the app is

accessible by all these system users. Since Google Drive will be used to update the files,

the type of files needed to be compatible with Google Docs: docs, sheets and slides thus,



Deleted: Figure 13

Deleted: 13


18

MS Word, MS Excel, and MS PowerPoint files alone can be used. Thus, all scanned files

were converted images and inserted into MS Word files so that Google Drive can read

them. Using the Google Drive also addresses accessibility issues as Google Drive are

only compatible with the most commonly used browsers: Internet Explorer, Mozarilla

FireFox, Safari, and Chrome.

Organizing

With a large number of files, there was need for some system for their

organization. One proposed design was by content classification as shown in the mid-‐

fidelity prototype in Figure 21 below. This classification was used through a task

analysis of the job roles and descriptions of teaching staff at the institution and from the

review of existing knowledge management systems in schools that were presented in

the Existing Examples section of this report.

Figure 21: Wireframe of general content classification of the Intelligent School



Deleted: Figure 14

Deleted: 14


19

Each category was further subdivided with additional drop down fields as shown in the

wire frame example in Figure 23 for content knowledge.

Figure 23: Example of organization and subdivision of content knowledge wireframe

Whilst Figure 21 and Figure 23 represent the mid-‐fidelity prototype, after

performing several iterations of organizing, the final design’s organization is illustrated

in the figure below with corresponding icons to match the content.





Deleted: Figure 15

Deleted: 15

Deleted: Figure 14

Deleted: Figure 15


20

Figure 25: General organization of the Intelligent School knowledge management app

In an effort to continue communities of practice established from the face-‐to-‐face

workshops, discussion forums organized by these same themes were added with an

appropriate icon symbol. Users are able to add a new discussion forum from within the

app that may share their implicit knowledge or any resource that they can codify

through the title as shown in Figure 26. An example of a discussion forum is shown in

Figure 28.

Figure 26: Creation capabilities to continue communities of practice in app





Deleted: 16

Deleted: Figure 17

Deleted: Figure 18

Deleted: 17


21

Figure 28: An example of a discussion forum

Store

The knowledge will be stored in Google Drive, and Google Plus and will be

accessed through the Intelligent School App, which will be accessible via Android and

iOS devices and through any desktop computer. The files would be stored under the

headings listed in the organization section. In order to ensure that the files are safe from

accidental deletion, the owner of each file would be prompted to password protect their

file submissions, and limit permissions on each document. Thus cloud-‐based services

will be used to store files for the application.



22

Distribute

Both push and pull technology will be used to actively and passively send

content to the user. By virtue of an app the user can get content sent to them by

allowing push notifications, mandatory information will be sent this way. Pull

technology will be used to allow users to access the database by choice.

Use

The phase involves accessing the data, adding knowledge and participating in the

communities of practice. The adoptability of the app can be measured directly and

indirectly. Through the developer’s version it can be seen the number of times the app

was viewed. Through the respective app stores, the number of times the app was

downloaded can be determined and through each menu feature linked to GoogleDrive,

the number of files added, the dates the files were updated can be determined which

determines use and adoptability of the app. This data can also be analyzed by other

metrics such as time series analysis.

Maintain

Maintenance of the knowledge management system will be done through

updating the initial Google Drive file that is linked to the app. This will be done at least

each year during the mandatory academic week preparation period. Given, that this

knowledge management system is being applied to a school environment, and there are

no organizational departments with the responsibility for knowledge and learning

management, the responsibility of maintaining the system will be shared among

teachers, and the relevant non-‐academic staff.


23

Thus, the main design features are: file sharing, file updating, blogs and

discussion forums, and notifications.

Evaluation of the Design

This is a small-‐scale user experience study aimed at gathering information about

the components of user experience: value, usability, adoptability and desirability (Guo,

2012) in the design of a knowledge management system for a school. These four

components are depicted in Figure 30 below.

Figure 30: Guo, F. (2012). More than Usability: The Four Elements of User Experience, Part I.

According to Holzinger (2005), usability is most often described as the ease of

use and acceptability of a system for a particular class of users carrying out specific

tasks in a specific environment. Ease of use affects the users’ performance and their

satisfaction, whilst acceptability affects whether the product is used (Bevan, 1995).

Thus, user testing involves watching representative users try to accomplish specific



Deleted: Figure 19

Deleted: 19


24

tasks using a product where one would learn what works and what doesn’t, why things

work and don’t and user needs one may have missed. (MichiganX, n.d.).

Methodology Numerous factors guided the evaluation of the design process. According to

Shavelson et al., (2003) a good research should pose important answerable questions

that are related to the available theory. Additionally, data gathering methods

appropriate to the investigation should be used that produce the findings.

A mixed methods design using design triangulation where the researcher

simultaneously collects qualitative and quantitative data was used.

Methods and Procedures

A combination of a usability survey, non-‐participant structured observations,

structured interviews and cognitive walkthroughs were used to inspect and test the

app.

The general aim of the usability survey is to collect a substantial amount of data

in order to draw conclusions about the suitability of the app’s design for use by staff at

the Intelligent School. The survey was deemed to be reliable as a standard

questionnaire instrument as it was adapted from www.usability.gov and is appended in

Appendix B.

Observations (see Appendix A) were also used as a user experience tool for

triangulation as means of comparing the evidence from the usability survey in order to

determine the accuracy of the information. Non-‐participant observers usually enter the

‘scene or research’ with knowledge of what they want to research and why and has the


25

advantage of the having ‘insider knowledge to the analysis of data and field notes

(Moyles, 2002). The observer requested participants to ‘ Think Aloud’ the steps that

they were thinking and taking which validated the ease of use answers on the survey.

Another observational strategy used required the user after the completion of each task

to say the phrase ‘”I am done”. This ensured that the observer could have validated

whether the task was indeed completed to specification and provided internal validity.

The observer observed each user one-‐on-‐one to reduce observer bias such as selective

attention where one selects what one wants to see and interpret from one’s perception

and values.

Post user experience interviews were also conducted. This took the form of

open-‐ended questions, so that the researcher could have ascertained additional pieces

of information that was missed in the questionnaire survey and from the observations.

The additional open-‐ended answers were used as a basis for alternative considerations

section, and the interview-‐scheduled is outline in Appendix C.

Validity and Reliability The research is believed to have internal and external validity as a standard

usability questionnaire correctly maps the usability phenomena being investigated.

However, this may be reduced if the respondents did not complete the given task in

entirety, but this was checked through observation. Cohen & Manion (1994) suggest

that a researcher’s judgment may be affected by ones close involvement to the group.

Thus, given that the usability testers were persons who worked in my department, my

judgment could have been skewed and affect the validity and reliability of the results


26

Ethics

The researcher acknowledges that fully ethical research is impossible according

to Busher and Clarke (1990). However, the researcher was sensitive to the socio-‐

political context as a member of the institution. Thus, explanatory letters as a means of

gaining informed consent from potential usability testers was used. Testers were

voluntarily used and were informed that they could have withdrawn at any stage in the

process. Additionally, the researcher protected the institution and the usability testers

through the use pseudo names for anonymity and privacy in the compilation of the

report.

Results and Analysis of Results The results of the task analysis are depicted in the Table 1 below. Tasks that

were successfully completed were assigned the colour green whereas the tasks that the

user found difficult or could not have completed were assigned the colours yellow and

green respectively.

Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8

User 1

User 2

User 3

User 4

User 5 Table 1: Results of Usability task analysis

Legend User successfully completed User completed with difficulty User unable to complete task

Deborah Marcelle� 7/22/17 9:04 PMDeleted: Table 1


27

From the table it can be seen that 0% of the users were able to find and

download the app without assistance. It was discovered that a subscription was

required in the App Store and Google Play Store, thus a link was provided for the users

to access the app in the form of a web page, and henceforth 100% percent of the

participants were apple to access the content of the app in web form.

The second task involved retrieving a named file and assessing whether the

categories for organizing the files were logical and the ease of accessibility of the file.

From the table it can be seen that only one out of five or 20% of the users experienced

minor difficulties in accessing the Spanish file. One user used an iphone 4G, and the

home screen prompted either to find on AppStore or Get it for Google. The user

intuitively chose the platform that corresponds to the phone, but only through trial and

error was able to access the interface through the platform for Google users. This, was

indeed an odd revelation that can frustrate users and decrease the app’s probability of

adoptability. Notwithstanding this issue, the user was able to correctly analyze the

location of the file, and successfully retrieve it.

The third task involved retrieving a file on differentiated instruction. 100% of

the users were able to complete the task successfully without assistance. From the

Think Aloud strategy one user expressed surprise that the file opened in another

window of the browser.

Task four involved adding to an already created discussion on professional

learning communities. From the observations, the screens did not appear with the logo

on the mobile device testers. Also the reply button did not seem obvious to users of the

mobile devices as such 60 % of the testers were heard from the Think Aloud wondering

where to click and choosing the wrong buttons. Through trial and error 80% of the

users were able to eventually post a reply to the discussion with minor assistance.


28

The fifth task involved creating a new discussion thread. From the first user test,

the link was empty, thus there was a delay in completing the task, as the app had to be

modified. It was also discovered that one need to be a member of the Google Group

before one can post any new discussion threads. Testers were then given access. 60% of

users were able to complete the task without assistance, whilst no tester was unable to

complete the task. From the Think Aloud process, there was unfamiliarity with the icons

and their location on the page for the creation process.

All participants were able to successfully retrieve the discipline log form. This

indicated that inherent part of organization of the files was appropriately done.

Participants were able to classify the file as an administration issue, and then further

determine from the choices that the file would be found in the forms folder. This 100%

success in performance closely correlates with the second task of retrieving the Spanish

Nationalities file, as it requires similar skills of searching the knowledge management

system and retrieving a specific file, though the two files were classified differently.

The seventh task of adding ones own file to the app proved to be the most

challenging of all the tasks to do without assistance. From the Think Aloud Process,

participants had to make several steps but were guided by the prompts on the screen

towards successful execution of the task. The process involved logging into GooglePlus

and making a link to a file rather than uploading a file as an attachment. Users

expressed frustration when they realized that they could not have uploaded a file from

their system directly. Some were promoted to use GoogleDrive and then provide the

link to that file on GooglePlus. Others were observed to be pasting hyperlinks from

online folders successfully to link the file to the relevant placeholder in the app.

The eight task of updating a contact form list was 80% unsuccessful. From

observations users were able to access the template for the Google Sheet for the contact


29

list, but were unable to edit it. From the Think Aloud Process, users tried clicking all

cells on the spreadsheet, re-‐orientating their devices but the task proved to be

impossible. Only one tester was successfully able to edit the GoogleSheet, but from the

observations it was realized that this specific user was re-‐directed to the already

downloaded GoogleSheet app where it was possible to edit the spreadsheet.

The post-‐test questionnaire responses from the five users achieved an average

score of 69, which corresponds to an average rating of usability on the SUS rating scale.

Alternative Considerations Based on the feedback obtained from the post-‐user experience interview the

following considerations can be made for future iterations of the prototype in order to

increase adoptability of the knowledge management system:

1. A brief video clip that serves as a walkthrough on what steps to take to perform

specific actions. The video can be played automatically when specific actions are

commanded, or alternatively the screen-‐cast videos can be placed in a help

section or embedded as responses in frequently asked questions section.

2. Screen tips can be used over icons so that users can become familiar with the

meaning of the icons, rather than using a trial and error approach to perform

specific tasks. This screen tip also extends into principles of accessibility.

3. Alerts and Reminders should be placed on when files need to be maintained e.g.

the GoogleSheet at the end of the academic year when teachers usually leave the

school. However, these maintain alerts should be specific to users who are

required to perform the specific task. The alerts and reminders can be used to

alert staff to external online training activities or professional learning

community events at the school.


30

4. The use of another cloud storage provider other than Google suite of products,

which does not require the use of a secondary app such as Google Drive app on

mobile devices in order to perform tasks in the Intelligent School app on iOS

systems.

5. Content can also be classified by form level simultaneously, or some tagging by

grade level of the content should be done for easier searches.

6. Another practical use of the app can be to collaboratively list in-‐house expertise

to be used in professional training events via Google Sheets on the app.

7. The logging of training events attended by staff to avoid duplication via another

GoogleSheet within the Intelligent School app can be done. This can also so serve

as a reminder for when re-‐training is needed via alerts e.g. first aid training

license.

Discussion

The design of the Intelligent School app was grounded in theory of knowledge

management cycles from the Haney’s (2006) model, where creation, identification,

organizing and codifying, storing, distribution, use and maintenance were the main

stages used during ideation. Eight tasks were performed by user-‐testers that closely

matched the phases of knowledge management life cycle applied to tasks staff at a

school would normally undertake in a distributed environment. From the results in

Table 1, users were able to complete seven out of eight assigned tasks with a moderate

degree of success. These tasks were judged through Think Aloud process, and the

verification that the tasks were actually completed for validity when the tester stated “I

am done.” This result was consistent with the method triangulation of the post user test

Deborah Marcelle� 7/22/17 9:04 PMDeleted: Table 1


31

SUS questionnaire that yielded a score of 69, which corresponds to average usability

rating. Since the questionnaire instrument is internally valid on user-‐experience, this

app ratings agrees with Lave and Wenger (1991) concept of knowledge needs to

present in authentic situations and learning experiences requires social interactions.

There are several issues that were not considered which may have changed the

reported results. One such issue was the type of device used by the tester, and the

model. The Intelligent School app was built based on the linking to the Google Suite of

products through one sign-‐in. However, it was discovered that some Google products

are not optimized for iPhone users. This was highlighted mainly in task 8, whereby most

users were unable to update the GoogleSheet if they did not have a secondary app

Google Sheet already installed on the phone. These users were prompted to install the

app at that stage of attempting to complete the task. It is believed that there should be

no need for a secondary app for one app to run successfully. The app is available fully

for desktop users and no usability issues were detected in that version. Thus, a larger

scale study could have stratified users by device type.

Whilst convenience purposive sampling was used, from the thoughts

expressed in the Think Aloud process, it was apparent that the frequency of use of

technology, and the types of activities one normally performs using technology could

have been used as a pre-‐screening criteria, thus ensuring that users along the

continuum of technology literacy were represented in the user testers.

Conclusion

The design of the Intelligent School app as a knowledge management system

used all the elements of the knowledge management life cycle. Thus, the design was

grounded in theory. The app proved to be moderately effective in its usability as most of


32

the typical tasks assigned which are related to the phases of the cycle were performed

with ease, and successfully executed. The app performed average from the usability

questionnaire, which aligns with the performance in the task. The suggestions made in

the interviews offered useful ways to improve the design of the app, which can increase

its adoptability and desirability.


33

References

Bevan, N. (1995). Measuring usability as quality of use. Software Quality Journal, 4(2),

115-‐130.

Boehm, B. W. (1988). A spiral model of software development and enhancement.

Computer, 21(5), 61-‐72.

Busher, H., & Clarke, S. (1990). The ethics of using video in educational research. using

Video Recordings for Teacher Professional Development, Leeds: University of Leeds,

School of education.

Carnevale, A. P. (1990). Workplace Basics Training Manual. ASTD Best Practices Series:

Training for a Changing Work Force. Jossey-‐Bass Inc., Publishers, 350 Sansome

Street, San Francisco, CA 94104.

Coakes, E., & Clarke, S. (2006). Communities of Practice. In Encyclopedia of knowledge

management (pp. 30-‐33). IGI Global.

Cohen, L., & Manion, L. (1994). Research Methods in Education. London: Routledge.

Darling-‐Hammond, L. (2006). Constructing 21st-‐century teacher education. Journal of

teacher education, 57(3), 300-‐314.


34

Fowlin, J. M., & Cennamo, K. S. (2017). Approaching Knowledge Management Through

the Lens of the Knowledge Life Cycle: a Case Study Investigation. TechTrends,

61(1), 55-‐64.

Guo, F. (2012). More than Usability: The Four Elements of User Experience, Part I.

Haney, D. (2006). Knowledge management, organizational performance, and human

performance technology. In J. Pershing (Ed.), Handbook of human performance

technology (pp. 619–639). CA: Pfeiffer.

Holzinger, A. (2005). Usability engineering methods for software developers.

Communications of the ACM, 48(1), 71-‐74.

Honingh, M., & Hooge, E. (2014). The effect of school-‐leader support and participation in

decision making on teacher collaboration in Dutch primary and secondary

schools. Educational Management Administration & Leadership, 42(1), 75-‐98.

Kidwell, J. J., Vander Linde, K., & Johnson, S. L. (2000). Applying corporate knowledge

management practices in higher education. Educause quarterly, 23(4), 28-‐33.

Kling, R., & Courtright, C. (2003). Group behavior and learning in electronic forums: A

sociotechnical approach. The Information Society, 19(3), 221-‐235.

KMOnline. (2004). In the know: Expert perspectives.-‐What is the best software supplier

for communities of practice? (2004). KMOnline, 7 (2), 4.

Deborah Marcelle� 7/21/17 6:16 AMFormatted: Indent: Left: 0", First line: 0"


35

Knapp, M. S., Copland, M. A., Ford, B., Markholt, A., McLaughlin, M. W., Milliken, M., &

Talbert, J. E. (2003). Leading for learning sourcebook: Concepts and examples.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.

Cambridge university press.

Little, S., Quintas, P., & Ray, P. (2002). Part III knowledge, innovation and human

resources. Managing Knowledge: An Essential Reader, London: The Open

University in association with SAGE Publications.

McDermott, R., & Consulting, M. (2004). How to avoid a mid-‐life crisis in your CoPs.

Knowledge Management Review, 7, 10-‐13.

MichiganX: UX501x Introduction to User Experience. (n.d.). Retrieved June 26, 2017,

from https://courses.edx.org/courses/course-‐v1:MichiganX UX501x

3T2016/info

Moyles, J. (2002). Observation as a research tool. Research methods in educational

leadership and management, 172-‐195.

O'Driscoll, T. (2003). Chronicling the emergence of human performance technology.

Performance Improvement, 42(6), 9-‐22.


36

Petersen, N., & Poulfelt, F. (2002). Knowledge Management in Action. Developing

knowledge and value in management consulting, 2, 33.

Salisbury, M. (2008). From instructional systems design to managing the life cycle of

knowledge in organizations. Performance Improvement Quarterly, 20(3-‐4), 131-‐

145.

Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of

education design studies. Educational researcher, 32(1), 25-‐28.

Siemens, G. (2014). Connectivism: A learning theory for the digital age.

Stolovitch, H. D., & Keeps, E. J. (1992). Handbook of human performance technology. San

Francisco:Jossey-‐Bass/Pfeiffer.

Van Tiem, D., Moseley, J. L., & Dessinger, J. C. (2012). Fundamentals of performance

improvement: Optimizing results through people, process, and organizations. John

Wiley & Sons.

Ward, J., & Aurum, A. (2004). Knowledge management in software engineering-‐

describing the process. In Software Engineering Conference, 2004. Proceedings.

2004 Australian (pp. 137-‐146). IEEE.

Wenger, E. (2001). Supporting communities of practice. A survey of community-oriented technologies.


37

Appendix A – Task Analysis Dear Participant,

Thank you for participating in today’s study. The purpose of the study is to test the

usability of a knowledge management system that was built as part of a design course

for the MSc. Instructional Design and Technology programme of which I am enrolled. In

today’s session I will ask you to perform some tasks using The Intelligent School app,

please Think Aloud while you are executing the steps and when you are finished say the

phrase “ I am done.” Afterwards, I will ask some questions about your experience using

the app.

Your participation is completely voluntarily; as such you can withdraw your consent at

any time. You may also decline to perform any particular task or decline to answer any

particular question.

Thank you

Please visit the App Store or Google Play store and perform the following tasks

1. Find and download the Intelligent School app

2. Search for a Spanish file named Nationalities

3. Retrieve a file on differentiated instruction strategies

4. Add to the discussion forum on professional learning communities

5. Create a new discussion forum entitled Science Fair

6. Find and Print a Discipline log form

7. Add one of your own files to the relevant section

8. Update the existing GoogleDoc named contact list for the given fields


38

Appendix B Questionnaire Dear Participant, Thank you for opting to participating in the second phase of the usability test on the

Intelligent School App. Your participation is important for the evaluation of the product

for my course. You may decline altogether or choose to leave any question blank.

Instructions: For each of the 10 criteria, tick in one of the columns along the continuum

from 1 through 5 where 1 is strongly disagree and 5 is strongly agree

Criteria Strongly

Disagree

1

2

3

4

Strongly

Agree

5

1. I think that I would like to use this system

frequently.

2. I found the system unnecessarily complex.

3. I thought the system was easy to use.

4. I think that I would need the support of a

technical person to be able to use this system.

5. I found the various functions in this system were

well integrated.

6. I thought there was too much inconsistency in

this system.

7. I would imagine that most people would learn to

use this system very quickly.

8. I found the system very cumbersome to use.

9. I felt very confident using the system.

10. I needed to learn a lot of things before I could

get going with this system.

Source: www.usability.gov


39

Appendix C – Post Usability Interview Schedule

1. Where does the system need the most improvement?

2. What, if anything else, would you use this system for?

3. Have you used systems that do similar things, to what this system does? How do

you compare them?

4. Is there anything else that you may wish to share?

Date post:	12-Jul-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

TheIntelligentSchool$$ $ $ TheIntelligentSchool$ · TheIntelligentSchool$$ $ $ $ iii$...

Documents