A Software Quality Model for Evaluating Medical...

Prince Sultan University College of Computer & Information Sciences

Department of Software Engineering

A Software Quality Model for Evaluating Medical Simulation Tools

Prepared By: Norah Naif Al-Romi

Under Supervision of Dr. Areej Al-Wabil

June 2015

Submitted in partial fulfilment of the requirements for the Degree of Master in Software Engineering at the Department of Software Engineering at the College of Computer and

Information Sciences

A Decision Support System for Evaluating

Medical Simulation Tools

By Norah Naif Al-Romi

This thesis was defended and approved on June 3, 2015

Supervisor: Dr. Areej Al-Wabil

Members of the Exam Committee

Dr. Areej Al-Wabil Chair

Dr. Nor Shahriza Abdulkarim Examiner

Dr. Iman Al-Momani Examiner

I

ACKNOWLEDGEMENT

First and foremost I want to express my highest gratitude to Allah almighty for providing

me guidance, ability, knowledge and patience in completing this thesis. I wish to thank my

supervisor, Dr. Areej AlWabil, Vice Dean of Academic Affairs at Prince Sultan University,

you have been a tremendous mentor for me. I would like to thank you for encouraging the

research and for allowing me to grow as a research scientist. Your advice on both research

as well as on my career have been priceless. I would also like to thank my committee

members, professor Nor Shahriza Abdulkarim, Dr. Iman AlMomani for serving as my

committee members even at hardship. I would especially like to thank the physicians,

quality managers and medical student for helping me collect data for my thesis. A special

thanks to my family. Words cannot express how grateful I am to my mother, father, and my

sister for all of the sacrifices that you’ve made on my behalf. I have to express my sincere

appreciation and gratitude for their support and understanding during my master journey.

At the end I would like express appreciation to my beloved husband Dr. Khaled AlRabiah

for his continuous support.

II

ABSTRACT This thesis describes a software quality model designed for assisting decision makers in

evaluating medical simulation systems. Decision-making in medical domains is an

increasingly complex task that involves a number of stakeholders, subspecialties and

technologies. Medical simulation tools create a lifelike situation for individuals to examine

the impact of decisions and changes in procedural activities in clinical and healthcare

contexts. They provide a relatively safe context for the patients and professionals as it

involves simulated human patients, scenarios of emergency response, and flow in

procedural activities (e.g. laparoscopic surgery). Evidence suggests that medical simulation

systems have the potential to improve the effectiveness, safety, and efficiency in health care

services. Moreover, it has been shown to consistently deliver significant value to the

organization, staff, or trainees in decision-making. Although medical simulation provided

ideal approaches for addressing healthcare issues, the number of successful software

implementation and development is inadequate since it is relatively small compared with

other established engineering disciplines, such as industrial and mechanical engineering.

Medical simulation issues have been highlighted in the literature, such as usability and

scalability. Considering a software quality model, which includes quality assurance factors,

specific to the medical simulation context is foreseen as an approach to augment current

practice in the design and development of such systems. This decision support system

works as an interactive tool and is designed to help in examining the effectiveness of

medical simulation tools, for specific contexts of use, before procurement for hospitals,

training centers, or education purposes. A systematic review of the literature on medical

simulation tools is conducted with a focus on quality assurance factors such as the ones

applied in the Mccall, Boehm, ISO 25022 and ISO 9126.

The contributions of this thesis are the conceptual design of the proposed decision support

system (DSS) and user acceptance testing of the developed system. The DSS is designed to

process the software quality metrics and conduct a comparative evaluation with other tools

in the domain and provides visualizations that assist in decision-making and quality

assessment. In addition, the software quality model that is designed for evaluating medical

simulation systems is introduced which aggregates the relevant quality assurance factors for

III

the context of medical simulation from previously developed software quality models.

Another contribution is the development of the medical simulation decisions support

system (DSS) which integrates the software quality model in a web-based system. A case

study is presented to examine the usability of the DSS in iterative user-centered design

cycles and in the user-acceptance testing of the system on a sample of medical simulation

tools.

IV

ABSTRACT IN ARABIC

ملخص االرسالة

في مجالل االمحاكاةة االطبيیة ووتطويیر أأنظمة تفاعليیة جوددةة االبرمجيیاتت االمستخدمة إإلى ددررااسة تقيیيیمهھھھذهه ااالططرووحة تهھدفف

ووذذلك لموااكبة االمستجدااتت في برمجيیاتت االمحاكاةة االطبيیة وو مجالل االرعايیة االصحيیة مساعدةة أأصحابب االقراارر فيمصممة ل

من حيیث آآليیاتت االتفاعل االمرئي وواالحسي وواالصوتي وو برمجيیاتت نظمةتقيیيیم أأنظمة االمحاكاةة باعتباررهھھھا من أأعقد ااألمعايیيیر

. ووااجهھاتت ااالستخداامم

نظراا يید ووذذلك لمعقدةة بشكل متزاا امم اا االمهھ لقراارر في االمجاالتت االطبيیة يیعد من االجهھاتت االمعنيیة باختيیارر ووااستخداامم لتعدددااتخاذذ اا

وواالمجاالتت االعلميیةاالتخصصاتت االدقيیقة متلباطط االرعايیة االصحيیة ووتعددد ووتقيیيیم جوددةة ااألنظمة ووااررتباطط االمنظومة االتشغيیليیة ب -

الفراادد . أأددووااتت االمحاكاةة االطبيیة تخلق حالة تالمس االوااقع وواالتي تهھدفف االي مساعدةة ااقنيیاتتاالتاالمستجدااتت في مجالل وواالبيینيیة

لدررااسة أأثر االقرااررااتت وواالتغيیرااتت في ااألنشطة ااإلجراائيیة االسريیريیة وواالرعايیة االصحيیة. هھھھذهه ااالجهھزةة آآمنهھ نسبيیا للمرضى

وواالمهھنيیيین ألنهھا تحاكي االمرضى من االبشرمن خاللل سيینارريیوهھھھاتت ااالستجابة لحاالتت االطوااررئئ٬، وواالتدفق في ااألنشطة

لدالئل إإلى أأنن أأنظمة االمحاكاةة االطبيیة لديیهھا االقدررةة على تحسيین االفعاليیة ااإلجراائيیة (مثل االجرااحة االتنظيیريیة). تشيیر اا

قدمم ذذلك٬، فقد تبيین أأنهھا ت االصحيیة. ووعالووةة على ة االرعايی فاءةة في خدماتت لك لسالمة وواا أأوو االمنشأةة كبيیرةة للمنظمة منفعةوواا

عمليیة صنع االقراارر. مجالل االرعايیة االصحيیة أأوو ااإلددااررةة االطبيیة وو ٬، وواالموظظفيین٬، أأوو االمتدرربيین في االطبيیة

اجحة لطبيیة االمقدمة تعد ن اةة اا ااجهھزةة االمحاك من اانن لرعايیة االصحيیة٬، ااال ااننفي مجالل االتدرريیب ووتنميیة االمهھاررااتت لعلى االرغم

إإلى أأنن آآليیاتت تقيیيیم االجوددةة لهھذهه ااألنظمة لم يیتوصل االباحثونن إإلى توحيیدهھھھا بما يیتوااكب مع االتطوررااتت ااألبحاثث االحديیثة تشيیر

مقاررنة مع غيیرهھھھا من االتخصصاتت ااالخرىى االمعمولل بهھا٬، مثل االهھندسة االصناعيیة وواالميیكانيیكيیة. ووقد تم تقنيفي االمجالل اال

ل االمحاكاةة االطبيیة مثفي مجالل ددررااسة االتحديیاتت االتي يیوااجهھهھا االمستخدميینلى عفي هھھھذهه االرسالة االعلميیة تسليیط االضوء

االبرمجيیاتت٬، يیتضمن عواامل ضمانن االجوددةة٬، خاصة الجهھزةة هھيیكل جوددةة تقدمم هھھھذهه االرسالة مقترحح لااالستخداامم. صعوبة

االمحاكاةة مصمم من مقرحاتت تقيیيیم جوددةة االبرمجيیاتت ااألساسي مع االتركيیز على االعواامل االمتخصصة بمجالل أأنظمة االطبيیة

يیستهھدفف تصميیم ووتقيیيیم نظامم لضمن منهھجيیة االدررااسة لهھذهه االرسالة االعلمي . تم تصميیم االهھيیكليیة االتفاعليیة االمحاكاةة االطبيیة

في ددررااسة فعاليیة أأددووااتت االمحاكاةة االطبيیة٬، لسيیاقاتت محدددةة االمختصيین ووااإلدداارريیيین لمساعدةة ااإلددااررةة االطبيیة أأوو االصحيیة

يیقومونن بتقيیيیم ااجهھزةة االمحاكاةة االطبيیهھ قبل ااعتماددهھھھا أأوو شراائهھا للمستشفيیاتت٬، وومرااكز االقرااررااالستخداامم حيیث اانن ااصحابب

االتدرريیب٬، أأوو ألغرااضض االتعليیم.

االبحث تمت مرااجعة ما تم ددررااستهھ مسبقا فيیما يیخص أأددووااتت االمحاكاةة االطبيیة مع االتركيیز على عواامل ضمانن في هھھھذاا

نظامم تقيیيیم ااجهھزةة . ووعالووةة على ذذلك٬، فإننا نقدمم االتصميیم االنظريي من ISOاالجوددةة مثل تلك االمطبقة في ماكولل٬، بوهھھھم٬، وو

االمقاررناتت مع أأددووااتت أأخرىى في االمجالل وويیقدمم االتصوررااتت االذيي يیعالج في ااالستداللل للتقيیيیم ووإإجرااء االمحاكاةة االطبيیة

V

االالززمة االتي تساعد في ااتخاذذ االقرااررااتت ووتقيیيیم االجوددةة. إإسهھاماتت هھھھذهه ااألططرووحة هھھھي عباررةة عن هھھھيیكلة جوددةة االبرمجيیاتت

. لتقيیيیم جوددةة أأجهھزةة االمحاكاةة االطبيیةاالمستخدمة لتقيیيیم أأنظمة االمحاكاةة االطبيیة. ووباإلضافة إإلى ذذلك٬، نظامم خاصص

VI

Table of Contents ACKNOWLEDGEMENT .......................................................................................................................... I

ABSTRACT .................................................................................................................................................. II

ABSTRACT IN ARABIC ........................................................................................................................ IV

LIST OF TABLES ................................................................................................................................. VIII

LIST OF FIGURES ................................................................................................................................... IX

LIST OF ABBREVIATIONS ................................................................................................................... X

CHAPTER ONE: INTRODUCTION ..................................................................................................... 1 1.1 MEDICAL SIMULATION TOOLS ................................................................................................................. 2 1.2 PROBLEM DEFINITION ................................................................................................................................. 4 1.3 RESEARCH SCOPE ......................................................................................................................................... 5 1.4 AIMS AND OBJECTIVES ............................................................................................................................... 6 1.5 RESEARCH QUESTIONS ............................................................................................................................... 6 1.6 METHODOLOGY ............................................................................................................................................ 7

1.6.1 Work Phase One: Data Collection .................................................................................................. 8 1.6.2 Work Phase Two. Model Designing ................................................................................................ 8 1.6.3 Work Phase Three. Model Developing .......................................................................................... 8

1.7 CONTRIBUTION .............................................................................................................................................. 9 1.8 THESIS STRUCTURE ..................................................................................................................................... 9

CHAPTER TWO: LITERATURE REVIEW .................................................................................... 10 2.1 MEDICAL SIMULATION ............................................................................................................................ 11

2.1.1 Medical Simulation Categorizations ............................................................................................ 12 2.1.2 Medical Simulation Challenges ..................................................................................................... 13 2.1.3 Human Factors in Medical Simulation ....................................................................................... 15 2.1.4 Medical Simulation Types ................................................................................................................ 17

2.2 DECISION SUPPORT SYSTEMS (DSSS) ................................................................................................ 18 2.2.1 Data-driven ............................................................................................................................................ 18 2.2.2 Knowledge-based ................................................................................................................................. 18 2.2.3 Document-driven ................................................................................................................................. 19 2.2.4 Communication-driven ...................................................................................................................... 19 2.2.5 Model-driven ......................................................................................................................................... 19

2.3 SOFTWARE QUALITY MODELS .............................................................................................................. 22 2.3.1 McCall’s Quality Model ................................................................................................................... 23 2.3.2 Boehm’s Quality Model ..................................................................................................................... 23 2.3.3 FURPS Quality Model ....................................................................................................................... 24 2.3.4 Dromey's Quality Model ................................................................................................................... 25 2.3.5 ISO 9126 Quality Model ................................................................................................................... 26 2.3.6 ISO/IEC 25000 SQuaRE ................................................................................................................... 27 2.3.7 Comparison between Software Quality Models and Quality Measures ......................... 29

CHAPTER THREE: SOFTWARE QUALITY MODEL FOR MEDICAL SIMULATION TOOLS ........................................................................................................................................................ 31

3.1 USER CENTERED DESIGN ............................................................................................................................... 32

VII

3.2 METRICS AND MEASUREMENTS IN THE QUALITY MODEL ................................................................. 37 3.2.1 Effectiveness Measures ...................................................................................................................... 37 3.2.2 Efficiency Measures ............................................................................................................................ 38 3.2.3 Satisfaction Measures ........................................................................................................................ 39 3.2.4 Freedom from Risk Measures ......................................................................................................... 40 3.2.5 Context Coverage Measures ........................................................................................................... 43

CHAPTER FOUR: MEDICAL SIMULATION DSS ....................................................................... 45 4.1 CONCEPTUAL DESIGN OF THE DSS .......................................................................................................... 46 4.2 PERSONAS AND SCENARIOS ......................................................................................................................... 47 4.3 SYSTEM VISUALIZATION ............................................................................................................................. 48

4.3.1 First Cycle .............................................................................................................................................. 48 4.3.2 Second Cycle ......................................................................................................................................... 49

4.4 CHAPTER SUMMARY ...................................................................................................................................... 50

CHAPTER FIVE: INTERACTIVE DECISION MAKING IN ASSESSING THE QUALITY OF MEDICAL SIMULATION TOOLS .............................................................................................. 51

5.1 SCENARIO ONE: EVALUATE A MEDICAL SIMULATION TOOL .................................................... 52 5.2 SCENARIO TWO: MEDICAL SIMULATION TOOLS REVIEW BASED ON THE QUALITY ASSURANCE FACTORS ............................................................................................................................................ 53 5.3 THE NEED FOR A SOFTWARE QUALITY MODEL FOR EVALUATING MEDICAL SIMULATION TOOLS 57 5.4 AIM AND SCOPE OF SOFTWARE QUALITY MODEL STUDY ........................................................... 57 5.5 EXPLORATORY STUDY ............................................................................................................................. 57

5.5.1 Participants ............................................................................................................................................ 57 5.5.2 Apparatus and Materials .................................................................................................................. 59 5.5.3 Test Material ......................................................................................................................................... 59 5.5.4 Task Scenarios ...................................................................................................................................... 59 5.5.5 Task List .................................................................................................................................................. 59

5.6 OBJECTIVE MEASURES OF SATISFACTION ........................................................................................ 60 5.7 SUBJECT MATTER EXPERTS REVIEWS ................................................................................................ 63 5.8 CHAPTER SUMMARY ................................................................................................................................ 64

CHAPTER SIX: CONCLUSION .......................................................................................................... 65 6.1 RESEARCH QUESTIONS ................................................................................................................................. 66 6.2 FUTURE WORK ................................................................................................................................................ 67 6.3 PUBLICATIONS ................................................................................................................................................. 67 6.4 REFERENCES .................................................................................................................................................... 68

APPENDIX A: CONSENT FORM FOR PARTICIPATION IN A RESEARCH STUDY ...... 74

APPENDIX B: SYSTEM USABILITY SCALE ................................................................................. 76

APPENDIX C: PERSONAS ................................................................................................................... 77

APPENDIX D: DSS DATA DICTIONARY ........................................................................................ 78

VIII

LIST OF TABLES

Table 1 Implementing DSSs and Examples from Previous Research ................................................... 20 Table 2 A Synthesis of DSS Categorizations in Medical Simulation ...................................................... 21 Table 3 Square Divisions ......................................................................................................................................... 28 Table 4 Quality Factors and Their Availability in Each Model ................................................................ 29 Table 5 UCD Participants ......................................................................................................................................... 34 Table 6 UCD Summarization and Results ......................................................................................................... 34 Table 7 Effectiveness measures ........................................................................................................................... 37 Table 8 Efficiency Measures .................................................................................................................................. 38 Table 9 Usefulness Measures ................................................................................................................................ 39 Table 10 Risk Mitigation Measures .................................................................................................................... 40 Table 11 Financial Measures ................................................................................................................................ 41 Table 12 Economic Measures ................................................................................................................................ 42 Table 13 Health And Safety ................................................................................................................................... 42 Table 14 Environmental Measures ..................................................................................................................... 43 Table 15 Context Completeness .......................................................................................................................... 43 Table 16 Flexibility Measures ............................................................................................................................... 44 Table 17 Participants Technical Experience ................................................................................................... 58 Table 18 Participants Demographic Information ......................................................................................... 58 Table 19 Task List ....................................................................................................................................................... 59 Table 20 Subject Matter Experts Review .......................................................................................................... 63

IX

LIST OF FIGURES

Figure 1 Comparison between the usage of medical simulation in Medical School Learning and the Hospital ..................................................................................................................................................................... 4 Figure 2 Research Design Methodologies ........................................................................................................... 7 Figure 3 Mapping Medical Simulation Categories ........................................................................................ 13 Figure 4 Surgical Simulations ............................................................................................................................... 17 Figure 5 Cosmetic and Plastic Surgery .............................................................................................................. 17 Figure 6 Dental Simulation .................................................................................................................................... 18 Figure 7 McCall Quality Model ............................................................................................................................. 23 Figure 8 Boehm Quality Model ............................................................................................................................. 24 Figure 9 FURPS Quality Model ............................................................................................................................. 25 Figure 10 Doremy's Quality Model ..................................................................................................................... 26 Figure 11 ISO 9126 Quality Model ..................................................................................................................... 27 Figure 12 ISO/IEC 25000 SQuaRE ...................................................................................................................... 28 Figure 13 User Centered Design ........................................................................................................................... 32 Figure 14 Medical Simulation Software Quality Models ........................................................................... 36 Figure 15 Conceptual Designs for DSS ............................................................................................................... 46 Figure 16 Persona 1 ................................................................................................................................................... 48 Figure 17 Persona 2 ................................................................................................................................................... 48 Figure 18 System Visualization First Cycle ...................................................................................................... 49 Figure 19 Visualization -‐ Iteration 2 ................................................................................................................... 50 Figure 20 Home Page ................................................................................................................................................ 52 Figure 21 Software Quality Model Embedded In the DSS ......................................................................... 53 Figure 22 Medical Simulation Tool Review ..................................................................................................... 54 Figure 23 Efficiency Measure ................................................................................................................................ 54 Figure 24 Medical Simulation Tool Measurement ........................................................................................ 55 Figure 25 Freedom Form Risk Measure ............................................................................................................ 55 Figure 26 View Results as a Chart ....................................................................................................................... 56 Figure 27 SUS Scores ................................................................................................................................................. 61 Figure 28 Relationship between Technical and Medical Simulation ................................................... 62 Figure 29 SUS Average Score for All Participants ......................................................................................... 63 Figure 30 Medical Simulation DSS ....................................................................................................................... 66

X

LIST OF ABBREVIATIONS

AAMC Association of American Medical

Colleges

BSC Balanced Score Card

CPI Collaborative, Participative and

Interactive

DSS Decision Support System

IV&V Independent Verification and

Validation

OR Operating Room

QA Quality Assurance

QME Quality Measure Elements

ROI Return On Investment

SDLC Software Development Life Cycle

SME Subject Matter Expert

SQuaRE Systems And Software Engineering

- Systems And Software Quality

Requirements And Evaluation

SUS System Usability Scale

TORS Transoral Robotic Surgery

UCD User Centred Design

1

Chapter One: Introduction

2

1.1 Medical Simulation Tools The proliferation of medical simulation has recently led to a surge in research investigating

complexity in data analysis and in the evaluation of tools for training and decision making

in different healthcare contexts [1]. Decision-making in medical domains is an increasingly

complex task that involves a number of stakeholders, sub-specialities and technologies.

Simulation is defined as the process of designing a model of a real system [2]. The model

often represents the operation or act of the selected system or process. Experiments are

often conducted using the model for the purpose either of evaluating various strategies for

the operation of the system over time or for understanding the system’s behaviour [2], [3].

Simulation can be classified in different ways depending on the interaction level of users

[4], the context of decision making, and characteristic of modeling [4]. For example,

simulation based on interaction levels of users includes active, passive, and cooperative;

whereas classification based on modeling techniques include three types [4]:

• Live Simulation: Involves real people and equipment performing an activity that

operates for real.

• Virtual Simulation: Involves real people that operate on simulated (computer

controlled) equipment in a virtual environment.

• Constructive Simulation: Involves all the elements including people, equipment

and environment are simulated.

In the past two decades, simulation and modeling have continued to become one of the

leading experimentation techniques, especially for problems associated with uncertainty

[5]. Simulation has been used in assisting decision making in many fields such as education

[6], management [7], and healthcare [8]. Nowadays, a rapidly growing interest has been

emerging in healthcare modeling and simulation in order to improve patient safety and

patient care [9].

Medical simulation is the utilization of technology related to education, training, and

management [9]. It creates a lifelike situation for individuals to practice decision-making

3

and procedural activities in a safe environment for the patients and professionals where it

involves simulated human patients, emergency response and simulated animation.

Evidence suggests that medical simulation improves the effectiveness, safety, and

efficiency in health care services [10][11]. Moreover, it consistently delivers significant

value to the organization, staff, or medical students involved in decision-making [11].

Medical simulation has been reported in the literature as categorizations in different ways.

For example, complexity of visualization, platforms, and contexts of use [12]. One way

which I followed, is categorized into:

● Clinical and Training Simulation: Is a training technique for physicians, medical

students, nurses, and other healthcare professionals, which is used to study, and

analyse the behaviour of diseases, including biological processes in human body

[3]. Example of this category include haptic device (e.g. robotic arm or endoscope

simulation) [13][14][15].

● Operational Simulation: Is a technique for modeling a process and is used for

capturing, and analysing health care operations, patient flow, service delivery, and

scheduling, healthcare business and optimization design [3]. Example of this

category includes the patient flow at the emergency department at a hospital [16].

● Managerial Simulation: Is a type of interactive training and feedback and is used

as a tool for managerial purposes, decision-making, strategic planning, and policy

implementation. Example of this category includes comprehensive management

planning for healthcare processes, staffing, equipment and buildings [17].

● Educational Simulation: Is training and educational techniques, where virtual and

physical objects are extensively used to help a learner explore, navigate or obtain

more information about the environment [3]. Example of this category include

haptic device (e.g. robotic arm or endoscope simulation) [14][15].

Brailsford (2007) classifies medical simulation models into three groups [18] which are;

Models of the human body that include biological processes, Models for modeling patient

flow in the clinic, ward, department, or hospital, and Models for strategies that are used for

strategic planning of the organization.

4

Notably, hospitals and medical schools increasingly relying on medical simulation systems

and tools in different specialties [1]. In

Figure 1 a survey was conducted by the Association of American Medical Colleges

(AAMC) to show uses of medical simulation.

Figure 1 Comparison Between The Usage Of Medical Simulation In Medical School Learning And The Hospital [19]

1.2 Problem Definition Although medical simulation provided ideal approaches for addressing healthcare issues,

the number of successful implementation and development is relatively small compared

with the manufacturing industry [18]. Modern hospitals and clinics are encountering high

levels of competition in both domestic and global markets. In addition, patients, physicians,

medical students and trainees are increasingly demanding for more quality in health care

services delivered with a reasonable cost. Notably, research in [20] has suggested that

medical simulation models needs to be structured in a way that it optimizes the safety and

quality of health care systems [20]. On the other hand, implementing qualified simulation

5

systems may increase the complexity of system and negatively influences the efficiency of

these systems. This consequently led to the emergence of software solutions that focused

more on simulation for basic science education and skill development when compared to

simulation for clinical training [21].

Issues prevalent in the literature that have been cited as challenges in modelling and

training in healthcare setting, which are relevant for medical simulation systems include:

● Complexity and multiple interactions associated with healthcare systems [22].

● High cost of simulation tools [4].

● Medical errors [23].

● Lack of reliable data and relevant tools [24].

● Usability problems and lack of a user-friendly interface [25].

The proposed solution for such issues is applying a software quality model for decision

support systems (DSSs) to evaluate the software quality of medical simulation tools based

on quality assurance factors (e.g. usability, accuracy, efficiency, performance, robustness,

and acceptance) [2]. Healthcare providers, physicians, medical students, trainees and other

healthcare professionals can use and benefit from such system especially in early

stages. Moreover, the selected tool went through a process for evaluation embedded in the

model and then, it will be linked with other tools in the domain and visualized in order to

assist in the decision making process. It is envisaged to help in investigating the

effectiveness of medical simulation tools before buying or approving them for hospitals,

training centres, or education purposes.

1.3 Research Scope This research investigates, develops, implements and tests a software quality model for a

DSS designed for evaluating medical simulation tools based on quality assurance factors.

The proposed software quality model assists decision makers in evaluating medical

simulation tools by embedding the quality factors into the system. Modeling for decision-

making development is an effective method that assists in investigating, testing or

understanding complex simulation tools. So, one of the main thrusts of this research is to

6

evaluate and improve these simulation tools especially in early stages. Developing such a

software quality model can be used as a pre-process, since it provides health care providers

the ability to evaluate the tool before approving it. Stakeholders were involved in the

iterative design of the DSS to assess the software quality model's appropriateness for the

medical simulation tools [26]. To measure and evaluate the proposed DSS, tests were

performed on a representative sample of medical simulation tools in the clinical and

training simulation domain (e.g. surgery, dermatology) and evaluated based on the model

that is embedded in the DSS.

1.4 Aims and Objectives • The main aim of this study is to gain an insight on medical simulations tools, their

challenges, and best practices to overcome these challenges, DSSs, and previous

software quality models.

• Propose a software quality model for the DSS in the context of medical simulation.

• Design and develop a DSS for evaluating medical simulation tools by applying

quality assurance factors and measures.

• Tool analysis on the medical simulation tools through a case study.

1.5 Research Questions How can software systems assist decision makers in assessing the quality of medical

simulation tools?

Subsidiary research questions:

1. Why do professionals need to evaluate medical simulation tools?

2. What are the DSSs used in medical domains?

3. What are the software development models used to design DSS in medical

simulation domains?

4. How can we test the usability, performance, robustness, and accessibility of the

medical simulation tools?

5. What are the widely accepted Software quality models used among Software

Engineers?

7

1.6 Methodology The research methodology was designed to adopt an experimental design strategy [27] and

the methodology used in collecting and analysing the data is a mixed approach (quantitative

and qualitative). The approach involved a User Centred Design (UCD) methodology. This

methodology is to ensure efficient, satisfying, and user-friendly experiences for the user

[28] because the product will be designed and developed based on the perspective of how it

will be understood and used by the stakeholder [29]. The flow of the research design

methodology is illustrated in Figure 2.

Figure 2 Research Design Methodologies

To elaborate more on the methodology lifecycle, each work phase is described in detail in

the following sections.

8

1.6.1 Work Phase One: Data Collection In this research, the data was collected through a mixed approach which combined

quantitative and qualitative data. Two methods were applied for data collection:

1. Literature Review: A systematic literature review of published work [30] in

medical simulation and software quality models computing has been applied to

help in understanding the research directions, best practices and collect the

required data for designing and developing an embedded medical simulation

software quality model in a decision support system (DSS).

2. User Centred Design (UCD): The ISO standard for the UCD methodology

has been applied in this research to ensure that the end users' needs and

limitation are taken into consideration [31]. Moreover, this participatory design

methodology facilitates the creation of efficient, satisfying, and user-friendly

systems because the product was designed and developed based on the

perspective of how it will be understood and used by the stakeholders [29].

This phase is described in details in chapter 2 and 3.

1.6.2 Work Phase Two. Designing the Model This phase involved a knowledge background that was gained after applying the

systematic literature review and user centred design methodology. This background

knowledge and insights into design considerations for simulation contexts in

general and medical domains in particular, guided the design and development of

the quality model for medical simulation software. This quality model adhered to

the International Organization for Standardization (ISO) standards. The ISO

standard selected to be followed is Systems and Software Engineering - Systems

and software Quality Requirements and Evaluation (SQuaRE) [32]. This phase is

described in details in chapter 3.

1.6.3 Work Phase Three. Developing the Model After designing the medical simulation software quality model, the DSS was

developed and the quality model was embedded in the system with the layers of

interfaces to provide intuitive interaction. The medical simulation software quality

9

model has been embedded in the system for the purpose of assisting decision

makers in evaluating medical simulation tools. The Software Development Life

Cycle SDLC guided this phase where requirements, analysis, design and

development adopted. This phase involved the creation of personas, to enhance

engagement and build empathy in the design process by gaining a clear

understanding of the DSS and it is targeted users [33]. Afterwards, three iterative

cycles of system visualization have been done. This phase has been described in

details in chapter 4.

1.7 Contribution In this thesis, the contribution of embedding a software quality model in a DSS for

evaluating medical simulation tools was described as it evolved in iterative cycles of

development. In addition, the design approach of UCD in developing the DSS and the

participatory design activities in assessing the software quality model with healthcare

decision makers can be emulated in similar contexts of software design and development.

1.8 Thesis Structure Chapter 2 is a review of literature related to medical simulation categorization, challenges,

human factors, and tools, in addition, to software quality models, and DSS. In Chapter 3,

we discuss and present Phase 2 that covers the medical simulation software quality model.

Chapter 4 “describes Phase 3, the DSS used for evaluating medical simulation tools. In

Chapter 5, a case study has been applied on the DSS including the software quality model.

We conclude the thesis in Chapter 6 by presenting how all the objectives and research

questions have been achieved during the work in this thesis, future work and publications.

10

Chapter Two: Literature Review

11

Quality models of medical simulation are an emerging field of research in software

engineering [34]. The design and development of DSSs often involves a good

understanding of the functional requirements of the systems in addition to the best practices

and standards in the domains relevant to the applied context [35]. However, the knowledge

that underlies the quality models of DSSs, either knowledge-based or model-based, is

unfortunately scattered throughout the literature. This chapter presents a systematic

literature review of published work in medical simulation and software quality models

computing in a way that helps software engineers in understanding the research directions

and best practices for designing and developing a DSS for medical simulation tools.

In this chapter, we start by presenting an overview on medical simulation and it is

challenges and DSSs. Following that, a review of software quality models and standards

examines the quality assurance factors used to assess DSS. We conclude with a comparison

between previous software quality models, and design recommendations for software

quality models in the context of medical simulation.

2.1 Medical Simulation Medical simulation is the utilization of technology related to education, training, and

management in medical contexts [10][20]. Simulation creates a lifelike situation for

individuals to practice decision-making and procedural activities in a safe environment for

the patients and professionals. For example, simulation can provide scenarios in which it

involves simulated human patients in clinical procedures or surgery [14]. Another

simulation example in medical contexts is in emergency response systems such as the

system in [36].

Evidence suggests that medical simulation improves the effectiveness, safety, and

efficiency in health care services [10] [11]. Moreover, it has been shown to consistently

deliver significant value to the organization, staff, or medical students in decision-making

[11][20].

12

2.1.1 Medical Simulation Categorizations Medical simulation has been reported in the literature as categorizations in different

ways. For example, complexity of visualization, platforms, contexts of use [12]. In

this section, we examine the different categorizations of medical simulation and

map the relevant categorizations across sub-domains to gain an insight into common

themes, trends and challenges in the simulation categories.

In [3], Barjis et al. categorized medical simulation tools into four categories as

clinical, operational, managerial, and educational.

• Clinical and Training Simulation: Is a training technique for physicians,

medical students, nurses, and other healthcare professionals, which is used

to study, and analyse the behaviour of diseases, including biological

processes in human body [3]. Example of this category include haptic device

(e.g. robotic arm or endoscope simulation) [13][14][15].

• Operational Simulation: Is a technique for modeling a process and is used

for capturing, and analysing health care operations, patient flow, service

delivery, and scheduling, healthcare business and optimization design [3].

Example of this category includes the patient flow at the emergency

department at a hospital [16].

• Managerial Simulation: Is a type of interactive training and feedback and

is used as a tool for managerial purposes, decision-making, strategic

planning, and policy implementation. Example of this category includes

comprehensive management planning for healthcare processes, staffing,

equipment and buildings [17].

• Educational Simulation: Is training and educational techniques, where

virtual and physical objects are extensively used to help a learner explore,

navigate or obtain more information about the environment [3]. Example of

this category include haptic device (e.g. robotic arm or endoscope

simulation) [14][15].

13

In [18], Brailsford (2007) classifies medical simulation models into three groups a

models of the human body that include biological processes, models for modeling

patient flow in the clinic, ward, department, or hospital, models for strategies that

are used for strategic planning of the organization.

In [37], Kathleen R. Rosen MD classified medical simulation into five types;

including Standardized patients, Human Patient Simulation, Virtual reality, Task

trainers, Software-based simulation. Researchers have varied in categorizing

medical simulation, where each author has a different name for the same category.

The following Figure 3 highlights the mapping.

Barjas [3] Brailsford[18] Kathleen R. Rosen MD [37]

Clinical Human body Standardized patients, Human patient Simulations

and Task trainer.

Operational Patient flow Virtual reality and Software-based.

Educational Human body Standardized patients, Human patients and Task

trainer.

Managerial Strategies Software-based simulation.

Figure 3 Mapping Medical Simulation Categories

2.1.2 Medical Simulation Challenges

Medical simulation is a multi-disciplinary and involves increasingly complex

problems and rapidly developing fields [3]. It is a theoretical and practical domain

that focuses on multi-methods, multi-paradigms, multi-modeling and multi-

disciplines [3][38]. One of the main challenges in medical simulation is the need for

medical simulators that truly apply multi-modeling to present visual, auditory,

haptic, and olfactory displays [39]. And based on that, the correlation of the

14

different displays is the challenge that lies here because there should be proper

relation between user perceives sensory cues and to user interactions [3][39].

Medical simulation values and benefits that are produced to improve clinical,

operational and management processes are clear and easy to perceive nowadays

[38]. And although medical simulation is now more known and prepared in the

healthcare industry, it is still a sophisticated and a highly technical tool for non-

technical user’s comprehension. This challenge triggers user resistance, which is

considered as barrier to a successful simulation implementation in healthcare. This

barrier exists in with the fact that detailed simulation such as medical simulation,

requires tremendous time and effort [38]. In addition, evidence has shown that

Human-Simulator Interfaces are often critical [25]. In order to avoid errors, poor

training, and to provide a complete solution for different cases in medical

simulation (e.g. applying and surgery on a patient, or training for healthcare

practices), a medical simulator designed interface should replicate that of the real

world and easily used [25]. Thus, the physician, medical student or any healthcare

practitioner should touch instruments that provide the same feel and look as their

real world counterparts. These tool’s requirements can be challenging, especially if

the simulator addresses open surgery where the degrees of freedom and the numbers

of instruments are significantly larger than in minimally invasive surgery. As a

conclusion for the previous challenge, user acceptance is an important matter in

healthcare simulation [28].

A simulation model can only provide good and accurate results based on the input

data, although the data collection is a challenge in healthcare simulation [39]. In

healthcare, often the medical simulation tool developers lack sufficient input data

for their simulation models, which leads to delivering rather approximate results

[38]. Data collection is a challenge due to not available useful formats for historic

data; data collection should take place over a long span of time; meeting with

healthcare professionals for gathering data collection and verification purpose is

also a hard task due to their busy schedules [3]. The input data need to be complete,

accurate and real. The entered data play a big role in assessing health professional in

15

decision making since they are considered as DSSs. In order to provide ideal data

collection it may require integrating simulation models with the organization

information systems to support the daily operation [38].

Validation and verification in medical simulation is a subject of extensive research

[40]. It is considered as a real devil because without applying verification and

validation methods, it would be risky, if not disastrous, to make any decisions or

forecasts based on the model outcomes [41]. To overcome this challenge,

innovative modeling approaches, model validation, especially for complex models

should be used [42]. And in order to enhance model verification an approach such

by applying CPI modeling, in which models are designed with the collaboration of

the users using the medical simulation tool and business owners [40]. Moreover,

validation quite differs from the verification process. A significant research

challenge may increase when developing a valid simulation model, designing valid

experiments based on the model, and carrying out a rigorous analysis of the

experiments [42].

The cost, is a major issue for medical modeling and simulation [40]. Although the

popular perception may be that the medical enterprise is well funded, the reality,

especially in medical education, is quite the opposite. The cost of simulators must

be significantly reduced if they are to become commonly available tools within the

medical school curriculum.

To summarize medical simulation challenges, evidence have shown that the cost is

major challenge. In addition, the validation and verification for medical simulation

tools is also considered as a challenge. Also, the complexity and data entered is a

challenge.

2.1.3 Human Factors in Medical Simulation Although medical simulation provided ideal approaches for addressing healthcare

issues, the number of successful implementation and development is relatively

small compared with the manufacturing industry [18]. Modern hospitals and clinics

are encountering high levels of competition in both domestic and global markets. In

addition, patients, physicians, medical students and trainees are increasingly

16

demanding for more quality in health care services delivered with a reasonable cost.

Notably, research in [20] has suggested that medical Simulation models needs to be

structured in a way that it optimizes the safety and quality of health care systems

[20]. On the other hand, implementing qualified simulation systems may increase

the complexity of system and negatively influences the efficiency of these systems.

This consequently led to the emergence of software solutions that focused more on

basic science education with simulation than less clinical training.

Human error plays a crucial role in the safety of medical simulation tools. Human

errors can frequently be traced back to deficiencies in the design of the human-

machine interface as been highlighted in the literature review [24]. If the system and

interface design was not designed with human capabilities and by considering the

limitations of the cognitive, perception and physical human factors, physician,

operators and healthcare providers are being placed in situations where the demands

imposed on them are unrealistic from a psychological point of view [43].

Subsequently, the result will be an inevitable error. The discipline of human factors,

or ergonomics, deals with the highlighted medical simulation issues and challenges

by designing interfaces that take into account human capabilities and limitations.

The lack of attention to human factors during the design phase seriously jeopardizes

the human safety. Following design principles related to medical simulation may

decrease human error and lead to a better understanding and using for the tools.

As an example of human factors design principles that can be adopted when

designing medical simulation tools include [43][44]:

• User should be provided with prompt feedback after each action.

• Make the functions of the various controls clear and obvious.

• Displayed messages should be easy to understand.

• Minimize the load on the users’ memory as much as possible.

• Increase efficiency by providing users with shortcuts.

• Performance and clinical evaluation.

• Background information of the medical simulation tool. Example of this

category includes intended purpose of the device, device components

17

general warnings conditions of device use, supplies and materials, and user

preparation.

2.1.4 Medical Simulation Types Medical simulation tools have been successful in the clinical, training and learning

context of healthcare. In this thesis, we are focusing on the clinical and training

simulation. Evidence has shown that medical simulation is increasingly being

adopted in surgical, dental and cosmetic usage contexts. Examples on such medical

simulation tools are illustrated in Figure 4, Figure 5, and Figure 6.

Figure 4 Surgical Simulations [45]

Figure 5 Cosmetic and Plastic Surgery [45]

18

Figure 6 Dental Simulation [45]

The simulation tools vary on the spectrum of complexity from simple limited scope

training on specific skills which implement direct-manipulation interfaces, to more

complex interactive procedure that require more multimodal interaction (e.g.

tangible interfaces and tactile feedback).

2.2 Decision Support Systems (DSSs) DSSs emerged in the 1970. It is often defined as a set of computer programs designed to

assist, and engage decision makers and supports individuals (e.g. stakeholders) in decision-

making and analysis as highlighted by [46] [47] [48].

The decision process refers to some techniques that include: gathering and exchanging

information; identifying scenarios, and producing a final choice among a set of options

[46][49]. A DSS can be classified as passive, or active and cooperative with regards to its

mediating role in decision [50][51]. Approaches applied for developing or categorizing

DSSs range from data levels to model-driven DSS categories [48][50][51].

2.2.1 Data-driven Data-driven DSSs are targeted by managers, staff and also product/service

suppliers. The data is used for querying a database or a data warehouse to find

answers for specific purposes. A Data-Driven DSS can provide the highest level of

functionality when applying On-line Analytical Processing. It is designed and

implemented via a mainframe system, client/server link, or via the web.

2.2.2 Knowledge-based Knowledge-based DSSs are systems with specialized problem-solving expertise.

Knowledge-based DSSs contain knowledge about a particular domain, the ability to

19

understand problems within that domain, and also can solve some of these problems.

It is designed and implemented via the web, or software running on stand-alone PCs.

2.2.3 Document-driven Document-driven DSSs uses processing technologies and computer storage to

provide document retrieval and analysis. Examples on this type of DSS are

databases that may include scanned documents, hypertext documents, images,

sounds and video. To access these kinds of documents a search engine is needed to

be associated with a document-driven DSS.

2.2.4 Communication-driven Communication-driven DSSs use network connection and communications

technologies to facilitate decision-relevant communication. This type of DSS assists

in conducting meetings, or for individual’s collaboration. Examples include tools

that use computer-based bulletin boards, video conferencing and groupware.

2.2.5 Model-driven Model-driven DSSs are systems that assist in analysing decisions. This type of DSS

emphasizes access to and manipulation model, optimization or financial. The data

and parameters used are limited in order to aid in decision-making. These DSS s can

be designed and implemented via software/hardware in stand-alone PCs,

client/server systems, or the web.

These DSS types also differ based on the type of communication network whether it is

LAN based or Web-based [52] as shown in Table 1.

20

Table 1 Implementing DSSs and Examples from Previous Research

Technology

DSS Type LAN Based Web-Based

Data-driven Thick client Thin client

Examples: [53] Examples: [54]

Knowledge - based Standalone PC Shared knowledge

Examples: [55] Examples:[56], [57],[58], [59]

Document-driven Limited to certain files such as Doc., .xls

Not limited to certain files such as HTML and search engines


Communication-driven Narrow scope Global scope


Model-driven Single user Multiple users

Examples: [62] Examples:[56], [63]

In the past 50 years, researchers have pointed of the rapidly growing computational

complexity in simulation which consequently lends itself to playing an increasingly

important role in decision-making in healthcare. In recent years, most of the DSSs are

considered as knowledge-base [63][64]. Evidence has suggested that medical DSSs have

been beneficial and considered as computational artefacts that assess various decision

making tasks in the medical domain (e.g. diagnosis, therapy planning, interacting with

patients) [58]. The authors I. Fatima, M. Fahim, D. Guan, Y.-K. Lee, And S. Lee, have

pointed out that in 1961 the first mathematical equation to aid in the diagnosis of congenital

heart disease has been proposed by Warner et al. and from this work many clinical DSSs

have been developed [64]. The most common Clinical Decision Support System (CDSS)

applications include alerts, diagnostic assistance, decision support in the prescription,

information retrieval, image recognition, and therapy critiquing and planning [65][66].

The authors I. Fatima, M. Fahim, D. Guan, Y.-K. Lee, And S. Lee, have designed and

developed a socially interactive clinical DSS where it can interact and automatically learn

21

new knowledge from users’ experience [64]. In addition, the proposed system supports user

high-level context recognition ability. Traditional CDSSs require physicians or patients to

manually enter patient information, which are used in decision-making. The proposed

CDSS is intelligent and generates patient-oriented decision when more related information

obtained.

A synthesis of DSS Medical simulation is listed in Table 2. To reflect on trends in the

design of DSS systems, this synthesis aggregates the DSS healthcare systems in a lens

looking at types which were reported in the literature and mapped to the DSS category. The

mapping in Table 2 refer to the DSS types as follows:

a. Data-driven b. Knowledge-driven

c. Document-driven d. Communication-driven

e. Model-driven

Table 2 A Synthesis of DSS Categorizations in Medical Simulation

Tools name

a b c d e

Dentist [45] x

CAE Healthcare AccuTouch [67]

x

The GI-Bronch Mentor [67]

x

Transoral Robotic Surgery (TORS) [68]

x

DentalNavi [45] x

Kidneys [45] x

Touch surgery [45] x x

Ear surgery [45] x

Heart surgery [45] x

22

Virtual dentist [45] x

Radiology 2.0: One Night in the ED [45] x

Radiology 2.0: Pregnant Appendicitis [45] x

Knee surgery [45] x

iURO Kidney [45] x x

Plastic surgery simulator [45] x

Virtual dentist [45] x

Medscape [45] x

2.3 Software Quality Models In software or anything else evaluating the quality means measuring value. A product with

a lower quality has less value than a product with a higher quality. Measuring and

evaluating the quality of software products is considered to be a fundamental task in the

different organizations [69][70]. Poor quality may lead to a diversity of issues such as task

failure, financial loss, permanent injury, or in critical systems it may lead to human life loss

[69]. Software quality has several definitions, for example, IEEE define it as the degree to

which a system, component, or procedure meets specified requirements. And in order to

meet the specified requirements software quality assurance factors should be applied due to

the variety and complexity of software which is increasing day after day [69]. To improve

the quality of software products and make them measurable, models containing quality

assuring factors have been developed and applied. These models are usually used to

support stakeholders in evaluating the quality of a software [70].

In the software engineering and systems engineering literature, several software quality

models have been repeatedly reported in different applied contexts, which are McCall’s

Quality Model, Boehm’s Quality Model, Dromey's Quality Model, FURPS Quality Model,

ISO 9126 Quality Model. Each model’s content is discussed briefly in the following

sections.

23

2.3.1 McCall’s Quality Model McCall Quality Model is a quality model presented by Jim Mccall in 1977 [71]. It is

one of the frequently cited predecessors of quality models in general and is

primarily aimed towards system development process and system developers

[71][72]. The McCall quality model mainly bridges the gap between users and

system developers by concentrating on a number of software quality factors that

reflect both the users’ vision and the developers’ priorities [72]. The McCall

quality model has three major characteristics for defining and identifying the quality

of a software product as shown in Figure 7.

Figure 7 McCall Quality Model [72]

2.3.2 Boehm’s Quality Model In 1978 Barry W. Boehm presented a quality model that addresses the

contemporary shortcomings of models by prediction. This quality model works by

quantitatively and automatically evaluating software’s quality by applying a set of

attributes and metrics [73]. Moreover, Boehm's model and McCall quality model

are similar, for example their quality model is structured in a hierarchical way as

24

shown in Figure 8. This model decomposes its characteristic into high-level

characteristic, intermediate level characteristic and primitive level characteristics.

Figure 8 Boehm Quality Model [72]

2.3.3 FURPS Quality Model Robert Grady and Hewlett Packard proposed a model called FURPS in 1987 [74].

The term FURPS is an abbreviation, which refers to Functionality, Usability,

Reliability, Performance and Supportability. FURPS quality model decomposes its

characteristics in two classifications of requirements that are functional requirement

and non-functional requirement [74] as shown in Figure 9. Functional requirement

define a specific behaviour where it has an input and an expected output, while non-

functional requirement that only include Usability, Reliability, Performance and

Super-operability and also called as URPS.

25

Figure 9 FURPS Quality Model [34]

2.3.4 Dromey's Quality Model R. Geoff Dromey presented a quality model in 1995. This model is considered as

one of the recent models that are similar to the McCall’s, Boehm’s and the FURPS

quality models [75][76]. Dromey quality model is a product based quality model

that states that the evaluation differs for each product which leads to a dynamic idea

of modeling is required [75]. Moreover, the model seeks to increase the

understanding due to the relationship between the attributes (characteristics) and

sub-attributes (sub- characteristics) of quality as shown in Figure 10.

26

Figure 10 Doremy's Quality Model [72]

2.3.5 ISO 9126 Quality Model ISO 9126 is an international standard for the evolution of software. The standard is

decomposed of four parts, which address the following subjects; Quality model

9126 Part-1, External metrics 9126 Part-2, Internal metrics 9126 Part-3, Quality in

use metrics 9126 Part-4 [72]. The ISO quality model is an extension to previous

quality models presented by McCall, Boehm, FURPS and Doremy [72]. This

international quality model evaluates the quality of a software product in terms of

internal and external quality assurance factors and their connection to characteristics

[72], [77]. ISO 9126 structure is decomposed of 2 levels. The highest level includes

the quality characteristics and the lowest level includes of the software quality

criteria (e.g. metrics [77]). The quality model has six characteristics consisting of

Functionality, Reliability, Usability, Efficiency, Maintainability and Portability as

shown in Figure 11. Moreover, these characteristics are further decomposed of 21

sub characteristics. The set of characteristics defined by the ISO 9126 quality model

are applicable to every kind of software [77].

27

Figure 11 ISO 9126 Quality Model [78]

2.3.6 ISO/IEC 25000 SQuaRE Although ISO/IEC 9126 have been considered as the best software quality model

since it is an extension and an enhanced version of previous software equality

models [79], evidence have found that it has major flaws [32]. An Example on such

flaws are ISO 9126 measurements is not consistent with the science and engineering

measurements so there is a real concern regarding validating the proposed measures

[79]. Therefore, ISO 9126 has been replaced with ISO/IEC 25000 SQuaRE due to

its extensive series of standards [72]. The SQuaRE is divided to five divisions as

shown Figure 12, which are the Quality Management division 2500n, Quality

Model Division 2501n, Quality Measurement Division 2502n, Quality

Requirements Division 2503n, and Quality Evaluation Division 2504n. The

SQuaRE division is elaborated more in Table 3[32].

28

Figure 12 ISO/IEC 25000 SQuaRE [32]

Table 3 Square Divisions

Type Description

ISO/IEC 2501n

Quality Model

Division

This division consists of detailed quality models for software

products, quality in use, data, and guidelines on how to use this

model.

ISO/IEC 2502n

Quality Measurement

Division

This division consists of software quality measurement model,

metrics, and guidelines on how to use this model.

ISO/IEC 2503n

Quality Requirements

Division

This division consists of software quality measurement model,

metrics, and guidelines on how to use this model.

SO/IEC 2504n Quality

Evaluation Division

This division consists of requirements, recommendations and

guidelines for evaluating a software product.

ISO/IEC 25050 25099

SQuaRE Extension

Division

This division consists of the Common Industry Formats for

usability reports and Requirements for quality of Commercial

Off-The-Shelf software.

29

2.3.7 Comparison between Software Quality Models and Quality

Measures Table 4 highlights the quality assurance factors and their availability in each

software quality model. Such comparison between the software quality models

assist in choosing the right model.

Table 4 Quality Factors and Their Availability in Each Model

Criteria Mccall Boehm Dormey ISO 9126

ISO/IEC 25010:2011

Correctness x x

Reliability x x x x x

Integrity x x

Usability x x x x x

Efficiency x x x x x

Maintainability x x x x x

Testability x

Interoperability x

Flexibility x x

Reusability x x x

Portability x x x x x

Clarity x

Modifiability x

Documentation x

Realistic x

Understandability x

Validity x

Functionality x x x

Generality

30

Economy

Compatibility x x

Security x x

31

Chapter Three: Software Quality Model

for Medical Simulation Tools

32

3.1 User Centered Design Since involving stakeholders provides a better understanding of evaluation and decision

making process a User Centered Design method has been applied in this research. A User

Centred Design (UCD) is a method in which the desires, requirements, and limitations of

end users of a product, process, or service are taken into consideration at each stage of the

design process [28].

Figure 13 User Centered Design

UCD has been applied by the semi-structured interview sessions conducted throughout the

project with professionals in both healthcare and software quality subject-matter domains in

different organizations. Semi-structured interviews are interviews with a small duration to

collect new ideas that are be brought up from the interviewee during the interview [80]. The

interviews conducted with these participants were designed as participatory design sessions

in which insights were elicited on specific aspects of the system. Sessions were carried out

33

in a semi-structured guided dialog mode, and ranged in duration from brief 20min sessions

to 50-60 sessions; overall, these sessions didn’t take more than one-hour conversations with

participants. We considered the DSS context; objectives; environment and goals. A set of

questions was asked to them in order to gain feedback and insights were integrated as

design considerations for the iterative cycles of developments of the DSS.

Semi-structured interviews were conducted as part of the UCD activities in which

participants from the group listed in Table 5 were presented with a set of enquiry-driven

questions. These questions include:

• What are the best techniques to evaluate a product?

• How is a product evaluated in your organization?

• What are the software quality models used in your organization?

• What is technical software quality model used in your company?

• Have you ever used a medical simulation system in the Operation Room (OR)

room? If yes, what are they?

• Are they easy to use? If not, what are the bottlenecks?

• How common is medical simulation used in your hospital?

• Have you been trained on these tools? If yes, for how long?

• Is there any DSS for medical simulation tools?

Answers were annotated and content analysis was conducted to elicit themes and trends in

the insights provided by the participants in this UCD activity. The information and

feedback from the participants were gathered as an aggregate set of design

recommendations, summarized in Table 6.

34

Table 5 UCD Participants

Participants Position Organization or

Company

Meeting day and

time

P1 Senior Project

Manager

PriceWaterhouseCooper 18-Feb-2015

P2 Quality Assurance

Manager

Saudi Skills Standards 25-Feb-2015

P3 Quality analyst Tamkeen Technologies 7-Mar-2015

P4 Dentist King Khalid University

hospital

9-Mar-2015

P5 Dental student Riyadh collage of

Dentistry and Pharmacy

15-Mar-2015

P6 Medical student King Saud University

P7 Dermatologist National Guard Hospital 2-Jan-2015 and 14-

Feb-2015

P8 Surgeon National Guard Hospital 1-April-2015

P9 Plastic surgeon National Guard Hospital 2-Mar-2015

Table 6 UCD Summarization and Results

UCD Results

• Most of the quality managers commonly follow standards in their organizations to

ensure product quality.

• Most of the software engineering domain participants that were interviewed follow

the ISO standards in their work.

• Organization commonly evaluates a product’s quality by following international

standards.

• The physicians using medical simulation tools are complaining from the tool’s

35

complexity, efficiency, usability, and high risk and considering them as a

bottleneck.

• The medical simulation tools are mostly abandoned due to their complexity and

difficulty to use.

• Using the medical simulation tools in a wrong way will affect the patients’ safety.

• Medical simulation tools are cost affective so they should be selected based on

measurements.

• It takes many days, effort and resources to be trained on medical simulation tools.

• Medical simulation tools are not commonly used in hospitals in the region, and in

Saudi Arabia in particular.

The literature in this research has highlighted medical simulation categories, challenges,

human factors and previous software quality models. After a long period of search in

software quality models and standards, the ISO and International Electro Technical

Commission (IEC) has been selected that form the specialized system for worldwide

standardization [32] in designing and developing the Software quality model. Findings

have led to creation of a proposed aggregate model for our DSS system; the Medical

Simulation Software Quality Model shown in Figure 14 that best matches and overcomes

the challenges found in the literature and by applying the UCD.

It is good practice to base the evaluation process on internationally agreed upon standards

which provide an authoritative source of reference. The selected standard is the Systems

and software engineering - Systems and software Quality Requirements and Evaluation

(SQuaRE) standard. It has been chosen due to its replacement of the ISO 9126. Moreover,

as highlighted in the literature the SQuaRE standard is an improved version of the ISO

9126 [81] that is an extension of previous work done by Mccall quality model which is

considered as the gold standard of software quality models [72]. Therefore, the presented

software quality model is a combination of previous software quality models with a focus

on criteria specific to medical simulation tools.

36

Figure 14 Medical Simulation Software Quality Models [32]

37

This International Standard defines quality in use measures for the characteristics listed in

ISO/IEC 25010, and is intended to be used together with ISO/IEC 25010. This International

Standard contains an explanation of how to apply software and computer system quality

measures, a basic set of quality measures for each characteristic, an example of how to

apply quality measures during the product life cycle. It includes as informative annexes a

quality in use evaluation process and a reporting format.

3.2 Metrics and Measurements in the Quality Model The models are comprised of key measurements for assessing the software product’s

quality. The measures were elicited form the referenced models and in this section, we

describe the key metrics for each measurement.

3.2.1 Effectiveness Measures Effectiveness measures ensure the completeness and accuracy with which users

achieve a specific requirement. And a QME is a quality measure element method.

An elaboration on this measure described in details in Table 7.

Table 7 Effectiveness measures [32]

Task name Description QME

Task completion What are the tasks completed correctly?

Z = A/B

A = number of tasks completed.

B = total number of tasks attempted.

Effectiveness task How much is the proportion of the required goal is achieved correctly?

{X = 1-∑Ai | X>0}

Ai= proportional value of each missing or incorrect performance in a task output.

Error frequency What is the error frequency made in a task by the user compared to a target value?

X= I/J

I= number of errors made by the user.

J= number of tasks.

38

3.2.2 Efficiency Measures Efficiency measures assess the accuracy and completeness in relation to resources

expansion with which users achieve a specific requirement. An elaboration on this

measure described in details in Table 8.

Table 8 Efficiency Measures [32]


Time How long does it take to complete a task.

W = Tt/At

Tt = target time.

At= Actual time.

Relative task time A comparison between a user completing a task and an expert.

X= J/I

I= normal user’s task time.

J= expert user’s task time.

Efficiency

A task compared with the target?

W = (Tt – At) / Tt

Tt = target time.

At= Actual time.

Task efficiency How efficient are the users? X = Te/ T

Te = task effectiveness.

T = task time.

Relative task

A comparison between the user efficiency and a target.

X= I/J

I= normal user’s task efficiency.

J= target task efficiency.

Economic productivity How cost-effective is the user? W = Te / C

Te = task effectiveness.

C = total cost of the task.

39

Productive proportion What proportion of the time are the user performing productive actions?

Z= Ta / Tb

T a = productive time

Tb = task time

Relative number of user actions

Is the minimum number of actions needed performed?

X=I/J

I= Number of actions performed by the user

J= Number of actions needed

3.2.3 Satisfaction Measures Satisfaction measures is the degree to which user is satisfied when using a software

product as shown in Table 9.

• Usefulness Measures Usefulness measures are the degree to which a user is satisfied with their

achievement of requirement, and result of using a software product. This measure is

related to the usability measures which have evolved over the years with their own

quality assurance factors specific to subjective and objective measures of user

experiences with interactive software products. An elaboration on this measure

described in details in Table 9.

Table 9 Usefulness Measures [32]


Discretionary usage

How much potential users using the system?

X = I/J

I= number of times that

specific software requirements

are used.

J = number of times intended to

be used.

Discretionary utilization of What is the average utilization X = ∑(Ai)/n

40

3.2.4 Freedom from Risk Measures Freedom from risk measure is the degree to which a software product mitigates a

potential risk to an environment, health, human life, and economic status.

• Risk Mitigation Measures Risk mitigation measure is the process of mitigating a potential risk to an

environment, health, human life, and economic status performed by a product

qualities (functional suitability, performance efficiency, compatibility, usability,

reliability, security, maintainability or portability). An elaboration on this measure

described in details in Table 10.

Table 10 Risk Mitigation Measures [32]


Risk mitigation

To what extent can product quality mitigate risk?

Z = A/B

A = Risk with high quality.

B = Risk with low quality.

functions

of functions?

Ai =Proportion of users using

function i.

B= number of function i.

Customer complaints

What proportion of complaints

submitted by customers?

Z = A/B

A = number of complaining

customers.

B = total number of customers.

41

• Financial Measures Financial measures are the process of assessing financial status related to economic

objectives, commercial property, efficient operation, and reputation. An elaboration

on this measure described in details in Table 11 and Table 12.

Table 11 Financial Measures [32]


Return Of Investment (ROI)

What is the return on investment?

Z = A/B

A = Benefits obtained.

B = Invested amount.

Time to achieve a return of investment

Is a return on investment achieved in an acceptable time?

Z = A/B

A = Time to reach ROI.

B = Acceptable time to reach ROI.

Relative business performance

How the business performance is compared to top class companies in the industry or in same business?

Z = A/B

A= IT investment amount or sales of the company.

B= IT investment amount or sales of target company

Balanced Score Card (BSC)

Do the benefits of IT investment evaluated using the Balanced Score Card meet objectives?

Z = A/B

A= BSC results.

B= BSC objectives.

Delivery time

Does delivery time and the length and number of late deliveries meet targets?

Z = A/B

A= Actual delivery times or late deliveries.

B= Target for delivery time or late deliveries.

Missing items

Do the number of missing items meet targets?

Z = A/B

A = Actual missing items

42

Table 12 Economic Measures [32]

• Health and Safety Measures Health and safety measures assess risky health and safety factors. An elaboration on

this measure described in details in Table 13.

Table 13 Health And Safety [32]


User health and safety frequency

Calculate the health problem among users of the software?

Z = A/B

A = number of users reporting health problems.

B = total number users.

User health and safety impact What is the safety of users X=N*T*S

B = Target missing items.


Revenue for each customer

Does the Revenue for each customer meet targets?

Z = A/B

A = Actual revenue from each customer.

B = Target revenue for customer.

Error with economic consequences

What is the frequency and size for human or system errors?

Z = A/B

A = Number of errors with economic consequences

B = Total number of usage situation.

Software corruption What is the frequency and size of software corruption?

Z = A/B

A= Number of occurrences of software corruption.

B= Total number of usage situations.

43

using the product? N = Number of affected people

T = Time

S = Degree of significance

Safety of people affected by use of the system

What is the incidence of hazard to people affected by use of the system?

X= I/J

I= number of people put at hazard

J = total number of people potentially affected by the system.

• Environmental Measures Environmental measures assess risky environmental factors. An elaboration on this

measure described in details in Table 14.

Table 14 Environmental Measures [32]


Environmental impact

What is the environmental impact of using software?

Z = A/B

A = environmental impact B = acceptable impact.

3.2.5 Context Coverage Measures Context coverage measures is a process ensuring the degree to which a product or

system can be used with effectiveness, efficiency, freedom from risk and

satisfaction in specified contexts of use. An elaboration on this measure described in

details in Table 15.

Table 15 Context Completeness [32]


Context complete

Proportion of the context of use a product can be used with acceptable usability?

Z = A/B

A = Number of contexts with unacceptable usability.

44

B = Total number of distinct contexts of description use.

• Flexibility Measures Flexibility measures is the process to ensure the degree to which a product or

system can be used with effectively, efficiently, free from risk and with satisfaction

in contexts and with flexibility. An elaboration on this measure described in details

in Table 16.

Table 16 Flexibility Measures [32]


Flexible context of use

How can the product be used with additional context of use?

Z = A/B

A = Number of additional contexts in to ensure usability

B = Total number of additional contexts in which the software might be used

Flexible design features

How much can the product adapt to meet the user needs?

X=I/J

I = Number of design features with compete flexibility

J = Total number of design features

45

Chapter Four: Medical Simulation DSS

46

4.1 Conceptual Design of the DSS The decision support system is designed to facilitate access to disparate information for a

collection of medical simulation tools. The quality assessment of software is a

comprehensive process of objective and subjective measures that need to be quantified to

aid in the sense-making process. To this end, the concept of the system was to build layers

of intuitive interaction on the aggregate software quality model. Minimalist approach was

considered in the interfaces so as to maintain the focus of building the knowledge base

when new medical simulation tools are introduced. In addition, to facilitate exploratory

approaches of trend analysis when querying the knowledgebase. Figure 15 depicts the high-

level concept diagram in which the users, tasks and contexts of use are envisioned. It is also

shown in the figure the DSS input and output.

Figure 15 Conceptual Designs for DSS

47

4.2 Personas and Scenarios

In requirements engineering [82], a technique called “persona” is often considered when

systems are designed for non-computational disciplines so as to accurately reflect on the

potential users of the system. Involving personas, which are essentially fictitious characters

to document technical, domain-knowledge and socio-economic profiles of target users, is as

an interaction design technique that assists in a software product development lifecycle

[33]. Personas are considered as design aids and are often elicited from the domain

knowledge or in participatory design sessions such as semi-structure interviews of UCD

activities, nevertheless they are considered fictional characters representing people who are

intended to use the software or have interest in it [83]. A persona’s description holds

information of fictional people, they have names, occupation, life stories and goals [83].

The purpose of applying this techniques is to enhance engagement and reality by gaining a

clear understanding of the software and it is targeted users [33] [83]. In addition, it will also

include a description of the usage patterns that a persona would have when using the

software [83].

A complementary artefact is wire-framing with low-fidelity prototypes so that they both

can be combined as communication tools in UCD activities with subject-matter experts to

elicit feedback in the iterative cycles of design and development.

In this thesis, I have applied this technique before starting the development phase to gain a

wider understanding of the DSS system and potential users in the healthcare profession.

The intended system persona in the Medical Simulation DSS is shown in Figure 16 and

Figure 17 to demonstrate the type of information communicated between the developer(s)

of the system and the stakeholders. Notably, other personas used in the design cycles of the

system are included in Appendix C.

48

Figure 16 Persona 1 representing an experienced medical simulation user

Figure 17 Persona 2 representing a user in a learning context in healthcare (medical student)

4.3 System Visualization The UCD activities for designing the DSS involved iteration in the Software Development

Life Cycle SDLC. Two iterative cycles were conducted to design the DSS for medical

simulation tools and the interaction evolved to reflect the design considerations and

recommendations provide by subject matter experts in both the healthcare and the QA

domains.

4.3.1 First Cycle The first cycle is introduced in Figure 18. A brief explanation of the system was

given to the subject matter experts along with the system’s functionality. The

49

system visualization has been presented to the subject matter experts in order to get

their feedback and insights on the system.

Figure 18 System Visualization First Cycle

4.3.2 Second Cycle After receiving the feedback from the experts about the system visualization for the

first cycle. Comments and feedback were applied in the second cycle, and the

changes in the design of the interface are depicted in Figure 19.

50

Figure 19 Visualization -‐ Iteration 2

4.4 Chapter Summary In this chapter, the concept of the system was describe as well as the low-fidelity

prototyping process for integrating the Software Quality Model knowledge base with the

interface in the DSS. The interface of the DSS was designed for populating the database as

well as querying the knowledge base. In the following chapter, we describe specific

contexts of usage in the envisioned healthcare simulation scenarios to reflect on the design

of the DSS as well as a pre-cursor step for independent verification and validation that was

sought in collaboration with subject matter experts in the UCD activities of the iterations of

this system's SDLC.

51

Chapter Five: Interactive Decision Making

in Assessing the Quality of Medical

Simulation Tools

52

This chapter will showcase the procedural flow in the DSS by applying two case studies on

the medical simulation software quality model embedded in the DSS. We presented two

scenarios to showcase the interaction within the DSS. Professionals in healthcare, one

physician and one dentist, were involved in the feedback elicitation sessions for this

workphase, and medical simulation tools for the context of training were selected to be

used and evaluated in the DSS.

5.1 Scenario One: Evaluate a Medical Simulation Tool The user will access the DSS to evaluate Dental Navi tool and will be able to compare it to

other tools in the same category. The home page contains two buttons, “Evaluate a medical

simulation tool” and “Medical simulation tools review” as shown in Figure 20.

Figure 20 Wireframe design of Home Page's interface

In our case, the user will press the “Evaluate a medical simulation tool” to evaluate the

Dental Navi tool. After that he was directed to the software quality model embedded in the

DSS as shown in Figure 21 Software Quality Model Embedded In the DSS. The user will fill

in the fields by entering each value for a certain measure and these measures (QME) are

explained in chapter 3 (e.g. Time to complete the task). The software quality model will

calculate all the measurements and will be saved in knowledge base. Following that, the

user is directed to the result page, which contain all the entered and calculated values. The

user was also able to compare the Dental Navi tool with other tools in the dental category to

assist him in decision making by choosing the most applicable tool.

53

Figure 21 Software Quality Model Embedded In the DSS

5.2 Scenario Two: Medical Simulation Tools Review Based On

the Quality Assurance Factors The user is a Professor at King Saud University in the Medical school and is searching for

medical simulation tools to be used for training and education purposes. The user accesses

the DSS to search for previous evaluated medical simulation tools in the fields needed. The

54

home page contains two buttons, “Evaluate a medical simulation tool” and “Medical

simulation tools review”. The user pressed the “Medical simulation review” button and was

directed to the review page shown in Figure 22. The review page contains different medical

simulation tools which can be compared with each other by pressing the table button shown

in Figure 23.

Figure 22 Medical Simulation Tool Review

After the user pressed the table button, he was directed to the page shown in

Figure 22 Medical Simulation Tool Review. If the user wants to review a tool based on a

certain measure which is very helpful in decision making, he/she can press the

measure/metric that he/she wants as depicted in Figure 23. Following that, and a page

containing the measurement will be displayed for further interaction in the DSS.

Figure 23 Efficiency Measure

55

Figure 22 22 shows all the evaluation values for each entered tool to assist in decision-

making. In addition, the DSS provides the user a functionality to view tools based on a

certain measure such as freedom from the risk measure shown in Figure 25.

Figure 24 Freedom From Risk Measure

The system provides another feature by allowing the user to view the results in a chart as

shown in Figure 26. After calculating and entering all the values for each measure they are

presented in a chart based on the entered values from the user. Such presentation gives the

user the ability to visualize the effectiveness of the system. As an example, Figure 26

shows the distribution of quality assurance factors that are relevant for the simulation.

56

Notably, this example shows a tool that scored relatively high on efficiency measures but

less on subjective satisfaction measures from actual users of the simulation tool.

Figure 25 View Results as a Chart

The usability-engineering techniques for independently verifying the system's utility from a

user’s point of view have been established in the literature [e.g. 84]. Usability techniques

assist in structuring the process of designing and developing a good user interface [85].

Usability testing was applied with subject matter experts and practitioners to elicit their

insights on the high-fidelity design of the design. Qualitative and quantities approaches

were combined in these UCD activities. To augment the UCD activities applied in the early

research stages for requirements discovery and engineering, a cognitive walkthrough

usability technique was applied in the validation process to conduct the IV&V independent

verification and validation to ensure that system was developed right and that the right DSS

was developed from the perspective of professionals in the healthcare context [84]. IV&V

independent verification and validation is defined as a series of technical and management

activities performed by an expert in a field other than the system developer with the

objective to improve the quality of a system [86].

57

5.3 The Need for a Software Quality Model for Evaluating

Medical Simulation Tools The proposed software quality model assesses decision makers to evaluate medical

simulation tools by embedding it into the DSS. Modeling for decision-making development

is an effective method that assists in validating, testing and understanding complex

simulation tools. So, one of the main thrust of this research is to evaluate and improve these

simulation tools especially in early stages. Developing such a software quality model can

be used as a pre-process, since it provides health care providers the ability to evaluate the

tool before approving it to be used in the hospital, university or institute. Stakeholders will

be involved who would include problem owners in order to assess in the software quality

model development for the medical simulation tools.

5.4 Aim and scope of Software Quality Model Study The aim of this study is to investigate the proposed medical simulation software quality

model by describing the techniques, findings and evaluations results. An experimental

study was conducted to evaluate the medical simulation software quality model including a

variety of quality measures embedded in a web-based DSS. A selected number of

participants were selected to take part in this study. In addition, a selected number of

medical simulation tools were used in the study.

5.5 Exploratory Study In the following sections, we describe the context of the experimental study including the

participants, apparatus, and tasks assigned to the users.

5.5.1 Participants This study included seven participants considered as subject matter experts in the

medical simulation field. The participants were physicians who are varied in their

specialty and are using medical simulation tools. The participant’s technical

experience and demographic information is shown in Table 17 and Table 18.

58

Table 17 Participants' Technical Experience

Participants Technical experience and background

Medical simulation tools usage/familiarity

High Medium Low High Medium Low

P1 x

x

P2 x

x

P3 x x

P4 x

x

P5 x x

P6 x x

P7 x x

Table 18 Participants' Demographic Information

Participants Demographic information

Age Education level

Job title Organization

P1 32 Senior Registrar

Dermatologist National Guard Hospital

P2 22 Bachelor degree

Medical Student King Saud University

P3 65 Consultant Surgeon Prince Sultan Military Hospital


Dental student King Saud University


Dentist Riyadh College of Dentistry and Pharmacy

P6 26 Master’s degree

Researcher KACST

P7 25 Master’s degree

Software engineer Tamkeen technologies

59

5.5.2 Apparatus and Materials All the sessions were conducted at the hospital with the physicians. The apparatus

and materials used are a Macbook Pro device for running the DSS, and an iPad to

demonstrate the medical simulation tools.

5.5.3 Test Materials

• A brief presentation of the system was presented to the users to give an

overview of the system and its functionality.

• Participants' Consent Form (Appendix A).

• A timer, paper and pencil.

• Task scenario for participants.

• SUS questionnaire (Appendix B).

5.5.4 Task Scenarios A list of tasks was given to the participants to test the system’s usability. The results

of applying these tasks were used to assist us in the validation and verification

process. The participants are stakeholders in the healthcare sector with diversity in

specialties and positions. The list of tasks contains various functions that could be

performed in the system. A collection of medical simulation tools was presented to

the participants. The participants selected one tool to be evaluated by the system.

5.5.5 Task List A task list was provided to the user to be applied on the medical simulation DSS.

The list of tasks is shown in Table 19.

Table 19 Task List

Test Case Test Step Expected Result

Select medical simulation tool.

1. Choose one tool from a variety of tools to be evaluated.

The tool will open and used for simulation purposes.

Verify the home page. 1. Go to localhost\medicalsimulationtools

The DSS home page will contain two buttons,

1. Evaluation a medical simulation tool

2. Review medical simulation tool

60

Evaluate a medical simulation tool.

1. Go to localhost\medicalsimulationtools

2. Click the “Evaluate a medical simulation tool button”

The system should display a page containing the software quality model embedded in the system.

Timer 1. Before using the medical simulation tool to be evaluated by the system, a timer should be opened to determine the tool usage time.

The amount of time a participants used the medical simulation tool to perform the needed tasks.

Use the selected tool 1. Perform the needed actions to complete a task.

After viewing the DSS, the needed measurements (e.g. error frequency and time to complete a task) will be written down by the participants in order to be entered in the software quality model.

Enter the values needed for each measurement

1. Fill in all the fields with values.

After filling all the needed fields and submitting the evaluation form, the system will display a page containing the final results for decision-making and will have the ability to compare with other tools.

Review previous evaluated tool

1. Go to localhost\medicalsimulationtools

2. Click the “Medical simulation tools review”

The system should display a page containing the all previous submitted tools with the ability to view all their measurements. In addition the participant can also view tool based on a certain measure.

5.6 Objective Measures of Satisfaction In the study we have used the System Usability Scale (SUS). It is considered an

inexpensive and an effective method used for assessing the usability of a system [87]. The

SUS is 10 statements, each one of them have a 5 point scale ranges from Strongly Disagree

to Strongly Agree [88]. It provides a scale from 0 to 100, where 0 is considered as negative

61

and 100 is considered as positive [88]. ISO 9241-11 suggests that usability measures should

cover efficiency, effectiveness, and satisfaction [87].

The session started by presenting a brief explanation of the system to the participants along

the list of task that should be applied on the DSS, Participants Consent Form, and the SUS.

At the end of the session, the participants filled the SUS forms. The results of the SUS for

all the participants are shown in Figure 27. The chart in Figure 27 shows that the majority

of participants had relatively high SUS score, which suggests that the system’s usability is

perceived to be high as reported by participants in the evaluation study.

Figure 26 SUS Scores

As shown in Figure 27, P3 has the lowest SUS score in the sample. Notably, this

participants reported having low technical skills as shown in Figure 28. Although he had

low technical skills and could not easily use the system, he provided positive feedback on

the perceived value of using this system in the context of medical simulation.

0

10

20

30

40

50

60

70

80

90

100

0 1 2 3 4 5 6 7 8

SUS score

SUS

scor

e

Participants in the usability study

62

Figure 27 Relationship between Technical and Medical Simulation

The relation between the participant’s technical experience and their usage of medical

simulation tools is shown in

Figure 27. The participants were asked to evaluate their self out of the score 9 based on two

aspects, their technical background and medical simulation tool uses. If the score was from

1-3 it indicated a low value, if the score was from 4-6 it indicated a medium value, if the

score was from 7-9 it indicated a high value [32]. This categorization helped us to

understand the participants experience and how it relates in the interpretation of findings

from the usability study.

The chart in Figure 29 shows the variation between average scores of the subject matter

experts and healthcare professionals. The average score is a comparison between how the

0

1

2

3

4

5

6

7

8

9

10

P1 P2 P3 P4 P5 P6 P7

Technical experience and background

Medical simulation tools uses

Participants in the usability study

Parti

cipa

nt’s

eva

luat

ion

63

subject matter experts see the system and how the healthcare professional also perceive its

usability in the context of healthcare simulation. Notably, Figure 28suggests that the

experts and actual users are relatively close in their perception of usability according to the

SUS guidelines [87].

Figure 28 SUS Average Score for All Participants

5.7 Subject Matter Experts Reviews In this usability testing study, two domain experts were involved in cognitive walkthroughs

to evaluate the system. The Subject Matter Experts (SME) were domain experts where their

involvement was sought to help in eliciting opportunities for improvement of the system’s

functionality. Their insights and recommendation are summaried in Table 20.

Table 20 Subject Matter Experts Review

Subject matter expert Insights and recommendations

SME1 SME1 applied the tasks in the list provided.

Insights about the DSS are:

0 20 40 60 80 100

Subject matter experts

Healthcare professionals

SUS Average

SUS Average

Types of participants in the usability study

SUS

aver

age

64

• The system is easy to use and beneficial for decision-making. Recommendations: None.

SME2 SME2 applied the tasks in the list provided.

Insights about the DSS are;

• The system is straight to the point, easy to use, and covers the needed functionality.

Recommendations:

• Since the software quality model is very long and comprehensive to use, the SME suggested dividing the software quality model into separate measures where the user has the ability to evaluate a tool based on certain measures and skip other quality measures (setting a default or null value where applicable).

• To improve system visualizations and decision making the evaluation results should be also provided in charts.

5.8 Chapter Summary In this chapter, we described the cognitive walkthrough sessions, which were conducted

with SMEs and professionals and the insights obtained from them with regards to the

system's functionality, interface, and context of use. The verification and validation of the

DSS with actual users was essential to reflect on the design features (e.g. interface,

accessibility, and usability). Findings suggested an addition to the growing technology

assistance tools for medical simulation in the DSS, as participants perceived it as an

effective repository for examining quality metrics across medical simulation tools.

65

Chapter Six: Conclusion

66

In this thesis, we examined designing and developing a software quality model for

evaluating medical simulation tools.

The research objectives were achieved across work phases. In work phase 1, a systematic

literature review was conducted to gain an insight on medical simulation tools, challenges,

and best practices to overcome these challenges. In work phase 2, the software quality

model has been designed and presented. In work phase 3, the web-based DSS was designed

and developed along with the software quality model. In addition, the system was

independently verified and validated in cognitive walkthroughs with experts and use testing

scenarios with expert and practitioners.

Figure 29 Medical Simulation DSS

6.1 Research Questions The research questions were addressed in this thesis and reported in the document. For the

first research question of “Why do professionals need to evaluate medical simulation

tools?” was addressed in chapter 2, section 1-3 in which the importance of evaluation was

highlighted for medical simulation. For the second research question “What are the DSSs

used in medical domains?”, was addressed in chapter 2 section 2 in which the medical

67

simulation DSS types were highlighted. For the third research question “What are the

software development models used to design DSS in medical simulation domains?” was

addressed in chapter 2 section 3 in which software quality models were described. For the

fourth research question, “How can we test the usability, performance, robustness, and

accessibility of the medical simulation tools?” was addressed in chapter 3 section 2 in

which software quality model and measurement is presented. Furthermore, the independent

verification and validation sessions provided insights into testing the quality of medical

simulation tools which was described in chapters 3 and 4. For the fifth research question

“What are the widely accepted Software quality models used among Software Engineers?”

the software quality models widely used was addressed in chapter 2 section 3

6.2 Future Work In our future work, the medical simulation DSS will be amended with more software

quality metrics and interaction functionalities. These features will include adding more

visualization and analytics for the quality measurements in the software quality model

which is embedded in the DSS. Continuous integration of human expert’s knowledge will

also be considered as for the system by keeping track of usage logs in the DSS. In addition,

Artificial Intelligence algorithms will be applied to the system in order to develop further

analytics for the knowledge-base.

6.3 Publications The research conducted as part of this thesis was disseminated in scientific venues of

publication in the form of work-in-progress during the work phases and is planned for

reporting as a journal publication for the final work phase described in chapters 3 and 4.

Two conference publications were accepted for presentation in 2015; as noted in the

enclosed list.

• Al-Romi, Norah (July 2015). Human Factors in the Design of Medical Simulation

Tools, 6th International Conference on Applied Human Factors and Ergonomics

(AHFE 2015) and the Affiliated Conferences, AHFE 2015, USA.

68

• Al-Romi, Norah (Augest 2015). A Quality Model Used in Evaluating Medical

Simulation Tools, 17th International conference On Human Computer Interaction,

HCII 2015.

6.4 References [1] Y. Okuda, E. O. Bryson, S. Demaria, L. Jacobson, J. Quinones, B. Shen, And A. I. Levine, “The Utility Of Simulation In Medical Education: What Is The Evidence?,” Mount Sinai Journal Of Medicine, Vol. 76, No. 4. Pp. 330–343, 2009.

[2] M. I. T. Opencourseware, “Introduction To Modeling And Simulation,” In Proceedings Of The 2004 Winter Simulation Conference R .G. Ingalls, M. D. Rossetti, J. S. Smith, And B. A. Peters, Eds., 2008, Pp. 21–23.

[3] J. Barjis, “Healthcare Simulation And Its Potential Areas And Future Trends,” Scs M&S Mag. –, Vol. 1, No. January, Pp. 1–6, 2011.

[4] B. Mielczarek And J. Uzialko-Mydlikowska, “Application Of Computer Simulation Modeling In The Health Care Sector: A Survey,” Simulation, Vol. 88, No. 2, Pp. 197–216, 2010.

[5] J. Bosire, “Comparing Simulation Alternatives Based On Quality Expectations,” 2007, No. 1, Pp. 1579–1585.

[6] D. C. C. Peixoto, R. F. Resende, And C. I. P. S. Pádua, “An Educational Simulation Model Derived From Academic And Industrial Experiences,” Proc. - Front. Educ. Conf. Fie, Pp. 691–697, 2013.

[7] P. Naraharisetty And S. Vanka, “Effectiveness Of Computer Based Management Simulations - A Case Study,” In Proceedings - 2012 Ieee 4th International Conference On Technology For Education, T4e 2012, 2012, Pp. 31–37.

[8] L. C. Dukes, J. Bertrand, M. Gupta, R. Armstrong, T. Fasolino, S. Babu, And L. F. Hodges, “Poster: Comparing Usability Of A Single Versus Dual Interaction Metaphor In A Multitask Healthcare Simulation,” In Ieee Symposium On 3d User Interface 2013, 3dui 2013 - Proceedings, 2013, Pp. 133–134.

[9] D. M. Gaba, “The Future Vision Of Simulation In Healthcare.,” Simul. Healthc., Vol. 2, No. 2, Pp. 126–135, 2007.

[10] J. A Sokolowski And C. M. Banks, “A Proposed Approach To Modeling And Simulation Education For The Medical And Health Sciences,” In 2010 Summer Simulation Multiconference, 2010, Pp. 284–289.

[11] J. Liu, X. Wang, And M. Cheng E, “Simulation Modeling And Analysis On Asset Planning For Emergency Medical System (Ems),” In Ieem2010 - Ieee International Conference On Industrial Engineering And Engineering Management, 2010, Pp. 1353–1357.

[12] C. C. Chen, J. S. Daponte, And M. D. Fox, “Fractal Feature Analysis And Classification In Medical Imaging.,” Ieee Trans. Med. Imaging, Vol. 8, No. 2, Pp. 133–42, 1989.

[13] G. Mani And W. Li, “3d Web Based Surgical Training Through Comparative Analysis,” In Proceedings Of The 18th International Conference On 3d Web Technology - Web3d ’13, 2013, P. 83.

69

[14] T. R. Coles, D. Meglan, And N. W. John, “The Role Of Haptics In Medical Training Simulators: A Survey Of The State Of The Art,” Ieee Trans. Haptics, Vol. 4, No. 1, Pp. 51–66, 2011.

[15] E. Samur, L. Flaction, And H. Bleuler, “Design And Evaluation Of A Novel Haptic Interface For Endoscopic Simulation,” Ieee Trans. Haptics, Vol. 5, No. 4, Pp. 301–311, 2012.

[16] X. Wang, X. Shen, And X. Liu, “Improving Patient Flow At Hospital Emergency Services; A Simulation Study,” In Icsssm11, 2011, Pp. 1–6.

[17] I. W. Gibson And B. L. Lease, “An Approach To Hospital Planning And Design Using Discrete Event Simulation,” In 2007 Winter Simulation Conference, 2007, Pp. 1501–1509.

[18] S. C. Brailsford, “Tutorial: Advances And Challenges In Healthcare Simulation Modeling,” In Proceedings Of The 2007 Winter Simulation Conference, 2007, Pp. 1436–1448.

[19] M. Passiment, H. Sacks, And G. Huang, “Medical Simulation In Medical Education: Results Of An Aamc Survey,” Education, No. September, 2011.

[20] S. B. Issenberg, W. C. Mcgaghie, I. R. Hart, J. W. Mayer, J. M. Felner, E. R. Petrusa, R. A. Waugh, D. D. Brown, R. R. Safford, I. H. Gessner, D. L. Gordon, And G. A. Ewy, “Simulation Technology For Health Care Professional Skills Training And Assessment.,” Jama, Vol. 282, No. 9, Pp. 861–866, 1999.

[21] A. Nishisaki, R. Keren, And V. Nadkarni, “Does Simulation Improve Patient Safety?: Self-Efficacy, Competence, Operational Performance, And Patient Safety,” Anesthesiology Clinics, Vol. 25, No. 2. Pp. 225–236, 2007.

[22] P. N. Lowe And M. W. Chen, “System Of Systems Complexity: Modeling And Simulation Issues,” Simul. Interoperability Stand. Organ. - Siso Eur. Simul. Interoperability Work. Euro Siw 2008, No. 2, Pp. 299–308, 2008.

[23] R. Blaser, M. Schnabel, D. Mann, P. Jancke, K. Kuhn, And R. Lenz, “Potential Prevention Of Medical Errors In Casualty Surgery By Using Information Technology,” In Proceedings Of The Acm Symposium On Applied Computing, 2004, Pp. 285–290.

[24] T. Eldabi, “Implementation Issues Of Modeling Healthcare Problems: Misconceptions And Lessons,” Proc. - Winter Simul. Conf., Pp. 1831–1839, 2009.

[25] A. Kushniruk, E. Borycki, J. G. Anderson, And M. M. Anderson, “Combining Two Forms Of Simulation To Predict The Potential Impact Of Interface Design On Technology-Induced Error In Healthcare,” 2008, Pp. 497–504.

[26] S. C. Brailsford, T. Bolt, C. Connell, J. H. Klein, And B. Patel, “Stakeholder Engagement In Health Care Simulation,” In Proceedings - Winter Simulation Conference, 2009, Pp. 1840–1849.

[27] M. Saunders, P. Lewis, And A. Thornhill, Research Methods For Business Students. Financialtimesprentice-Hall, 2009.

[28] C. Abras, D. Maloney-Krichmar, And J. Preece, “User-Centered Design,” Bainbridge, W. Encycl. Human-Computer Interact. Thousand Oaks Sage Publ., Vol. 37, No. 4, Pp. 445–56, 2004.

[29] K. Vredenburg, J.-Y. Mao, P. W. Smith, And T. Carey, “A Survey Of User-Centered Design Practice,” In Proceedings Of The Sigchi Conference On Human Factors In Computing Systems Changing Our World, Changing Ourselves - Chi ’02, 2002, No. 1, P. 471.

[30] M. Zarour, A. Abran, J.-M. Desharnais, And A. Alarifi, “An Investigation Into The Best Practices For The Successful Design And Implementation Of Lightweight Software Process Assessment Methods: A Systematic Literature Review,” J. Syst. Softw., Vol. 101, Pp. 180–192, 2015.

70

[31] M. Maguire, “Methods To Support Human-Centred Design,” Int. J. Hum. Comput. Stud., Vol. 55, No. 4, Pp. 587–634, 2001.

[32] I. E. C. Jtc, “Systems And Software Engineering - Systems And Software Quality Requirements And Evaluation ( Square ) – Measurement Of Quality In Use,” 2012.

[33] J. Pruitt And J. Grundin, “Personas  : Practice And Theory,” In Proceedings Of The 2003 Conference On Designing For User Experiences, 2003, Pp. 1–15.

[34] D. Samadhiya, S.-H. Wang, And D. Chen, “Quality Models: Role And Value In Software Engineering,” In International Conference On Software Technology And Engineering, 2010, Pp. 320–324.

[35] C. D. Buckingham, A. Ahmed, And A. Adams, “Designing Multiple User Perspectives And Functionality For Clinical Decision Support Systems,” In 2013 Federated Conference On Computer Science And Information Systems U6 - Ctx_Ver=Z39.88-2004&Ctx_Enc=Info%3aofi%2fenc%3autf-8&Rfr_Id=Info:Sid/Summon.Serialssolutions.Com&Rft_Val_Fmt=Info:Ofi/Fmt:Kev:Mtx:Book&Rft.Genre=Proceeding&Rft.Title=2013+Federate, 2013, Pp. 211–218.

[36] E. S. Binstadt, R. M. Walls, B. A. White, E. S. Nadel, J. K. Takayesu, T. D. Barker, S. J. Nelson, And C. N. Pozner, “A Comprehensive Medical Simulation Education Curriculum For Emergency Medicine Residents,” Ann. Emerg. Med., Vol. 49, No. 4, 2007.

[37] K. R. Rosen, “The History Of Medical Simulation,” J. Crit. Care, Vol. 23, No. 2, Pp. 157–166, 2008.

[38] S. M. Sanchez, T. Ogazon, D. M. Ferrin, And T. J. Ward, “Emerging Issues In Healthcare Simulation,” In Proceedings - 2000 Winter Simulation Conference, 2000, Pp. 1999–2003.

[39] K. Anderson, “Modeling And Simulation Grand Challenges: An Or/Ms Perspective R. Pasupathy, S.-H. Kim, A. Tolk, R. Hill, And M. E. Kuhl, Eds,” In Proceedings Of The 2013 Winter Simulation Conference, 2013, No. Johns 2008, Pp. 3059–3065.

[40] K. Kunkler, “The Role Of Medical Simulation: An Overview.,” Int. J. Med. Robot., Vol. 2, No. 3, Pp. 203–210, 2006.

[41] J. A. Aucar, N. R. Groch, S. A. Troxel, And S. W. Eubanks, “A Review Of Surgical Simulation With Attention To Validation Methodology.,” Surg. Laparosc. Endosc. Percutan. Tech., Vol. 15, No. 2, Pp. 82–89, 2005.

[42] J. H. Magee, “Validation Of Medical Modeling & Simulation Training Devices And Systems,” In Studies In Health Technology And Informatics, 2003, Vol. 94, Pp. 196–198.

[43] L. Lin, R. Isla, K. Doniz, H. Harkness, K. J. Vicente, And D. J. Doyle, “Applying Human Factors To The Design Of Medical Equipment: Patient-Controlled Analgesia.,” J. Clin. Monit. Comput., Vol. 14, No. 4, Pp. 253–263, 1998.

[44] J. R. Callan And J. W. Gwynne, “Human Factors Principles For Medical Device Labeling,” 1993.

[45] “App Store Downloads On Itunes.” [Online]. Available: Https://Itunes.Apple.Com/En/Genre/Ios/Id36?Mt=8. [Accessed: 03-May-2015].

[46] A. Adla And P. Zarate, “A Cooperative Intelligent Decision Support System,” Service Systems And Service Management, 2006 International Conference On, Vol. 1. Pp. 763–769, 2006.

[47] F. Z. F. Zhou, B. Y. B. Yang, L. L. L. Li, And Z. C. Z. Chen, “Overview Of The New Types Of Intelligent Decision Support System,” In 2008 3rd International Conference On Innovative Computing Information And Control, 2008, Pp. 1–4.

71

[48] D. J. Power And R. Sharda, “Model-Driven Decision Support Systems: Concepts And Research Directions,” Decis. Support Syst., Vol. 43, No. 3, Pp. 1044–1061, 2007.

[49] G. Desanctis, R. B. Gallupe, M. Science, And N. May, “A Foundation For The Study Of Group Decision Support Systems,” 2007, Vol. 33, No. 5, Pp. 589–609.

[50] C. Schultewolter, “Towards A Framework For The Generic Specification Of Model-Driven Decision Support Systems  : Classification Criteria Of Model Relationships,” Pp. 1–10, 2010.

[51] D. J. Power, “A Brief History Of Decision Support Systems,” Decis. Support Syst., Vol. 4, No. 1969, Pp. 1–18, 2007.

[52] D. J. Power And N. Iowa, “Web-Based And Model-Driven Decision Support Systems  : Concepts And Issues,” In Americas Conference On Information Systems, 2000, Pp. 1–4.

[53] N. I. B. Adnan And Z. Tasir, “Online Social Learning Model,” In 2014 International Conference On Teaching And Learning In Computing And Engineering, 2014, Pp. 143–144.

[54] W. Cheung And C. Hsu, “The Model-Assisted Global Query System For Multiple Databases In Distributed Enterprises,” Acm Trans. Inf. Syst., Vol. 14, No. 4, Pp. 421–470, 1996.

[55] R. Stern And B. Sagot, “Population Of A Knowledge Base For News Metadata From Unstructured Text And Web Data,” In Proceedings Of The Joint Workshop On Automatic Knowledge Base Construction And Web-Scale Knowledge Extraction, 2012, Pp. 35–40.

[56] M. A Shwe, B. Middleton, D. E. Heckerman, M. Henrion, H. E.J., H. P. Lehman, And G. F. Cooper, “Probabilistic Diagnosis Using A Reformulation Of The Internist-1/ Qmr Knowledge Base,” In Methods Of Information In Medicine, 1991, Vol. 30, Pp. 241–255.

[57] S. U. N. Yu And X. U. Tianwei, “A Formal Model Of User Knowledge Base Systems In Intelligent Tutoring Systems,” 2010, No. 60903131.

[58] H. Wang, “Medcase  : A Template Medical Case Store For Case-Based Reasoning In Medical Decision Support,” 2013, Pp. 962–967.

[59] M. Siahbani, R. Vadlapudi, M. Whitney, And A. Sarkar, “Knowledge Base Population And Visualization Using An Ontology Based On Semantic Roles,” In Proceedings Of The 2013 Workshop On Automated Knowledge Base Construction - Akbc ’13, 2013, Pp. 85–90.

[60] K. S. Kadambi, J. Li, And A. H. Karp, “Near-Field Communication-Based Secure Mobile Payment Service,” In Proceedings Of The 11th International Conference On Electronic Commerce - Icec ’09, 2009, P. 142.

[61] T. X. Bui And M. Jarke, “Communications Design For Co-Op: A Group Decision Support System,” Acm Trans. Inf. Syst., Vol. 4, No. 2, Pp. 81–103, 1986.

[62] A. Steck, A. Lotz, And C. Schlegel, “Model-Driven Engineering And Run-Time Model-Usage In Service Robotics,” In Proceedings Of The 10th Acm International Conference On Generative Programming And Component Engineering - Gpce ’11, 2011, P. 73.

[63] H.-T. Wang And A. U. Tansel, “Medcase: A Template Medical Case Store For Case-Based Reasoning In Medical Decision Support,” In Proceedings Of The 2013 Ieee/Acm International Conference On Advances In Social Networks Analysis And Mining - Asonam ’13, 2013, Pp. 962–967.

[64] I. Fatima, M. Fahim, D. Guan, Y.-K. Lee, And S. Lee, “Socially Interactive Cdss For U-Life Care,” Proc. 5th Int. Confernece Ubiquitous Inf. Manag. Commun., Pp. 1–8, 2011.

72

[65] S. Subramanian, S. Hoover, B. Gilman, T. S. Field, R. Mutter, And J. H. Gurwitz, “Computerized Physician Order Entry With Clinical Decision Support In Long-Term Care Facilities: Costs And Benefits To Stakeholders.,” J. Am. Geriatr. Soc., Vol. 55, No. 9, Pp. 1451–7, Sep. 2007.

[66] J. A. Osheroff, Md, Facp, Facmi, And Editor-In-Chief, Improving Medication Use And Outcomes With Clinical Decision Support:: A Step By Step Guide. Himss, 2009.

[67] D. J. Desilets, S. Banerjee, B. A. Barth, V. Kaul, S. R. Kethu, M. C. Pedrosa, P. R. Pfau, J. L. Tokar, S. Varadarajulu, A. Wang, L. M. W. K. Song, And S. A. Rodriguez, “Endoscopic Simulators,” Gastrointest. Endosc., Vol. 73, No. 5, Pp. 861–867, 2011.

[68] B. W. O’malley, G. S. Weinstein, W. Snyder, And N. G. Hockstein, “Transoral Robotic Surgery (Tors) For Base Of Tongue Neoplasms.,” 2006.

[69] D. Jamwal, “Analysis Of Software Quality Models For Organizations,” Int. J. Latest Trends Comput. (E-Issn 2045-5364), Vol. 1, No. 2, Pp. 19–23, 2010.

[70] M. Kläs, C. Lampasona, And J. Münch, “Adapting Software Quality Models: Practical Challenges, Approach, And First Empirical Results,” In Proceedings - 37th Euromicro Conference On Software Engineering And Advanced Applications, Seaa 2011, 2011, Pp. 341–348.

[71] J. A. Mccall, P. K. Richards, And G. F. Walters, “Factors In Software Quality,” Nat’l Tech. Inf. Serv., Vol. 1, 2 And 3, No. November, 1977.

[72] B. Singh And S. P. Kannojia, “A Review On Software Quality Models,” Proc. - 2013 Int. Conf. Commun. Syst. Netw. Technol. Csnt 2013, Pp. 801–806, 2013.

[73] B. W. Boehm, J. R. Brown, And M. Lipow, “Quantitative Evaluation Of Software Quality,” Proc. 2nd Int. Conf. Softw. Eng., Pp. 592–605, 1976.

[74] A. B. Al-Badareen, M. H. Selamat, And M. A Jabar, “Software Quality Models  : A Comparative Study,” 2011, Pp. 46–55.

[75] R. G. Dromey, “Cornering The Chimera,” Ieee Softw., Vol. 13, No. 1, Pp. 33–43, 1996.

[76] R. G. Dromey, “A Model For Software Product Quality,” Ieee Trans. Softw. Eng., Vol. 21, No. 2, Pp. 146–162, 1995.

[77] P. Botella, X. Burgués, J.-P. Carvallo, X. Franch, G. Grau, J. Marco, And C. Quer, “Iso/Iec 9126 In Practice: What Do We Need To Know?,” In Software Measurement European Forum 2004, 2004, Pp. 297–306.

[78] H. Pedram, D. K. Moghaddam, And Z. Asheghi, “Applying The Iso 9126 Model To The Evaluation Of An E-Learning System In Iran,” Iran. J. Inf. Process. Manag., Vol. 27, No. 2, Pp. 495–517, 2012.

[79] D. St-Louis And W. Suryn, “Enhancing Iso/Iec 25021 Quality Measure Elements For Wider Application Within Iso 25000 Series,” In Iecon Proceedings (Industrial Electronics Conference), 2012, Pp. 3120–3125.

[80] Z. Cooper And C. Fairburn, “The Eating Disorder Examination: A Semi-Structured Interview For The Assessment Of The Specific Psychopathology Of Eating Disorders,” Int. J. Child Adolesc. Heal., Vol. 6, No. 1, Pp. 1–8, 1987.

[81] L. Z. Aspiazu, “De Madrid Development Of A Model For Security And Usability Master Thesis,” 2013.

[82] G. Miller And L. Williams, “Personas  : Moving Beyond Role-Based Requirements Engineering.”

73

[83] J. Grudin And J. Pruitt, “Personas , Participatory Design And Product Development  : An Infrastructure For Engagement,” Design, Vol. 2002, Pp. 144–161, 2002.

[84] R. Mack And J. Nielsen, “Usability Inspection Methods,” Acm Sigchi Bull., Vol. 25, No. 1, Pp. 28–33, 1993.

[85] D. Mayhew, “The Usability Engineering Lifecycle,” In Chi’99 Extended Abstracts On Human Factors In Computer Systems, 1999, Pp. 147–148.

[86] D. T. Brunner, J. D. Arthur, R. E. Nanee, And V. Tech, “Independent Verification And Validation: A Missing Link In Simulation Methodology?,” In Proceedings Of The 1996 Winter Simulation Conference, 1996.

[87] A. Bangor, T. Staff, P. Kortum, J. Miller, And T. Staff, “Determining What Individual Sus Scores Mean  : Adding An Adjective Rating Scale,” Vol. 4, No. 3, Pp. 114–123, 2009.

[88] J. Brooke, Sus - A Quick And Dirty Usability Scale. 1996.

74

Appendix A: Consent Form for

Participation in a Research Study

Consent Form for Participation in a Research Study

Prince Sultan University

Title of Study Usability testing for a medical simulation decision support system

Description of the research and your participation

You are invited to participate in a research study conducted by Norah AlRomi and Dr.

Areej AlWabil. The purpose of this research is to apply usability testing on a medical

simulation Decision Support System. Your participation will involve in applying a list of

tasks on the system.

Risks and discomforts

There are no known risks associated with this research.

Potential benefits

This research may help us to understand and evaluate the Decision Support System by

subject matter experts and stakeholders

Protection of confidentiality

We will do everything we can to protect your privacy. Your identity will not be revealed in

any publication resulting from this study.

Voluntary participation

Your participation in this research study is voluntary. You may choose not to participate

and you may withdraw your consent to participate at any time. You will not be penalized in

any way should you decide not to participate or to withdraw from this study.

75

Contact information

If you have any questions or concerns about this study or if any problems arise, please

contact Norah AlRomi at Prince Sultan University. If you have any questions or concerns

about your rights as a research participant, please contact the Prince Sultan University.

Consent

I have read this consent form and have been given the opportunity to ask questions. I

give my consent to participate in this study.

Participant’s signature_______________________________ Date:_________________

A copy of this consent form should be given to you.

76

Appendix B: System Usability Scale System Usability Scale

Strongly Strongly

disagree agree

1. I think that I would like to

use this system frequently

2. I found the system unnecessarily

complex

3. I thought the system was easy

to use

4. I think that I would need the

support of a technical person to

be able to use this system

5. I found the various functions in

this system were well integrated

6. I thought there was too much

inconsistency in this system

7. I would imagine that most people

would learn to use this system

very quickly

8. I found the system very

cumbersome to use

9. I felt very confident using the

system

10. I needed to learn a lot of

things before I could get going

with this system

77

Appendix C: Personas Personas

78

Appendix D: DSS Data Dictionary

Date post:	08-May-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

A Software Quality Model for Evaluating Medical...

Documents