“Hospital Information System Evaluation”

the UNIVERSITY of GREENWICH

Dissertation

Title:

“Hospital Information System Evaluation”

Student’s name:

Triantafyllia Doumpa

Supervisor’s name:

Dr. Prodromos Chatzoglou

2009

MSc in Finance and Financial Information Systems

Acknowledgements

I would like to acknowledge the assistance of Dr. Prodromos

Chatzoglou, Professor of the Democritus University of Thrace, Greece,

supervisor of the present dissertation, for his valuable contribution in

supervising my progress in every step, from the beginning until its

completion.

Furthermore, it would be ungrateful not to thank the directors of

the IS departments of the hospitals that participated in the present

research, for our great cooperation during the data collection process.

To conclude, I would like to thank my colleagues and friends, and

especially my family for their help and support that they have given to me

from the begging of the postgraduate programme.

-ii-

Table of Contents

ACKNOWLEDGEMENTS ....................................................................... ii

TABLE OF CONTENTS.........................................................................iii

LIST OF FIGURES ............................................................................. v

LIST OF TABLES ..............................................................................vi

ABSTRACT ....................................................................................vii

INTRODUCTION............................................................................. 1

CHAPTER 1 .................................................................................... 3

HOSPITAL INFORMATION SYSTEMS: THE CASE OF GREECE ........... 3

1.1. IT AND HEALTHCARE .................................................................. 3

1.2. DEFINITIONS, AIM AND STRUCTURE OF HIS ....................................... 6

1.3. THE VALUE OF HOSPITAL INFORMATION SYSTEMS ................................ 8

1.4. THE STATUS OF HIS IN GREEK HOSPITALS......................................... 9

CHAPTER 2 .................................................................................. 10

INFORMATION SYSTEMS EVALUATION ........................................ 10

2.1. THE IMPORTANCE OF EVALUATION ..................................................10

2.2. THE DELONE AND MCLEAN MODEL OF INFORMATION SYSTEM SUCCESS .....12

2.3. APPROACHES TO IS SUCCESS MEASUREMENT .....................................15

2.3.1. Cost-benefit analysis .......................................................16

2.3.2. System Usage.................................................................16

2.3.3. User Satisfaction .............................................................17

2.4. USER SATISFACTION MEASUREMENT MODELS .....................................17

2.4.1. Bailey and Pearson’s user satisfaction measure....................18

2.4.2. Ives, Olson and Baroudi user satisfaction measure...............19

2.4.3. Doll and Torkzadeh measurement of EUCS..........................23

-iii-

2.4.4. Chin and Lee model of EUCS.............................................26

2.4.5. Mahmood et al., research model of factors affecting IT EUS ..27

CHAPTER 3 .................................................................................. 30

HOSPITAL INFORMATION SYSTEM EVALUATION: THE RESEARCH30

3.1. USER SATISFACTION AS A MEASURE OF SUCCESS ................................30

3.2. THEORETICAL FOUNDATIONS ........................................................32

3.3. THEORETICAL MODEL AND HYPOTHESIS ............................................33

3.3.1. User background.............................................................34

3.3.2. System Quality ...............................................................35

3.3.3. Information Quality .........................................................36

3.3.4. Service quality................................................................37

3.4. METHODOLOGY RESEARCH...........................................................38

3.4.1. Instrument Development..................................................38

3.4.2. Research Population and Sample .......................................42

3.4.3. Research Procedures .......................................................42

3.4.4. Data analysis..................................................................45

CHAPTER 4 .................................................................................. 54

CONCLUSIONS AND RESEARCH LIMITATIONS ............................. 54

4.1. CONCLUSIONS.........................................................................54

4.2. RESEARCH LIMITATIONS .............................................................55

REFERENCES................................................................................ 57

APPENDIX A: QUESTIONNAIRE.................................................... 63

APPENDIX B: STATISTICAL ANALYSIS......................................... 66

-iv-

List of Figures

FIGURE 1.1: FUNCTIONAL DESIGN FOR THE INTEGRATED HIS ........................... 7

FIGURE 2.1: THE D&M IS SUCCESS MODEL ..............................................13

FIGURE 2.2: UPDATED D&M IS SUCCESS MODEL ........................................14

FIGURE 2.3: A MODEL FOR MEASURING END-USER COMPUTING SATISFACTION.....25

FIGURE 2.4: FORMATION OF END-USER COMPUTING SATISFACTION ..................27

FIGURE 2.5: RESEARCH MODEL OF FACTORS AFFECTING IT END-USER SATISFACTION

.............................................................................................28

FIGURE 3.1: RESEARCH MODEL OF MEASURING USER SATISFACTION ..................34

FIGURE 3.2: USER BACKGROUND ...........................................................50

FIGURE 3.3: SYSTEM QUALITY...............................................................50

FIGURE 3.4: INFORMATION QUALITY .......................................................50

FIGURE 3.5: SERVICE QUALITY..............................................................50

FIGURE 3.6: USER SATISFACTION ..........................................................51

-v-

List of Tables

TABLE 1.1: BENEFITS OF USING INFORMATION AND COMMUNICATION TECHNOLOGIES

IN THE HEALTH CARE SECTOR............................................................ 5

TABLE 2.1: EVALUATION OF ALTERNATIVE UIS MEASURES..............................19

TABLE 2.2: IVES, OLSON AND BAROUDI USER SATISFACTION MEASURE...............22

TABLE 3.1: DEFINITIONS AND SUPPORTED LITERATURE .................................41

TABLE 3.2: DEMOGRAPHIC CHARACTERISTICS OF RESPONDENTS .......................44

TABLE 3.3: FACTOR AND RELIABILITY ANALYSIS ..........................................46

TABLE 3.4: STATISTICS CONCERNING STRUCTURAL EQUATION MODELS ...............51

-vi-

ABSTRACT

Abstract

The purpose of this research study is to contribute to the

development of an instrument for the evaluation of Greek Hospital

Information Systems. Information technology has rapidly penetrated into

the healthcare sector with integrated information systems, and it has been

proved that it can lead to better decision support, organizational support

and even influence the patient outcome. This is why the evaluation of

HISs is a very critical issue. After an extended and systematic review of

the existing literature on the basis of published studies, user satisfaction

was chosen as the measurement of IS success. Furthermore, a research

was conducted in order to detect the factors that affect the IS user

satisfaction. This study was based on the DeLone and McLean’s (2003)

model of IS success and, also, on widely validated user satisfaction

models. System quality, information quality, service quality and user

background were selected in order to measure user satisfaction.

Moreover, fourteen factors were chosen with corresponding items that

according to the literature measure the above four factors. A hypothetical

research model was designed and based on this the questionnaire that

was used in the research was generated. The questionnaire was

distributed at 4 Greek hospitals; the General Hospital of Heraklion Crete

“Venizeleio-Pananeio”, the General University Hospital of Alexandroupoli,

the General Hospital of Thessaloniki “Papageorgiou” and, finally, the

General Hospital of Thessaloniki “Georgios Papanikolaou”, that agreed to

participate to the survey. 100 completed questionnaires were collected.

The statistical analysis included factor and reliability analysis in order to

estimate the adequacy of the measurement model and, after that

Structural Equation Modeling was performed to test the structural models

fit. The conclusion is that user background, information quality and service

quality directly and positively affect user satisfaction confirming the initial

three hypotheses (H1, H3, and H4). The diversity concerned H2 which

have shown that system quality influence user satisfaction only indirectly

through information quality and not directly as it was initially stated.

Keywords: user satisfaction, hospital information systems, evaluation

-vii-

INTRODUCTION

Introduction

Information technology (IT) has already become one of the most

significant elements that lead to success. Nowadays, IT has entered into

every kind of organizations by making tremendous progress; transition

from personal computing to integrated information systems (IS).

Healthcare sector is one, among many other that have adopted

information systems. Hospital communities have been using information

systems since 1960, where they began with their simplest form, mainly as

financial systems (Zviran M., 1990). Nowadays, Hospital Information

Systems (HIS) have changed dramatically since their first entrance into

the healthcare sector and the continuously development of information

technology. HISs turned out to be multi-functional systems that combine

different subsystems, such as accounting and financial systems, patient

registration applications, medical records etc (Zviran M., 1990). At the

same time, the number of the organizational personnel who directly

interact with computers have also recorded an explosive growth. These

people use information systems in their every day job to accomplish

multiple applications, such as word processing, spreadsheets, statistics,

databases, etc (Harrison, A.W. and Rainer, R.K., 1996). The expanded

range of users and the importance of IS in the healthcare organizations

constitute the evaluation of these systems an extremely important issue,

in order to test the effectiveness of an IS. According to DeLone and

McLean (1992), the most widely known measure of information system

evaluation is user satisfaction. The literature suggests that user

satisfaction can play a vital role and can affect users' behavior toward

computer use, which in turn can also affect the actual usage of the system

(Harrison, A.W. and Rainer, R.K., 1996). Through evaluation the whole

healthcare system can be improved.

The aim of this study is to evaluate Greek hospital information

systems and their success. In order to accomplish this, the present

research focused on the development of an instrument, through which the

evaluation process could take place. More analytically, the factors that

-1-

INTRODUCTION

influence, positively or negatively, users' satisfaction have been detected

based on the literature. The selection of these factors, which the research

model of measuring user satisfaction was based on, was a very critical

issue of this study. Furthermore, the questionnaire that was used for the

purpose of this study was developed according to the measurement

model. The use of questionnaires allows the researcher to survey different

participants and permits respondents to have sufficient time to think

about the questions. The participants in the present survey were the

actual users of 4 Greek hospitals; General Hospital of Heraklion Crete

“Venizeleio-Pananeio”, General Hospital of Thessaloniki “Georgios

Papanikolaou”, General Hospital of Thessaloniki “Papageorgiou”, and

General University Hospital of Alexandroupoli. The data that were

gathered from the selection of the questionnaires were used in order to

run all the appropriate statistical analysis that is going to be discussed in

latter section.

The structure of the present survey consists of 4 Chapters. In

Chapter 1, Hospital Information Systems and their value in healthcare

sector are analysed. Additionally, the penetration and the progress of

information technology in Greek hospitals are also stated. Chapter 2

contains all the relatively theory about information system evaluation.

One of the most known models of information system success is

introduced; the DeLone and McLean model. Furthermore, measurement

models of user satisfaction are also represented. Chapter 3 contains the

research that was conducted for the present survey. The selection of user

satisfaction as the surrogate measure and, the relative theory that the

theoretical model and the research hypothesis were based on are

analysed. Moreover, the research methodology, such as the population

sample approach, the research procedures and the development of the

instrument are also thoroughly analysed. Finally, the data analysis and the

results are demonstrated. Last, Chapter 4 is mainly composed of the

limitations that came up during the research and the most important

conclusions that resulted from the analysis.

-2-

CHAPTER 1 HOSPITAL INFORMATION SYSTEMS: THE CASE OF GREECE

Chapter 1 Hospital Information Systems: the case of Greece

Hospital information systems (HIS) have changed spectacularly

since the introduction of computers in health care organizations, and

these changes were inevitable because of the information technology (IT)

changes, but also because of the realization of the benefits of

computerized hospitals.

The idea of developing a hospital information system began in the

early 1960s and its development mainly concerned of financial and partial

of patient care systems which have failed as being discontinued (Zviran

M., 1990). These systems were designed to provide a money-oriented

return on investment and streamline patient admissions. Another attempt

took place in the mid 1970s, where systems still didn’t focus to patient

care but only on financial applications. In the late 1970s, finally, software

packages expanded, as far as the patient care was concerned, being more

flexible and economic in relation to previous periods (Zviran M., 1990).

However, after years, many hospitals also adopted specialized systems in

other areas, such as laboratory, pharmacy and medical records, with more

emphasis to patient interest, leading to today’s latest form of the hospital

information systems which have become a necessity for a health care unit

to be competitive and to be able to deal with the demands of our days.

When choosing an information system (IS), hospitals must avoid making

mistakes and must consider their actual needs when purchasing a

healthcare software product (Thompson A.M., 1990; Zviran M., 1990).

1.1. IT and Healthcare

Taking into account the rapid development of technology, it was a

matter of time for the information technology to enter in every field of our

daily life and, consequently, in every public and private organization.

Nowadays, it is outstanding to imagine health care sector without

-3-


information technology (IT). The remarkable increase of the use of IT in

health care sector and, especially, in hospitals makes it clear that it is of

vital matter to leave the past behind. Paper-based records and written

documentation are replaced with computer based records and, generally,

with information systems that provide patient safety and improve the

quality and efficiency of health care systems. Different innovations, such

as electronic patient records and especially hospital information systems,

have significantly changed the workflow in healthcare (Aggelidis V.P. and

Chatzoglou P.D., 2008). The benefits that are generated by the use of

information and communication technologies in the health care sector are

categorised and demonstrated in Table (1.1).

Consequently, there is no doubt that paper-based records and

written documentation can no longer follow the needs of the modern

health care and their use reaches its limits as different technological

achievements, such as hospital information systems (HIS), take control

(Uslu, A.M. and Strausberg, J., 2008). It has been said that “given the

fragmented nature of health care, the large volume of transactions in the

system, the need to integrate new scientific evidence into practice, and

other complex information management activities, the limitations of

paper-based information management are intuitively apparent”

(Chaundry, B. et al., 2006; as found in Uslu, A.M. and Strausberg, J.,

2008, pp. 675).

-4-


Health Care Players

Benefits

Administration units

• Policy development and decision-making are strongly supported by effective and on-time information gathering and distribution.

• Easier adaptation to eEurope challenges. • Supply control; better budget monitoring. • Overall improvement in the way citizens are served.

Hospitals

• Increased efficiency in communication between hospitals, administration units, social security services, careers, physicians, and citizens.

• Personnel familiarization with information technologies through Internet-access operations.

• Patient-record traffic support. • Reinforcement of the need to build health care

information systems (HCISs) and local networks in hospitals.

• Utilization of the developed Intranets. • Better information services for the citizens. • Advanced telematic services (eg, telemedicine

applications in difficult-to-reach regions).

Health care personnel

• Meets the increased need for telecommunications not only for medical, but also for compensation reasons.

• Participation in care chains and relevant coordination. • Physicians' collaboration. • Patients'-history data retrieval. • Continuing education services; familiarization with new

technologies through special training programs. • Interaction with patients to provide advice or

prescriptions.

Citizens

• Use of the Internet for health-related information retrieval.

• Information and communication technologies will increase interest in citizens' health-issues management.

• Creation of the appropriate infrastructure for future provision of special health services for specific population groups (eg, in-house services for older people or patients with long-lasting attendance and nursing needs).

Table 1.1: Benefits of using information and communication technologies in the health care sector (Source: Lampsas, P. et al., 2002, pp. 3)

-5-


1.2. Definitions, aim and structure of HIS

Hospital information systems play a vital role in the quality of care

that is provided in the health care units and in the organization’s daily

operations. In order to understand what an information system is, it is

useful to start with explaining what constitutes a system and information.

First of all, a system consists of inputs, processing and outputs and also

combines different variables that are interrelated, organized and depends

on each other. Secondly, information can be defined as the outcome of

data processing which can be used to aid in decision making (Yusof, M.M.

et al., 2008). Based on the above, and according to Yusof et al. (2008, pp.

378), an information system is defined as “a group of interrelated

processes implemented to aid in enhancing the efficiency and

effectiveness of an organization in performing its functions and attaining

its objectives”. Another definition of an HIS, according to Aggelidis and

Chatzoglou (2008, pp. 101), is: “a hospital information system is a

computer based system designed to facilitate the management of the

administrative and medical information within a hospital”. Furthermore, an

HIS is “an information system used within a healthcare organization to

facilitate communication, perform record-keeping, or otherwise support

the functions of the organization” (Shortliffe, E.H. et al.; as found in Nahm

E.S. et al., 2007, pp. 283). To complete with, according to Ammenwerth

et al. (2007, pp. 216), a HIS can be defined as “the complete information

processing and information storing subsystem of a hospital, including both

computer-based and paper-based information processing tools”.

Due to the complex environment that hospitals function in, the use

of HIS is of great importance. The main aim of HIS is to improve the

quality of care that is provided. This can be achieved by capturing, storing

and retrieving accurate and timely data, by reporting those data in an

effective way and allowing transferability of them to other applications

within the hospital environment (Zviran, M., 1990). According to Zviran

(1990), the most important advantages of HIS are:

a) the capacity of combining data sources in an integrated database

-6-


b) the unique power to shift data among applications

c) the capacity to provide health personnel with a variety of data as

far as the patient treatment is concerned, and

d) the flexibility that is provided to users to move among applications

of their choice.

As mentioned above, hospitals are complex environments that use

integrated information systems. So, it is useful to understand the

structure of HIS and the four major functional groups with the additional

subsystems, that constitute such a system. Figure (1.1) portrays the

functional design of a Hospital Information System.

Figure 1.1: Functional design for the Integrated HIS (Source: Zviran, M., 1990, pp 34-37)

-7-


1.3. The Value of Hospital Information Systems

To begin with, it can be said that nowadays no one can imagine

health care sector without the use of information technology. Information

technology is constantly growing, and more and more hospitals embed

innovations which are always improved, due to requirements of modern

society. Someone could appraise the value of HISs by recording the

multiple benefits that are generated by their use and comparing them with

the old and historical paper based records.

Generally, integrated HISs are estimated as the mean through

which patient’s safety and the quality of health care systems are always

improved. In relation to paper based records, it is for sure that they

prevail because they provide to clinicians real-time decision support, with

a full patient profile available, leading them to the correct decisions (Jha,

A.K. et al., 2008). With the use of information technology, clinicians

achieve the reduction of unnecessary testing to patients and avoid wrong

prescriptions that would lead to undesirable results. In other words, the

use of an integrated information system allows the creation of a well

organised historical patient profile, with information about allergies,

treatment results, and medication given which lead to a better patient

care (Bakker, A.R., 2007). Another important factor is that the use of

HISs positively affect personnel’s productivity and diagnostic quality due

to well designed and well informed databases that function within the

system. It has also been proved that the use of information technology

can improve revenues. According to some researchers, “one clinic realised

a 12% increase in their revenues the 12-month period post-

implementation compared to pre-implementation” (Clayton, P.D. et al.; as

found in Handel, D.A. and Hackman, J.L., 2008, pp. 2). Another institution

realised “a savings of $1.67 for every dollar invested in their information

system” (Souther, E., 2001; as found in Handel, D.A. and Hackman, J.L.,

2008, pp. 2).

At this point, and having mentioned all the above, it must be stated

that a HIS is nothing more than a tool that it is useless without human

-8-


interference. The information produced by the system is always related to

the available data of the system and to the software quality, but at the

end all kinds of decision and different solutions to problems always

depend on humans and their capabilities.

1.4. The status of HIS in Greek hospitals

National governments have treated healthcare as a matter of great

importance. Technological innovations such as HISs are the mean through

which the service quality that is provided by the health care units can be

improved. Well organized efforts began at the end of 1980s in Greece with

the first Community Support Framework (1st CSF) that aimed to introduce

information technology in health care sector, and continued with the 2nd

CSF, in the mid-1990s. Finally, in 2000, and through the 3rd CSF, a great

attempt took place with the purpose of creating information-oriented

organizations by introducing integrated information systems into public

Greek hospitals (Aggelidis, V.P. and Chatzoglou, P.D., 2008).

According to a survey of the Information Society of Greece that was

conducted in 2008, although the penetration of information technology in

health care sector was satisfied to a great level, their use was surprisingly

very low. Furthermore, the outcomes of another survey have also shown

that despite the high penetration of IT in the administrative sector of the

hospitals, the medical and the nursing sector faces a low level of

computerization (Vagelatos, A. et al., 2002).

-9-

CHAPTER 2 INFORMATION SYSTEMS EVALUATION

Chapter 2 Information Systems Evaluation

In this part, issues related to Hospital Information System

evaluation and the relative theories are going to be analysed. At the

beginning, basic terms such as the real meaning of evaluation, the

reasons why it is so important to evaluate and the time when this takes

place will be discussed. Furthermore, the most important and common

used methods and theories are going also to be discussed.

2.1. The importance of evaluation

Nowadays, it is generally accepted that the use of information

technology has become a necessity for the health care sector. As it have

been mentioned in previous sections, many advantages derive from its

use. The most important are the reduction of errors which are caused by

humans, well informed patient records, and the improved quality of the

health care system (Jha, A.K., et al., 2008; Hayrinen, K., et al., 2008).

On the other hand, there are some disadvantages that can also

derive from the use of information technology. First of all, integrated

information systems are very expensive to implement (Kluge, E.H.W.,

2007). Additionally, the loss of productivity and efficiency by the staff is

another disadvantage that derives from the fact that at the beginning the

system is time-consuming, which can also lead to patient neglect

(Edwards, P.J., et al., 2008).

To sum up, and with all the above stated, someone can realise that

the evaluation of Hospital Information Systems is a very important issue

and the procedure must be as accurate as possible. Through evaluation,

disadvantages are reduced and improvements are achieved;

improvements that will remake the system itself more friendly to the user,

and help them realise the importance of its usage, that consequently may

contribute to an upgraded structure of a health care unit (Burkle, T., et

-10-


al., 2001). It is of vital importance that during the evaluation process both

the technology that is used and the role of the users that participate and

their relation to the technology must be taken into account (Ammenwerth,

E., et al., 2003).

The term “evaluation” has been given by different researchers.

According to Aggelidis, V.P. and Chatzoglou, P.D. (2008, pp. 100),: “an

evaluation research is the systematic collection and analysis of

information in order to support decision-making concerning projects,

processes or methods”. Based on Aggelidis and Chatzoglou (2008), the

evaluation process is initially composed of the explanation of the criteria

the researcher sets for the evaluation, then the collection of the

appropriate information as far as the evaluated object is concerned and,

finally, the determination of the value of the results. Another definition is

the following: “evaluation can be defined as the decisive assessment of

defined objects, based on a set of criteria to solve a given problem”

(Heinrich, L., 1999; as found in Ammenwerth, E., et al., 2003, pp. 126).

Evaluation studies can be distinguished in two categories;

summative and formative. Summative evaluation tries to illustrate and

examine the result of an information technology in clinical routine, while

formative evaluation tries to positively improve the information technology

used by providing the developers with useful comments (Ammenwerth, E.,

et al., 2003). Furthermore, the evaluation of Hospital Information

Systems can take place at different phases and by different people at each

phase (Nahm, E.S., et al., 2007):

- During the development of the system, where the evaluation is

initially conducted by the vendor.

- During the implementation of the system, where the system is

evaluated by the organization.

- After the implementation of the system, where the evaluation is

conducted by various, internal and external, users.

Indeed, evaluation starts during the development of the system and

it can be categorised into verification, validation, assessment of human

factor and clinical assessment of clinical effect (Burkle, T., et al., 2001):

-11-


Verification is taking place during the development of the system

and it examines whether the system is being structured according to its

primary specification requirements and verifies the level of

appropriateness, completeness, and consistency of the system.

Validation is the process that takes place later and confirms if the

system functions according to its initial design in real working conditions.

Assessment of human factor is the next process that takes place,

and tries to check whether the system is acceptable by the users. Even if

a system has passed the phases of verification and validation, its design

might not be easy in use, and thus difficult to handle in real time. The

usefulness and the usability of the system are checked.

Clinical assessment of clinical effect is the last phase, and from this

answers derive; answers that concern the patient outcome.

To conclude, different approaches have been developed concerning

information systems evaluation and each of them have both positive

characteristics and flaws as well. According to Ammenwerth, E., et al.

(2003), there is no standard method of how to evaluate an integrated

information system. Bokhari, R.H. (2005, pp. 211) stated that “the

evaluation of a system in terms of its success is an inherently complex

phenomenon”. In the next section, one of the most well known model of

information systems success and three basic measures of success are

analysed.

2.2. The DeLone and McLean Model of Information System Success

In 1992, DeLone and McLean published a paper in which they have

tried to present some subsistence and structure to the depended variable

of an IS success. Thus, they proposed a taxonomy and an interactional

model, known as the D&M Model, as a basic structure and instrument for

understanding the concept and the functionality of IS success. The D&M

Model targeted to combine previous research about IS success in order to

offer a more precise guidance for further analysis (DeLone, W.H., and

McLean, E.R., 2003). As mentioned above, based on previous theories, a

-12-


multidimensional model of IS success was postulated. According to

DeLone and McLean (1992), the six dimensions of IS success are the

following:

• “system quality”, which measures technical success

• “information quality”, which measures semantic success, and

• “use”, “user satisfaction”, “individual impacts”, and “organizational

impacts”, which measure effectiveness success.

It is very important to be stated that although these dimensions

come from very old theories, they are still very powerful and they are still

used in our days. It must become clear that these dimensions are

mutually connected and not independent (Pitt, L.F., et al., 1995).

According to the theory described in DeLone and McLean (2003), a model

proposes that an IS is firstly produced enclosing all the characteristics that

demonstrate the level of the system quality and information quality. In

turn, all end-users that consume their daily time with these characteristics

can either be satisfied or dissatisfied from the information that is provided

to them or by the system itself. The use of the system and the

information generated by the system, subsequently influence each user

individually as far as their every day job is concerned, and at the end

positive or negative organizational impact arise. The resultant D&M IS

Success Model is demonstrated in Figure (2.1).

Figure 2.1: The D&M IS Success Model

(Source: DeLone and McLean, 2003, pp. 12)

In few words, the best system quality is expected to lead to greater

user satisfaction and more use of the system, which would entail positive

-13-


effects on individual productivity and consequently to improved

productivity of the organization. The purpose of combining success

taxonomy with the model of IS success was to assist the understanding of

the possible causal correlations of the dimensions of success and to

provide a more in-depth presentation of these correlations. Finally, it must

be said that the D&M IS Success Model is one of the most well known,

which is proved by the fact that during the period 1993 to 2002,

approximately 285 surveys used the model as a basis to their theory, a

number that with no doubt reveals the level of its success (DeLone, W.H.,

and McLean, E.R., 2003).

Relying on different researches and the changes in the

management of information systems, followed in the period after the

presentation of their first version of the model, in 2003 McLean and

DeLone presented a revised and Updated Model of IS Success, and this is

stated below in Figure (2.2):

Information Quality

SystemQuality

Service Quality

Net Benefits

UserSatisfaction

IntentionTo Use Use

Figure 2.2: Updated D&M IS Success Model

(Source: DeLone and McLean, 2003, pp. 24)

According to the D&M updated model, in the already known

dimensions: “information quality” and “system quality”, has been added

one more; “service quality”. Pitt et al., (1995) have stated that “the most

common used measures of IS success focus on the products and not so

-14-


much on the services that the IS provides. If the measure of service

quality is excluded from the survey, there is a major risk for the

researchers to assess the IS success incorrect”. Another change in the

updated model is the existence of another measure; “intention to use”.

The difference between “intention to use” and “use” is that the first is an

attitude and the second is a behavior. Furthermore, the measure of “use”

and “user satisfaction” still exist and remain closely correlated. “Use”

precedes “user satisfaction”, but positive “use” will lead to greater “user

satisfaction”. In turn, this will lead to greater “intention to use” and,

consequently, to greater “use”. As a result of all these interrelated

measures, specific “net benefits” will occur (DeLone, W.H., and McLean,

E.R., 2003).

2.3. Approaches to IS success measurement

A large number of possible measurements of IS success exists and

this is because an information system can be viewed from many different

angles. Specifically, there are two perspectives: the organizational

viewpoint and the socio-technical viewpoint (Au, N., et al., 2002). From

the organizational point of view, emphasis is given to the information that

is provided from the IS and the way this interface with the user. This

perspective has been criticized claiming that ignores the human element.

On the other hand, the socio-technical viewpoint focuses on individual

needs. Furthermore, as it has been mentioned above, six dimensions of IS

success have been identified by DeLoan and McLean, (1992): system

quality, information quality, information use, individual impact, and

organizational impact. All of these six dimensions characterise the IS both

from the organizational viewpoint and the socio-technical viewpoint.

DeLoan and McLean (2003) later added one more dimension; service

quality.

To continue with, in latest years the discussion has primarily

focused around three different measures of success:

1. Cost-benefit analysis

-15-


2. System usage

3. User satisfaction

In the next sections these approaches are thoroughly analised.

2.3.1. Cost-benefit analysis

In a cost-benefit analysis procedure, the actual value of the IS, as

far as the organization is concerned, can be described as the difference

between the benefits in terms of the organizational effectiveness. The

cost-benefit analysis has been subjected under great criticism because it

is difficult to prove that benefits are connected to the information system

and because the costs and benefits are difficult to be measured in terms

of monetary value (Au N. et al., 2002).

2.3.2. System Usage

Another measure of IS success is system usage. This measure

shows in which level users trust the effectiveness of the system. Systems

usage is much easier to put into operation and it can be defined as “either

the amount of effort expended interacting with an information system or,

less frequently, as the numbers of reports or other information products

generated by the information system per unit time” (Trice A.W. et al.,

1988; as found in Bokhari R.H., 2005, pp. 213). There are different ways

of measuring the IS success, and these are (Au N. et al., 2002):

• The actual time that the users are linked with the system

• The amount of patient or client records that have been registered

• The amount of computer functions that have been used

Criticism is also applied to this method because it is relevant only

when it is voluntary (Ives, B., et al., 1983).

-16-


2.3.3. User Satisfaction

The other measure of IS success that is considered to be the most

widely known among the researchers is user satisfaction. According to Au

N. et al. (2002, pp. 453), user satisfaction is defined as “the IS end-user’s

overall affective and cognitive evaluation of the pleasurable level of

consumption-related fulfillment experienced with the IS. IS end-users

refer to non-technical personnel who use or interact with the system

directly”. User satisfaction is a multiple validated measure by many

researchers unlike the other two measures mentioned above (Bailey, J.E.,

and Pearson, S.W., 1983; DeLone, W.H., and McLean, E.R., 2003; Chin,

W.W., and Lee, M.K.O., 2000). Information system users evaluate the

quality of the system in their daily life. If users are not satisfied with the

quality and the functions provided by the system, the quality of the

information generated and, generally, by the services of the system, they

will not use it, or they will not use it correctly (Ribiere, V., et al., 1999).

More specific, in hospitals, which are highly sensitive environments, if the

personnel is not satisfied with the information system it is very likely to

reject it. Therefore, for a hospital information system to be successful, it

must not be very complicated in use, it must be adapted based on user’s

needs, it must be friendly in use and meet user expectations (Ribiere, V.,

et al., 1999).

2.4. User satisfaction measurement models

According to the literature, the most important user satisfaction

measurement models, which are used for the evaluation of Hospital

Information Systems, are the following:

• Bailey and Pearson, 1983

• Ives et al., 1983

• Doll and Torkzadeh, 1988

• Chin and Lee, 2000

• Mahmood et al., 2000

-17-


2.4.1. Bailey and Pearson’s user satisfaction measure

The Bailey and Pearson (1983) model of computer user satisfaction

is the first essential model, which is referred as the basis for the

development of further research on the measurement of end-user

satisfaction, with the ultimate aim of creating instruments for measuring

the IS success. Although the specific model is quite old, still remains one

of the most widely known and used. Bailey and Pearson (1983) created

the model based on the definition of user satisfaction in relation to a given

situation. They defined satisfaction as “the sum of the user’s weighed

reactions to a set of factors”,

=

= ∑1

n

i ij ijj

S R W

where

Rij = the reaction to factor j by individual i

Wij = the importance of factor j to individual i

This equation calculates the satisfaction as the sum of a user’s

positive or negative reaction in relation to each factor, and on the

significance that is defined by the user for a specific factor.

After reviewing 22 studies of the computer/user interface, Bailey

and Pearson identified and suggested 36 factors that affect user

satisfaction. Having the list completed, these authors proceeded to

different tests in order to examine the completeness and the accuracy of

the list. Two more factors were added after recommendations by some

professionals. This new and expanded list was then distributed to 32

middle manager users in 8 different organizations to comment on the

importance of each factor. All factors turned out to be more or less

important for each interviewer, based on their jobs. Through the analysis,

one factor was brought up four times from the interviewers and because

of the frequency that has been mentioned, that factor was added in the

list, which was then constituted from 39 factors. The questionnaire, which

-18-


included all the 39 factors was analytically tested and revealed that it is a

reliable instrument to use for measuring user satisfaction (Bailey, J.E.,

and Pearson, S.W., 1983).

2.4.2. Ives, Olson and Baroudi user satisfaction measure

Ives et al. (1983) had introduced their own instrument of user

satisfaction at the same year with Bailey and Pearson’s model (1983). At

the beginning Ives et al. reviewed and analysed all the existing models of

user satisfaction measurement in order to develop their own instrument.

For this reason the following measures of Gallapher (1974), Jenkins and

Richetts (1979), Lacker and Lessig (1980), and Bailey and Pearson (1983)

were studied. Through extended analysis for each model and after

locating their dominant characteristics, as well as their advantages and

disadvantages, a classification of those 4 measures was generated in

relation to (Ives, B., et al., 1983):

• Derivation (empirical or otherwise)

• Amount of empirical support

• Level of coverage (product, system services)

• Number of indicators in the measure

Table (2.1) encompasses a summary of the four measures reviewed:

Measure Derived From

Empirical Support

Level of Coverage

Number of Indicators

Gallapher

Empirical

Adequate

Product

18

Jenkins and Ricketts

Literature and interviews

Inadequate

Product

5

Larcker and Lessig

Interviews

Adequate

Product

2

Bailey and Pearson

Literature, interviews,

and empirical

Adequate

Product and

support

39

Table 2.1: Evaluation of Alternative UIS Measures (Source: Ives, B., et al., 1983, pp. 787)

-19-


In order to decide which one of the four models mentioned above

would be used for further research, Ives et al. (1983) assumed that the

chosen model should embrace the experience with sufficient support,

which should cover satisfaction in relation to information system product

and the support services offered, and generally should provide multiple

indicators. Based on the above criteria, the Bailey and Pearson measure

was selected. The aim of the study was to reproduce and duplicate Bailey

and Pearson’s findings as far as the validity of the instrument is

concerned, to proceed in further testing of the validity, to lessen the

overall measure in a way of maintaining its credibility, and to produce a

short form of the instrument for further research (Ives, B., et al., 1983).

The research that was conducted referred to 800 production

managers of manufacturing organizations in U.S. Two questionnaires were

sent to them in different timing (Ives, B., et al., 1983). The first was the

Bailey and Pearson information system satisfaction measure, while the

second was a 4-item measure for overall satisfaction. All the appropriate

tests took place, such as a) reliability of the measure, b) content validity

of the questionnaire, c) predictive validity, and d) construct validity of the

measure. A sort form of the instrument was generated according to two

phases; the first contained the elimination of the factors that showed

undesirable psychometric qualities, and the second contained the

elimination of some items within factors in order to reduce the completion

time of the questionnaire. The elimination of the appropriate factors was

depended upon an extended analysis and judgment. All scales were

classified according to reliability, content validity, and construct validity.

Continuously, 6 factors of the initial 39 were chosen for elimination, and

those were (Ives, B., et al., 1983):

1. Competition with EDP unit

2. Chargeback method

3. Vendor support

4. Computer language used

5. Security of data

6. Format of output

-20-


Furthermore, the reduction of the 4-items for each factor to 2-items

took place, for the short form of the instrument to be finally a reality. But

still the questionnaire was too big to be completed in a quick way. For this

reason Ives et al. (1983) continued the process by eliminating factors with

undesirable psychometric characteristics, and by keeping those scales

with factor loadings of 0.50 or higher. Table (2.2) shows the factors

composing the original UIS instrument of Bailey and Pearson (1983), and

the Ives et al. (1983) short form questionnaire, and the factors that

consists the instrument.

-21-


1. Relationship with the EDP staff* 2. Processing of requests for changes to existing systems * 3. Means of input/output with the EDP centre 4. Interdepartmental competition with the EDP unit 5. Confidence in systems 6. Timeliness of output information 7. Chargeback method of payment for services 8. Perceived utility (worth versus cost) 9. Vendor support of hardware and software 10. Computer language used to interact with systems 11. Expectation (expected versus actual level of computer based support) 12. Correction of errors 13. Security of data 14. Degree of EDP training provided to users * 15. Users' understanding of systems * 16. Users' feelings of participation * 17. Currency (up-to-dateness) of the output information 18. Attitude of the EDP staff* 19. Reliability of output information* 20. Top management involvement in EDP activities 21. Format of output 22. Response/turnaround time 23. Determination of priorities for allocation of EDP resources 24. Convenience of access (to utilize the computer capability) 25. Relevancy of output information (to intended function)* 26. Volume of output information 27. Personal job effects resulting from the computer-based support 28. Accuracy of output information * 29. Precision of output information * 30. Communication with the EDP staff * 31. Organizational position of the EDP function 32. Time required for new systems development * 33. Personal control of EDP service received 34. Schedule of recurring output products and services 35. Documentation 36. Completeness of the output information * 37. Technical competence of the EDP staff 38. Flexibility of systems 39. Integration (automated sharing of information) of system database

Table 2.2: Ives, Olson and Baroudi user satisfaction measure (Source: Ives, B., et al., 1983, pp. 793) (*) Factors conserved in the Ives et al. (1983) short form questionnaire

To conclude with, it can be said that the Bailey and Pearson (1983)

measure, and the sort form by Ives et al. (1983), still remain two of the

most common used measures of IS user satisfaction, in order to value IS

success. It would be considered as a slight not to state that there are also

some negative perspectives for both instruments, especially when

-22-


someone attempts to apply them into HIS evaluation procedures. These

disadvantages are mentioned below (Ribiere, V., et al., 1999):

• The original factors of Bailey and Pearson (1983) are relatively old;

they were developed in 1977, and because of the rapid

development in computer technologies they can be considered as

invalid.

• The health care environment is completely different from other

organizations, and so the factors and the structure of the

questionnaires must be adapted to them.

• Costumers or users do not have the potentiality to make comments

or to suggest any changes.

• No research takes place for the user’s profile.

• Improvement directions are not prioritised.

2.4.3. Doll and Torkzadeh measurement of EUCS

To start with, the rapid growth in technologies and the different

changes in the way a user interact with the IS, were the main reasons

that drove Doll and Torkzadeh (1988) to the development of a new model

of measuring user satisfaction. After 1988 users had direct relationship

toward the IS, and the already existed models were accommodated to

support indirect interaction. They measured the overall user satisfaction

without evaluating user satisfaction according to the different

characteristics of the IS, such as ease of use. According to Doll and

Torkzadeh (1988), the main reasons for the development of an instrument

are the following:

• to focus on the satisfaction with the information provided by the

system

• to contain items in order to evaluate the ease of use of an

application

• to provide Likert-type scales

• to be a new, short, and easy to use for both practice and academic

research

-23-


• to be able to use it across many applications

• to enable researchers to examine the relationship between end-user

computing satisfaction (EUCS) with independent variables.

Taking all the above into consideration, Doll and Torkzadeh (1991)

emphasised that one major goal for the development of the proposed

model, was to contribute to the IS evaluation, in order to improve the IS

itself. Furthermore, Doll and Torkzadeh’s proposed model (1988) aims to

revise and validate Ives et al., (1983) model. Initially, and taken into

consideration the fact that many items from Ives et al. instrument seemed

less appropriate in an end-user environment, they subtracted them from

the new proposed instrument. These are:

• Relationship with the EDP staff

• Processing of requests for system changes

• Attitude of EDP staff

• Communication with EDP staff

• Time required for system development

• Personal control of EDP services

Additionally, the EDP staff/services and user knowledge/ involvement

were also excluded from the revised instrument.

For the development of the new instrument, previous surveys

related to the end-user computer satisfaction measurement were

thoroughly reviewed and based on them, Doll and Torkzadeh (1988) end

up with a 40-item instrument, where 7 of them were related to ease of

use, and 2 more measured perceived overall satisfaction. The initial

questionnaire, which was composed of these 40-items, was handed out to

96 end-users of five different firms. The instrument was supported by the

end-users overall satisfaction and by the specific aspects that satisfied or

dissatisfied them. Furthermore, through multiple tests and by eliminating

items according to the researchers ended up with 23 items. Five more

items were also subtracted from the instrument because their differences

were meaningful. To continue with, the new 18-item questionnaire was

administrated to 44 firms, and the results were that 618 appropriate

questionnaires were gathered for further analysis. Factor analysis

-24-


identified 12 new items categorised into 5 factors which constituted the

overall end-user satisfaction factor. Figure (2.3) modifies the model for

measuring EUCS and reveals the coded items of the instrument:

CONTENT

C1: Does the system provide the precise information you need? C2: Does the information content meet your needs? C3: Does the system provide reports that seem to be just about exactly what you

need? C4: Does the system provide sufficient information?

ACCURACY

A1: Is the system accurate? A2: Are you satisfied with the accuracy of the system?

FORMAT

F1: Do you think the output is presented in a useful format? F2: Is the information clear?

EASE OF USE

E1: Is the system user friendly? E2: Is the system easy to use?

TIMELINESS

T1: Do you get the information you need in time? T2: Does the system provide up-to-date information? Figure 2.3: A Model for Measuring End-User Computing Satisfaction

(Source: Doll and Torkzadeh, 1988, pp. 268)

Doll and Torkzadeh (1988) model is referred to be as one of the

most common used which has been tested into different environments,

-25-


and appears to have adequate reliability and validity across a variety of

applications.

2.4.4. Chin and Lee model of EUCS

In this section, Chin and Lee’s (2000) model is going to be

analysed. It is a model which has been based on Doll and Torkzadeh’s

(1988) instrument and its upgrade. Chin and Lee (2000) presented a new

set of measures focusing on five factors that already existed and by

adding one more factor; the operating speed. The researchers claimed

that according to the theory, there should be a strong correlation between

the operating speed of the system and overall user satisfaction.

Furthermore, in the already existed five factors, researchers added more

measures on each factor focusing most on satisfaction.

In the proposed model, except of the measure of the overall user

satisfaction, prior expectations and subsequent results from using the

system, and also how well the system fulfills users’ desires, were also

taken into consideration (Chin, W.W., and Lee, M.K.O., 2000). In figure

(2.4) the proposed model is represented:

-26-


A. PriorExpectations

C. Post-HocPerceptions

B. PriorDesires

D. Overall Discrepancy

With Expectations

F. Overall DiscrepancyWith Desires

E. Overall Evaluation of Expectation Discrepancy

G. Overall Evaluation of

DesireDiscrepancy

H. ExpectationBased

Satisfaction

I. DesireBased

Satisfaction

Direct and multiplicativecombination



J. Overall End-User

Satisfaction with an IS

Figure 2.4: Formation of End-User Computing Satisfaction

(Source: Chin, W.W., and Lee, M.K.O., 2000, pp. 556) The terms expectations and desires are concepts that must not be

confused. According to Chin and Lee (2000), expectations are what the

user expects to get by the system, and desires are what the user wants to

get by the system. For example, the user may expect low performance

from the IS, which is supported by the IS department which in turn lack of

knowledge, but he also actually desire a lot more from the IS, and vice

versa. To conclude, the Chin and Lee (2000) proposed model assumes

that satisfaction appears from both direct and multiplicative combinations

of expectation and desire based satisfaction.

2.4.5. Mahmood et al., research model of factors affecting IT EUS

Through an extended literature review of 45 surveys that have been

published between 1986 and 1998, Mahmood et al., (2000) attempted to

gather the factors that act as determinants of user satisfaction. The aim of

their study was to determine the extent on which 9 variables, which were

-27-


identified in these surveys, influence user satisfaction. These 9 variables

are the following:

1. User expectations

2. Ease of use

3. Perceived usefulness

4. User attitude towards information system

5. Organizational support

6. Perceived attitude of top management

7. User experience

8. User skills

9. User involvement in system development

The literature revealed that these variables fell into three major

categories: a) perceived benefits, b) organizational support, and c) user

background. All the above are demonstrated in Figure (2.5):

Figure 2.5: Research model of factors affecting IT end-user satisfaction

(Source: Mahmood et al., 2000, pp. 753)

The figure reveals that the first factor, perceived benefits, includes

the job-related benefits the user believes that will gain by using the

system, which in turn will affect the overall usage of the system. The

-28-


second factor includes characteristics of the user, such as user experience,

skills, and user involvement in the system development. Finally, the third

factor includes organizational support, and user and top management

attitude toward information systems.

The results of Mahmood et al., (2000) research revealed that

perceived benefits, such as user expectations, ease of use, and perceived

usefulness are strongly correlated to end-user satisfaction. Additionally,

end-user satisfaction was found to be strongly affected by user

background, and variables such as user experience and skills, and user

involvement. Finally, the relationship between end-user satisfaction and

organizational support were found to be statistically significant.

-29-

CHAPTER 3 HOSPITAL INFORMATION SYSTEM EVALUATION: THE RESEARCH

Chapter 3 Hospital Information System Evaluation: The research

The present chapter contains the research that took place for the

evaluation of hospital information systems, based on the replies to the

structured questionnaires, which will be described in the following

sections. After studying the research framework which concerns the

specific research, all the theory concerning user satisfaction, as a

surrogate measure of IS success, is summarised. Furthermore, all the

models and relative instruments on which our hypothetical model, and

consequently the questionnaire, was based are also discussed.

Additionally, the procedure for the selection of the population sample, the

research procedures for the detection of the hospitals that participated in

the survey, and the development of the instrument that was used in the

research are thoroughly analised. Last, the statistical analysis that took

place is thoroughly discussed with the appropriate data analysis, and with

the most important measures of the construct validity and reliability, and

factor analysis. After that and with the use of the Structural Equation

Modeling approach, the overall model with the extracted path coefficients

are also demonstrated.

3.1. User satisfaction as a measure of success

The selection of the evaluation method is the first thing a

researcher has to do, before starting any survey. Initially, the method that

is considered to be the most appropriate for the case of IS in Greek

hospitals, has been chosen. According to the literature (Au, N. et al.,

2002), there are three main approaches to evaluate IS effectiveness, and

these are: a) cost-benefit analysis, b) system usage, and c) end-user

satisfaction measurement. To start with, the cost-benefit analysis was the

first to be rejected, because it is difficult to prove that a specific benefit is

a result of an information system itself. More precisely, this method is too

-30-


hard to get implemented into Greek hospitals, because of the lack of

appropriate data (before after the implementation and the usage of the

IS). On the other hand, the method of system usage has also been

rejected as an option, because of the fact that system usage is not always

voluntarily, but most of the times personnel is forced to use the IS. So,

system usage is not able to generate valuable results for the IS

effectiveness and success. According to Ives et al., (1983), an information

system is not considered as a success when its users are dissatisfied with

its functionality. It is for sure that when users are not satisfied with the

quality of the system, the quality of the information and the services

provided, the system will be rejected or not used correctly (Ribiere et. al.,

1999). This is valid especially into hospitals where the personnel may be

suspicious towards new technologies and easily neglect them (Anderson,

J.G., 1997).

At this point, and having had the term user satisfaction explained, it

is of great importance to determine the users of IS as well. Users are all

the stakeholders of an HIS that are involved in its operation and

functionality. According to Ribiere et al. (1999), HIS users can be

categorised into internal and external. As internal users can be

characterised the nursing staff, doctors, the administrative staff, and

generally all those directly related to the use of an HIS. On the opposite

side, external users are the patients, suppliers, insurance providers, and

everyone that is indirectly related to an HIS. The present research focuses

on internal users only.

To sum up, user satisfaction is the most common sited method to

evaluate IS effectiveness and the one that it has been used by many

researchers. It is the measure that through the evaluation helps different

problems during the functionality or the use of the system to be located

(Salmela, H., and Turunen, P., 1997), and it is the one that is going to be

used in the present research.

-31-


3.2. Theoretical foundations

In the previous chapter all the models that measure the IS success

and the factors that affect computer user satisfaction were studied, in

order to a model for the measurement of IS success with user satisfaction

as the surrogate measure to be developed.

Furthermore, the up-dated IS success model of DeLone and McLean

(2003) refers to three major dimensions that affect the quality and

effectiveness of an IS and have great impact on user satisfaction: “system

quality”, “information quality” and “service quality”. The D&M IS success

model has been empirically tested and validated and has indicated

significant relationships between system quality and user satisfaction,

between information quality and user satisfaction, and between service

quality and user satisfaction (DeLone W.H. and McLean E., 2003).

Another evaluation instrument of measurement user information

satisfaction is the Doll and Torkzadeh (1988) instrument which is the most

widely known and the one that has been validated several times (Xiao L.

and Dasgupta S., 2002; Deng X. et al., 2008). They developed a 12 item

instrument which is comprised of 5 components: content, accuracy,

format, ease of use, and timeliness. Chin and Lee (2000) extended the

model by adding another factor; system speed, and by filling in more

questions in each factor.

Finally, according to Mahmood M.A. et al., (2000) the factors that

affect IT user satisfaction are spited into categories: perceived benefits

and convenience, user background and involvement and organizational

attitude and support. To continue with, nine variables have been identified

for each one of the above factors: perceived usefulness, ease of use, user

expectations, user skills, user involvement in system development,

organizational support, perceived attitude of top management toward the

project, and user attitude toward information systems. The results were

positive support for the influence of all nine variables on user satisfaction

but not to varying degrees. The most significant relationships were found

-32-


to be user involvement in systems development, perceived usefulness,

user experience, organizational support and user attitude towards the IS.

3.3. Theoretical model and hypothesis

As it has been mentioned above there are three measures of IS

success: cost-benefit analysis, system usage and user satisfaction. In this

study the evaluation of Greek hospitals information systems is going to be

conducted with the measurement of user satisfaction because it is proved

to be the most accurate measure according to different researchers.

According to DeLone and McLean (2003) model there are three

dimensions that are related to user satisfaction: a) system quality, b)

information quality and c) service quality, and also according to Mahmood

et al. (2000) another big dimension that is related to user satisfaction is

d) user background. Initially, from the Mahmood et al., (2000) model the

variables that measure “user background” have been selected, and these

are: user experience, user skills, and user training. Furthermore, from

DeLone and McLean (2003) model, internal support and external support

were selected as the variables that compose the factor “service quality”.

The measure of “information quality” is comprised by content, accuracy,

format, and timeliness that were chosen from Doll and Torkzadeh (1988)

model, and by data security that was chosen from Ives et al. (1983)

model. Finally, ease of use, system speed, screen interface, and error

recovery were also selected to compose the typical measures of the factor

“system quality” (Doll and Torkzadeh, 1988; Chin W.W. and Lee M.K.O.,

2000; Ribiere V. et al. 1999; Ives B. et al. 1983). To conclude with, these

four dimensions are going to be used in the theoretical model as the four

main factors that positively influence user satisfaction. The model is

presented in Figure (3.1).

-33-


Ease of Use

System Speed

Screen Interface

Content

Accuracy

Format

Timeliness

Internal Support

External Support

Error recovery

Data security

User Satisfaction

Information Quality

Service Quality

System Quality

User Background

Experience

Skills

Training

H1

H2

H3

H4

Figure 3.1: Research model of measuring user satisfaction

3.3.1. User background

According to the literature, user background can be measured with

experience, training, and user’s skills (Mahmood, M.A. et al., 2000;

Igbaria, M. and Nachman, S.A., 1990). It has been recognised that user’s

experience is, among others, an important factor, which can be associated

to IS success by leading to greater satisfaction. Igbaria (1990) points out

the need of continuously educated and experienced personnel as far as

computers are concerned. This comes from the findings that computer

experience is related to computer anxiety decrease and, thus to the

enhancement of user’s confidence and satisfaction (Igbaria, M., 1990).

Many researchers have shown that user experience can positively affect

-34-


user satisfaction even though it is hard to measure (Guimaraes, T. et al.,

1992). Of course, some researchers have found that computer experience

is not significantly correlated to user satisfaction (Lawrence, M, and Low,

G., 1993), others have found some correlation with satisfaction (Palvia,

P.C., 1996), and others have indicated that previous experience with

computers is a significant factor related to user satisfaction (Mahmood,

M.A. et al., 2000). Furthermore, training has also been identified as a

factor that affects IS effectiveness and user satisfaction (Igbaria, M.,

1990; Davis, S.A., and Bostrom, R.P., 1992). Given the fact that, user

training is directly connected to satisfaction and, thus, to system usage, it

is of vital matter that organizations should provide their personnel with

continuously training on new technologies by encouraging them to

participate in different seminars and presentations (Igbaria, M., 1990).

Last but not least, user’s skills with computers and, generally with IT

technology have also been identified as a factor that is directly associated

with user satisfaction and performance (Torkzadeh, G., and Lee, J., 2003).

Their findings have shown that user’s computing skills can help the user to

accept new computer applications and, increase their involvement with

information systems and, thus, user satisfaction.

Based on the above and the hypothetical model developed, the first

research hypothesis that has to be tested is formulated as following:

H1. User background will positively affect the hospital personnel

satisfaction.

3.3.2. System Quality

According to the literature, system quality is one of the most

important factors that influence user satisfaction and it can be measured

with ease of use, system speed, screen interface, and error recovery

(DeLone W.H. and McLean E., 2003; Doll, W.J., and Torkzadeh, G., 1988;

Chin W.W. and Lee M.K.O., 2000; Ribiere V. et al. 1999; Ives B. et al.

1983; Bailey, J.E., and Pearson, S.W., 1983). When a system is easy to

use, and complex procedures do not exist, it is more likely to be accepted

-35-


by the users (Davis, F.D., 1989). According to Igbaria et al., (1995), it is

very important for the organizations to be very careful when designing an

information system. A system must be developed based on user’s abilities

and needs, in order to succeed ease of use and, then, user satisfaction.

Furthermore, another variable that leads to high levels of user satisfaction

with an IS is system speed. The rapid development of technology, and the

constantly increased demands by the organizations to their staff constitute

the speed that the applications are executed one of the most important

variable that influence IS success (Chin W.W. and Lee M.K.O., 2000). To

continue with, screen interface and error recovery are also stated among

the most significant variables that affect user satisfaction (Ribiere V. et al.

1999). The work environment that is offered to the user by the system on

the one hand and, the ability to correct a false information or mistake on

the other hand are key factors that lead to IS success (Ribiere V. et al.

1999).

Based on the above and the hypothetical model developed, the

second research hypothesis that has to be tested is formulated as

following:

H2. System Quality will positively affect the hospital personnel

satisfaction.

3.3.3. Information Quality

According to the literature, information quality mostly refers to

measures of information systems output (Pitt, L.F. et al., 1995). Typical

measures of information quality contain content, accuracy, format,

timeliness, and data security (Etenazi-Amoli, J., and Farhoomand, A.F.,

1996; Chin W.W. and Lee M.K.O., 2000; Doll, W.J., and Torkzadeh, G.,

1988). To begin with, the content of the information provided by the

system will satisfy the users if it fits to their needs (Doll, W.J., and

Torkzadeh, G., 1988; Chin W.W. and Lee M.K.O., 2000). At the same way,

researchers have shown that, measures that refers to information product

items, such as the accuracy of the output information, the format and the

-36-


material design of the layout, and the timeliness of the information

available to the users, positively influence user satisfaction (Chin W.W.

and Lee M.K.O., 2000, Doll, W.J., and Torkzadeh, G., 1988; Bailey, J.E.,

and Pearson, S.W., 1983). Finally, the confirmation and the confidence

that data are secured from unauthorised alteration or loss leads to greater

user satisfaction (Etenazi-Amoli, J., and Farhoomand, A.F., 1996; Bailey,

J.E., and Pearson, S.W., 1983).



following:

H3. Information Quality will positively affect the hospital

personnel satisfaction.

3.3.4. Service quality

Service quality is another major factor that, according to DeLone

and McLean (2003), affects user satisfaction and IS effectiveness. Typical

measures of service quality mainly include internal and external support

(Etenazi-Amoli, J., and Farhoomand, A.F., 1996; Thompson, R.L. et al.,

1991). It is for certain that, when the IS department (internal support),

and the external vendor (external support) provide help and solutions to

different problems that derive from computer usage, users feel more

confident towards obstacles, which, in turn, leads to greater satisfaction

and system usage as well (Thompson, R.L. et al., 1991; Etenazi-Amoli, J.,

and Farhoomand, A.F., 1996).



following:

H4. Service Quality will positively affect the hospital personnel

satisfaction.

-37-


3.4. Methodology Research

3.4.1. Instrument Development

Many researchers have spent a lot of time to find a valid instrument

for user satisfaction. Among these researchers, Bailey and Pearson (1983)

were the first to develop an instrument that was composed of 39 items

that measure user satisfaction. Later on, Ives et al. (1983) created a short

form version of 13 items from the initial Bailey and Pearson’s (1983)

instrument. Furthermore, Doll and Torkzadeh (1988) revised the Ives et

al. (1983) instrument and created a 12 item, which measured end-user

computing satisfaction based on five factors: content, accuracy, format,

ease of use, and timeliness. Their instrument is one of the most common

used in the field of user satisfaction and the one that has been validated

by many researchers. To continue with, Doll and Torkzadeh’s (1988)

instrument was later enhanced and revised by Chin and Lee (2000). They

added one more factor; system speed, and enriched each factor with more

items in the instrument. The present research uses the revised Doll and

Torkzadeh’s (1988) tool by Chin and Lee (2000) as a basis, because it is

an instrument that is well validated and widely used, and it has been

developed especially for end-user computing applications (Chen, L-da et

al., 2000). Furthermore, the factors and the items that compose the

instrument are suitable for the present research in Greek hospitals.

The rapid development of technology in all organizations, and the

great responsibilities that users face in their every day working life when

handling large amounts of data, classify previous experience, user skills,

and training in computers very important factors that lead to user

satisfaction and, thus, IS success (Mahmood, M.A. et al., 2000; Lawrence,

M, and Low, G., 1993; Etenazi-Amoli, J., and Farhoomand, A.F., 1996;

Igbaria, M., 1990; Mosley, I.T., 2001). Additionally, the work environment

(screen interface) that is offered by the system, the abilities of the system

to correct mistakes and to protect data from loss are considered to be

factors that play significant role on user satisfaction and IS effectiveness

-38-


(Etenazi-Amoli, J., and Farhoomand, A.F., 1996; Bailey, J.E., and Pearson,

S.W., 1983; Ribiere V. et al. 1999). Furthermore, many researchers

claimed that the support that is offered to the users, either by the IS

department of the organization, or by the vendor of the IS, has great

effect on user satisfaction (Chen, L-da et al., 2000; Etenazi-Amoli, J., and

Farhoomand, A.F., 1996; Thompson, R.L. et al., 1991).

Therefore, the present research includes all the above aspects. All

the definitions and their supported literature of the variables that were

used for the research model of measuring user satisfaction are

demonstrated at Table (3.1). The literature review that was mentioned

resulted in the structure of the questionnaire that has been distributed to

the Greek hospitals (see Appendix A).

The questionnaire was divided in two parts. Part I contained 6

questions which concerned general information (demographic

characteristics) about the respondents. Questions 1 to 4 involved

information about the sex and the age of the respondents, their

educational level, and their position at the hospital that they were

occupied. Questions 5 and 6 determined the length of time the respondent

has been working in the healthcare field, and the length of time the

respondent has been using the hospital’s information system.

In part II the respondent were asked to rate the statements on a

five-point Likert scale, where 1 = strongly disagree, 2 = disagree

somewhat, 3 = neutral, 4 = agree somewhat, and 5 = strongly agree. The

five-point Likert scale was selected because it gives the opportunity to the

respondents to chose to state neutral, as opposed to other Likert scale

types. This part consisted of five sets of items.

The first set, from item A1.1 to item A3.3, requested the

respondents to rate each statement, which consisted information about

the user background, as far as their previous experience, their skills, and

their training were concerned. A strongly agree response suggested a high

level of satisfaction an, a strongly disagree response suggested a low level

of satisfaction to all items of the questionnaire.

-39-


In the second set, items B1.1 to B1.5 referred to their level of

agreement to statements that concerned the ease of use of the system;

items B2.1 to B2.4 consisted statements about the level of satisfaction

with the speed of the system; items B3.1 to B3.4 referred to screen

interface of the work environment of the system; B4.1 to B4.3 consisted

statements about the way the system controlled the error recovery. Items

B1.1 to B4.3 of the second set of the questionnaire measured the quality

of the system.

The third set indicated the respondents to state their level of

satisfaction with the quality of the information that the system offered to

them. Items C1.1 to C1.5 referred to the quality of the information

content offered by the system; items C2.1 to C2.4 consisted statements

about the accuracy of the information provided; items C3.1 to C3.5

consisted statements about the format of the information; items C4.1 to

C4.5 referred to the respondents level of agreement to statements that

concerned the timeliness of the information available to them; items C5.1

to C5.3 referred to the security of the data generated by the system.

The fourth set of the questionnaire indicated the respondents to

state their level of satisfaction as far as the service quality was concerned.

Items D1.1 to D1.3 referred to internal support provided to the users by

the IS department, and items D2.1 to D2.3 referred to external support

provided to the users by the vendor of the IS.

Last but not least, the sixth set of the questionnaire indicated three

statements about the overall satisfaction with the use of the information

system, rating them to a five-point Likert scale, where 1 = very

dissatisfied or extremely dissatisfied, and 5 = very satisfied or extremely

satisfied.

All the data that were gathered from the collected questionnaires

and, the statistical analysis that was generated by them and follows in the

next section resulted with the use of the statistical SPSS program. At the

beginning, confirmatory factor analysis was conducted in order to

estimate the adequacy of the measurement model (Chang, M.K. 1998)

and, after that, Structural Equation Modelling (SEM) was also performed in

order to test the structural model’s fit (Anderson, J.C., and Gerbing, G.W.,

1998) with the use of the AMOS software.

-40-


Factor Definition Items Supporting

Experience “The level of previous experience with computer systems” (Lawrence & Low, 1993)

3

Lawrence & Low (1993)

Training

“The amount of specialized instruction and practice that is offered to the user to increase the user’s proficiency in utilizing the computer capability that is unavailable” (Bailey & Pearson, 1983)

4

A2.1: Etenazi & Farhoomand (1996) A2.2-A2.4: Igbaria (1990)

Skills

“The ability to use computer-based systems in the specific functional areas of information acquisition, storage retrieval, analysis, interpretation, and presentation by using software applications” (Mosley, 2001)

3

Mosley, I.T. (2001)

Ease of Use

“The degree to which a person believes that using a particular system would be free of effort” (Davis, 1989)

5

B1.1-B1.2: Doll & Torkzadeh (1988) B1.3: Chin & Lee (2000) B1.4-B1.5: Venkatesh et al., (2003)

System Speed

“The speed with which a computer system responds to different tasks” (Chin & Lee, 2000)

4

Chin & Lee (2000)

Screen Interface

“The work environment which the systems offers to the user for the importing, processing as well as exporting of the information” (Ribiere et al., 1999)

4

Ribiere et al., (1999)

Error Recovery

“The ability of the system to offer you a chance to correct a mistake (incorrect information)” (Ribiere et al., 1999)

3

B4.1-B4.2: Ribiere et al., (1999) B4.3: Bailey & Pearson (1983)

Content

“The extent of the information content provided by the system that fits the users need” (Chin & Lee, 2000)

5

C1.1-C1.3: Doll & Torkzadeh (1988) C1.4-C1.5: Chin & Lee (2000)

Accuracy

“The correctness of the output information” (Bailey & Pearson, 1983)

4

C2.1- C2.2: Doll & Torkzadeh (1988) C2.3-C2.4: Chin & Lee (2000)

Format

“The material design of the layout and display of the output content” (Bailey & Pearson, 1983)

5


Timeliness

“The availability of the output information at a time suitable for its use” (Bailey & Pearson, 1983)

5


Data Security

“The safeguarding of data from misappropriation or unauthorized alteration or loss” (Bailey & Pearson, 1983)

3

Etenazi & Farhoomand (1996)

Internal Support

“The amount of support provided to the users of the system from the IS department of the organization” (Chen et al., 2000)

3

D1.1-D1.2: Chen et al. (2000) D1.3: Etenazi & Farhoomand (1996)

External Support

“The amount of support provided to the users of the system from the staff of the vendor of the IS” (Etenazi & Farhoomand, 1996)

3

D2.1: Etenazi & Farhoomand (1996) D2.2-D2.3: Thompson et al., (1991)

Table 3.1: Definitions and Supported Literature

-41-


3.4.2. Research Population and Sample

For the purpose of this survey, the first thing that should be

determined is the selection of the population that will participate on the

research. Although the idea of hospital information systems began in the

early 1960s (Zviran M., 1990), the first well organised attempt in Greece

began at the end of 1980s. That means that HISs are still in progress in

Greece, in relation to other nations, and ISs are not used by the whole

personnel of the hospitals, but mainly by the administrative staff.

According to the chief of the IS department from the General Hospital of

Xanthi, by the end of 2009 and through the fourth Community Support

Framework (4th CSF), schedules have been organised for the introduction

of a system, part of the integrated HIS, to the whole nursing staff of the

hospitals. It is very important that the population sample must not include

people that generally work in hospitals, but the actual users of the ISs,

the ones that use the system in their every day life at work. These are all

the stakeholders that participate in the IS functionality. Thus, the

population sample was restricted to healthcare personnel, such as medical

staff, nursing staff, and administrative staff from every department of the

hospital. Additionally, the research was limited to hospitals in Greece,

which were randomly selected to participate in the research.

3.4.3. Research Procedures

The gathering of all the appropriate data was performed in different

phases. At the beginning, a letter was sent and a week later a telephone

contact with the chief of the IS department of 6 Greek Hospitals took

place. The hospitals were the followings: General Hospital of Xanthi,

General Hospital of Heraklion Crete “Venizeleio-Pananeio”, General

Hospital of Thessaloniki “Georgios Papanikolaou”, General Hospital of

Thessaloniki “Papageorgiou”, General Panarkadiko Hospital of Tripolis, and

General University Hospital of Alexandroupoli. In the present research, we

wanted to include hospitals that are not located in the Athens (capital)

-42-


region. This is why hospitals from different and diverse regions of Greece

were selected. The discussion concerned their willingness to participate in

the survey. After the initial contact, a preliminary questionnaire was sent

to the chief of the IS department of each hospital, in order to ascertain

the quality and the meaning of the questions that composed the

questionnaire, and to rate the relevance of the items in terms of user

satisfaction with the IS. After that the initial questionnaire was partly

adjusted to end up with the final form (see Appendix A).

The next phase started with the postage of the questionnaires by

mail, accompanied with the formal letter that was sent at the beginning. A

self-addressed, stamped envelope was enclosed for the convenience of

the IS department, in order to return the completed questionnaires back

to the researcher. The questionnaires were sent in 22nd of June to all

hospitals simultaneously and the collecting date was arranged to be after

2 weeks. Unfortunately, a small delay was inevitable because of the

summer months, where the most staff is on holidays in Greece. The

collection ended at the first week of August. From the 140 questionnaires

that initially were sent, a total number of 100 questionnaires were finally

gathered from 4 hospitals. The General Panarkadiko Hospital of Tripolis

and the General Hospital of Xanthi were not able to continue participating

in the research. More precisely, 41 questionnaires were gathered from the

General Hospital of Heraklion Crete “Venizeleio-Pananeio”, 29 from the

General University Hospital of Alexandroupoli, 18 from the General

Hospital of Thessaloniki “Papageorgiou” and, finally, 12 from the General

Hospital of Thessaloniki “Georgios Papanikolaou”. All of the 100

questionnaires found to be complete and usable for research, achieving a

rate response of 71.40% percent. The demographic profile of the

respondents is demonstrated in Table (3.2). The Table shows that the

research sample is composed of 39% nursing staff, 8% doctors and 53%

administrative staff. Furthermore, the sample consists mainly of female,

between the ages of 31-50, well educated, reaching a 10 year experience

with using an information system.

-43-


Frequency (persons)

Frequency (%)

Sex Female Male

73 27

73% 27%

Age

20-30 31-40 41-50 >50

missing

13 31 37 11 8

13% 31% 37% 11% 8%

Educational level

High School Bachelor’s Degree Master’s Degree Doctoral Degree

19 69 10 2

19% 69% 10% 2%

Personnel

Nursing Staff Doctors

Administrative Staff

39 8 53

39% 8% 53%

Years in the present job

<15 years ≥15 years missing

68 27 5

68% 27% 5%

Years using the IS

<10 years ≥10 years missing

57 18 25

57% 18% 25%

Table 3.2: Demographic characteristics of respondents

-44-


3.4.4. Data analysis

i) Construct validity and construct reliability

It must be stated that it is very important to verify that the items

which composed the questionnaire explicate defined ideas through

analysis of validity and reliability. In the present survey, a factor analysis

was performed (see Appendix B) by using the Kaiser-Meyer-Olkin (KMO)

measure of sampling adequacy and the Bartlett’s Test of Sphericity, which

are recommended for measuring construct validity (Hair, J. et al., 1995).

Additionally, the Cronbach’s (a) reliability test was used in order to

estimate the internal consistency of measurements (Straub, D.W. et al.,

2004). Finally, the Total Variance Explained (TVE) score was also used in

order to measure the cumulative percentage of the variance that is

explained by all factors. The results of the factor and reliability analysis,

and the results of the descriptive statistics are summarised at Table (3.3).

To begin with, through the statistical analysis and as Table (3.3)

reveals, the results of the descriptive statistics show that items A1.1 to

A1.3 that constitute the factor “experience” have mean scores between 3

and 4 indicating that the respondents have a relatively positive perception

toward these constructs. The same indications are also revealed for items

A3.1 to A3.3 that compose the factor “skills”, items A2.1 and A2.4, B1.1

to B1.5, B2.1 to B2.4, B3.1 to B.3.4, B4.1 to B4.3 that compose the

factors “training”, “ease of use”, “system speed”, “screen interface”, and

“error recovery” respectively. Mean scores of items C1.1 to C1.5, C2.1 to

C2.4, C3.1 to C3.5, C4.1 to C4.3, and C5.1 to C5.3 that constitute the

factors “content”, “accuracy”, “format”, “timeliness”, and “data security”

respectively are also between 3 and 4, expect from items C4.4 and C4.5

that are below 3. Items A2.2 and A2.3 are also below 3 showing that

training is a result of personal effort. Last, items D1.1 to D1.3 of the

factor “internal support”, D2.1 to D2.3 of the “external support”, and

items SAT1 to SAT3 of the overall satisfaction factor resulted mean scores

between 3 and 4, also showing the high positive perception by the

respondents.

-45-


1 2 3 4 5 Statistics Mean Std. Dev.

Loadings

Experience

A1.1 A1.2 A1.3

2,0 3,1

11,2

5,1 9,2

14,3

15,2 22,4 27,6

47,5 40,8 27,6

30,3 24,5 19,4

K.M.O.=0,704 Bartlett’s Sig=0,000 (TVE)=73,279 Cronbach (a)=0,806

3,99 3,74 3,30

0,920 1,029 1,254

0,858 0,885 0,825

Training

A2.1 A2.2 A2.3 A2.4

8,4 48,4 36,7

10,5 17,9 8,2 4,1

16,8 10,5 9,2

12,2

32,6 14,7 29,6 29,6

31,6 8,4

16,3 54,1


3,68 2,17 2,81 4,34

1,257 1,389 1,577 0,849

Dropped 0,895 0,895

Dropped

Skills

A3.1 A3.2 A3.3

3,1 3,1

16,2

5,1 10,2 18,2

16,3 15,3 21,2

44,9 32,7 23,2

30,6 38,8 21,2


3,95 3,94 3,15

0,978 1,111 1,380

0,840 0,881 0,813

Ease of Use

B1.1 B1.2 B1.3 B1.4 B1.5

2,0 1,0 2,0 7,1 3,1

10,0 12,2 12,1 4,0

12,4

20,0 15,3 23,1 18,2 20,6

48,0 50,0 45,5 42,4 44,3

20,0 21,4 17,2 28,3 19,6


3,74 3,79 3,64 3,81 3,65

0,960 0,955 0,974 1,113 1,031

0,759 0,896 0,855 0,810 0,767

System Speed

B2.1 B2.2 B2.3 B2.4

2,1 7,1 7,1

12,2

13,5 9,2 6,1

18,4

20,8 24,5 24,2 24,5

46,9 44,9 42,4 26,5

16,7 14,3 20,2 18,4


3,63 3,50 3,63 3,20

0,987 1,077 1,093 1,284

0,840 0,887 0,865

Dropped

Screen Interface

B3.1 B3.2 B3.3 B3.4

4,1 2,0 4,1 4,0

10,3 10,2 9,2

10,1

26,8 22,4 25,5 31,3

40,2 40,8 37,8 26,3

18,6 24,5 23,5 28,3


3,59 3,76 3,67 3,65

1,038 1,006 1,063 1,119

0,889 0,887 0,896 0,905

Error Recovery

B4.1 B4.2 B4.3

10,2 9,1 7.3

15,3 20,2 16,7

25,5 27,3 28,1

34,7 27,3 39,6

14,3 16,2 8,3


3,28 3,21 3,25

1,191 1,206 1,066

0,931 0,947 0,920

Content

C1.1 C1.2 C1.3 C1.4 C1.5

7,1 6,1 7,1 7,2 5,2

8,2 10,2 7,1

15,5 10,4

22,4 18,4 24,5 15,5 20,8

45,9 44,9 38,8 41,2 45,8

16,3 20,4 22,4 20,6 17,7


3,56 3,63 3,62 3,53 3,60

1,085 1,107 1,126 1,191 1,061

0,866 0,934 0,906 0,938 0,916

Accuracy

C2.1 C2.2 C2.3 C2.4

7,1 7,1 3,1 4,1

8,2 8,2 9,2

10,3

25,5 23,5 22,4 21,6

41,8 39,8 38,8 37,1

17,3 21,4 26,5 26,8


3,54 3,60 3,77 3,72

1,095 1,128 1,043 1,097

0,895 0,961 0,934 0,880

Format

C3.1 C3.2 C3.3 C3.4 C3.5

7,1 7,1 8,2 7,1 7,1

8,2 7,1 4,1

12,2 13,3

29,6 24,5 20,4 19,4 14,3

48,0 39,8 51,0 49,0 51,0

7,1 21,4 16,3 12,2 14,3


3,40 3,61 3,63 3,47 3,52

0,992 1,118 1,069 1,086 1,114

0,918 0,931 0,939 0,931 0,930

C4.1 C4.2 C4.3

3,1 5,1 5,5

11,2 10,2 6,6

17,3 16,3 22,0

45,9 40,8 42,9

22,4 27,6 23,1

3,73 3,76 3,71

1,031 1,122 1,068

0,923 0,955 0,912

Timeliness

C4.4 C4.5

26,5 18,9

21,4 21,1

27,6 24,2

18,4 25,3

6,1 10,5

K.M.O.=0,695 Bartlett’s Sig=0,000 (TVE)=52,178 84,354 Cronbach (a)=0,740

2,56 2,87

1,236 1,282

0,896 0,887

Data Security

C5.1 C5.2 C5.3

6,2 4,1 3,1

8,2 15,3 14,4

38,1 30,6 36,1

22,7 27,6 26,8

24,7 22,4 19,6


3,52 3,49 3,45

1,138 1,124 1,061

0,863 0,947 0,934

Internal Support

D1.1 D1.2 D1.3

4,0 8,1 5,1

12,1 7,1

16,2

8,1 13,1 14,1

42,4 40,4 43,4

33,3 31,3 21,2


3,89 3,80 3,60

1,124 1,195 1,142

0,926 0,934 0,886

External Support

D2.1 D2.2 D2.3

4,0 3,1 5,1

15,2 14,3 11,1

35,4 41,8 49,5

29,3 23,5 21,2

16,2 17,3 13,1


3,38 3,38 3,26

1,057 1,031 0,996

0,908 0,900 0,853

Overall Satisfaction

SAT1 SAT2 SAT3

4,0 4,0 3,0

4,0 2,0 2,0

47,5 47,5 50,5

30,3 34,3 32,3

14,1 12,1 12,1


3,46 3,48 3,48

0,929 0,885 0,850

0,949 0,964 0,934

Table 3.3: Factor and reliability analysis

-46-


Starting with the factor analysis, it can be stated that KMO is above

the 0,500 threshold for all factors (Hair, J. et al., 1995). More precisely,

KMO varies between 0,500 and 0,870. The Cronbach’s (a) reliability test

revealed values more than the 0,600 threshold (Malhotra, N., 1999), while

TVE score is above 0,500 for all factors (Straub, D.W., 1989). Last but not

least, the analysis has shown that factor loadings are at acceptable levels

for all items. At this point, it must be pointed out that 3 out of 52 items

were dropped out of the analysis. Additionally, it must stated that the

statistical analysis has indicated that the items originally composed the

factor “timeliness” have loadings in two factors (see Appendix B),

separating this way items C4.1 to C4.3 from items C4.4 to C4.5. This may

have happened because of the reverse meaning of the C4.4 and C4.5

items.

The results demonstrated at Table (3.3) suggest that, users have a

very satisfactory background with acceptable experience in programs for

personal computing and information systems as well. As far as the

training factor is concerned, it is obvious that, it is a result of personal

effort and self study rather than a privilege that should be provided to

them by the IS department of the organization or the vendor of the IS.

Furthermore, and as far as factor skills is concerned, users believe that

they have a satisfactory level of the ability to interact and execute

software packages very easy, such as entering data, formulas and

generating calculations.

To continue with, users are highly satisfied with their interaction

with the system which is understandable, friendly in use and easy to

handle; highly satisfied with the speed that the system operates; highly

satisfied with the screen interface of the system, such as the screen

layout and colors which are pleasant to the user. Moreover, users are

satisfied with the ability that the system offers to correct different

mistakes in a simply way.

It is very interesting to see that most of the users are very satisfied

as far as the information provided to them by the system. They believe

that the information delivered to them covers their needs with regards to

-47-


the content, the accuracy, and the way that the information is presented

to them. As far as the factor timeliness is concerned, it can be stated that

generally users are satisfied with the time that takes to generate useful

information from the system. The factor data security is the one that most

users are neutral towards it, revealing this way that they are not very

sure or convinced that the system secures data against unauthorised

alteration or physical damage.

Furthermore, as far as the internal and external support factors are

concerned, it cab be said that most of the support provided to the users

comes from the IS department of the organization and then from the

vendor of the system.

Finally, the results from the factor overall satisfaction reveals that

most of the users hold a neutral position. This may mean that the system

needs improvements so that users to become more satisfied with their

whole operation of the system.

ii) Structural Model Fit

At the beginning of this section it is necessary to revise some basic

models that were used in the present survey. To begin with, it must be

said that DeLone and McLean (1992) claimed that user satisfaction can be

measured with three factors: information quality, system quality and

service quality. Furthermore, according to Mahmood et al., (2000) another

dimension that is related to user satisfaction is user background. Finally,

from DeLone and McLean (2003) revised model service quality is

considered to be another factor that is directly connected with user

satisfaction. To continue with, in this part of the survey it has been tested

whether the 14 items that were demonstrated in figure (3.1) measure the

4 factors stated above and, in turn, whether these 4 factors measure user

satisfaction. In few words, the hypothetical model (figure 3.1) of this

research was tested. The hypothetical model was tested using the

Structural Equation Modeling approach. For this reason and according to

the needs of this study five fit measures were used in order to evaluate

the overall model fit. These were the following: chi-square/degree of

-48-


freedom (x2/df), goodness-of-fit index (GFI), comparative fit index (CFI),

normed fit index (NFI), and root mean square residual (RMR). In the first

part, “user background” was tested whether it can be measured with

experience, training and skills; “system quality” with ease of use, system

speed, screen interface, and error recovery; “information quality” with

content, accuracy, format, timeliness and error recovery; “service quality”

with internal and external support. The results that were generated with

the use of the AMOS software package are demonstrated at Figures (3.2),

(3.3), (3.4) and (3.5).

To begin with, Figure (3.2) which refers to user background shows

that experience is strongly related to user background (0,87), skills is also

strongly related to user background (0,77) and training is related in an

acceptable level (0,50) also with user background. Furthermore, Figure

(3.3) reveals that ease of use (0,84), system speed (0,55), screen

interface (0,71) and error recovery (0,63) are strongly related to the

factor system quality. In Figure (3.4), it can be observed that the four first

factors, meaning content (0,88), accuracy (0,89), format (0,90) and

timeliness (0,60) are strongly related to information quality, and data

security (0,43) reveals that it is related to the factor in an acceptable

level. Finally, internal support with a loading of 0,77 and external support

with 0,90 shows that they are also strongly related to service quality, as it

can be seen in Figure (3.5). Here it can be stated that there is no change

from the initial hypothetical model.

As far as the second part is concerned, Figure (3.6) represents the

overall model and Table (3.4) the summarised results from the testing of

the models with the use of the Structural Equation Modeling approach.

-49-


Figure 3.2: User Background

Figure 3.3: System Quality

Figure 3.4: Information Quality

Figure 3.5: Service Quality

-50-


Figure 3.6: User Satisfaction

CMIN/DF GFI NFI CFI RMR User Background

2,022 0,964 0,965 0,981 0,027

System Quality

8,956 0,929 0,858 0,868 0,051

Information Quality

3,579 0,912 0,941 0,908 0,038

Service Quality

1,022 0,977 0,962 0,984 0,006

User Satisfaction

5,762 0,893 0,842 0,853 0,119

Table 3.4: Statistics concerning structural equation models

-51-


As it can be observed, the CMIN/DF score is within the acceptable

levels, below 5 (Harrison, A.W. and Rainer, R.K., 1996) except from the

system quality which is at 8,956. However, it is still acceptable. GFI score

is above the 0,90 threshold (Bollen, K.A. and Long, J.S., 1993), CFI score

is also close to the 0,90 threshold (Smith, T.D. and McMillian, B.F., 2001),

while NFI is close to 0,90. Last, RMR score is below the 0,1 threshold

(Bollen, K.A., 1989; Hair, J.F. et al., 1992) except from users satisfaction

value which is very close (0,119). Furthermore, Figure (3.6) represents

both the overall model and the extracted path coeffients within the model

and the adjusted R2 score, which explains 53% of the variance in user

satisfaction. As it can be noticed, examining the relationship between user

background and user satisfaction, it is obvious that there is a weak

positive (0,23) relationship. This relation reveals that the more

experienced the user is, with adequate skills and training provided by the

organization itself or by the vendor may improve satisfaction with the

system. As far as the relationship between information quality and user

satisfaction is concerned, it can be said that there is a highly positive

relation (0,54). This relation shows that if the information provided to the

users covers their needs, such as accuracy of the information and content

which satisfies their daily needs, on time delivered information, then their

satisfaction with the of the system will be improved. Furthermore, the

model reveals that there is a highly positive impact between service

quality and user satisfaction (0,43). This relation shows that the better

service provided to the user when needed, internal or external, generates

more confidence to the users and, thus, higher satisfaction. Finally, it can

be observed that there is a very high relationship between system quality

and information quality (0,88).

Summarising, from the initial four hypotheses Figure (3.1) three of

them have been proved through this present survey. Hypothesis 1 (H1)

showing that user background positively affect user satisfaction;

hypothesis 3 (H3) showing that information quality positively affect user

satisfaction, and hypothesis 4 (H4) showing that service quality positively

-52-


affect user satisfaction. Hypothesis 2 (H2) which was referred to the direct

positive relationship between system quality and user satisfaction has

been rejected. At this point, it must be said that it has been found that

system quality indirectly affect user satisfaction through information

quality.

-53-

CHAPTER 4 CONCLUSIONS AND RESEARCH LIMITATIONS

Chapter 4 Conclusions and research limitations

4.1. Conclusions

This survey has examined the relationship between user satisfaction

and user background, user satisfaction and system quality, user

satisfaction and information quality and, finally, user satisfaction and

service quality. For this purpose all the relative theory and previous

studies were thoroughly analysed and the testing hypothesis were easily

generated. It is useful to remind of the sample’s characteristics that were

used in the present survey. The sample was composed of 4 Greek Hospital

from different and diverse regions of Greece. The demographic

characteristics have indicated that most users of hospital information

systems are occupied at the administrative sector of their organization

reaching a 53 percentage among the 100 respondents that participated in

the research. In order to test whether the relationship between the

dependent variable and the independent variables that were demonstrated

at Figure (3.1) exist and confirm the results from previous researchers,

the statistical program SPSS was used as the basic instrument in this

study. Firstly, factor analysis was conducted in order to estimate the

adequacy of the measurement model, and the construct validity and

reliability of the instrument. Secondly, Structural Equation Modeling was

performed in order to test the structural model fit and examine the paths

of the model, with the use of the AMOS software package. The statistical

analysis has indicated that the four initial hypotheses (see Figure 3.1) of

the research model have been slightly modified. The present survey have

shown that user background directly positively affect user satisfaction

(H1), confirming this way the research of Mahmood et al., (2000) in which

the first hypothesis was based on. The modification concerns the DeLone

and McLean (2003) three factors that have been proved to affect user

satisfaction. Information quality (H3) and service quality (H4) have been

-54-


tested and have shown that directly positively affect user satisfaction. The

diversity concerns the factor system quality and its relationship to user

satisfaction. The results from the Structural Equation Modeling approach

have shown that hypothesis 2 (H2) is rejected in this present research,

meaning that system quality does not affect user satisfaction directly, but

affects it indirectly through information quality (see Figure 3.6). This

change may have occurred because of the misunderstanding of the

questions by the respondents of the questionnaire, since system quality

have been proved to be one of the major factors that has direct impact on

user satisfaction by other researchers (DeLone, W.H. and McLean, E.R.

1992; DeLone, W.H. and McLean, E.R. 2003).

Summarising, it can be stated that the evaluation of Hospital

Information Systems through the user satisfaction measurement is a

major research field, especially in Greece where it still remains in primary

stage. Further research is necessary and critical in order to reveal the

factors that really affect user satisfaction with information systems. The

results of these findings may lead hospital organizations to realise that the

use of information technology is, nowadays, very important and its use

may lead to better operation of the healthcare sector. But, information

technology is useless, if the users that daily interact with information

systems are dissatisfied with their functionality and reject them.

4.2. Research Limitations

This research embraced different limitations. The limitations

involved the sample size, which is considered as a small one (100

respondents) that was gathered from four Greek hospitals. Furthermore,

the respondent’s completion of the questionnaire and the questionnaire

itself may be considered as limitation as well, meaning that the responses

might have been biased because of the time pressure, and some of the

statements of the questionnaire might have been unclear. Moreover,

many critical factors that have direct influence to user satisfaction, such

as system flexibility, user’s expectations, relevancy, user’s anxiety etc,

-55-


were omitted to keep the research simple. Finally, as a limitation can be

considered the fact that in Greek hospitals most of the IS users come from

the administrative staff and little from the nursing staff, where in other

nations the situation is completely different.

-56-

REFERENCES

References

Aggelidis, V.P. and Chatzoglou, P.D. (2008), “Methods for evaluating

hospital information systems: a literature review”, EuroMed Journal

of Business, Vol. 3, No. 1, pp. 99-118

Ammenwerth, E., Ehlers, F., Hirsch, B. and Gratl, G. (2007), “HIS-

Monitor: An approach to assess the quality of information

processing in hospitals”, International Journal of Medical

Informatics, Vol. 76, pp. 216-225

Ammenwerth, E., Graber, S., Herrmann, G., Burkle, T. and Konig, J.

(2003), “Evaluation of health information systems: problems and

challenges”, International Journal of Medical Informatics, Vol. 71,

pp. 125-135

Anderson, J.C. and Gerbing, G.W. (1998), “Structural equation modeling

in practice: a review and recommended two-step approach”,

Psychological Bulletin, Vol. 103, pp. 411-423

Anderson, J.G. (1997), “Clearing the way for physicians’ use of Clinical

Information Systems”, Communication of the ACM, Vol. 40, No. 8,

pp. 83-90

Au, N., Ngai, E.W.T. and Cheng, T.C.E. (2002), “A critical review of end-

user information system satisfaction research and a new research

framework”, The International Journal of Management Science, Vol.

30, pp. 451-478

Bailey, J.E., and Pearson, S.W. (1983), “Development of a tool for

measuring and analyzing computer user satisfaction”, Management

Science, Vol. 29, No. 5, pp. 530-545

Bakker A.R., (2007), “The need to know the history of the use of digital

patient data, in particular the HER”, International Journal of Medical


Bokhari, R.H. (2005), “The relationship between system usage and user

satisfaction: a meta-analysis”, The Journal of Enterprise

Information Management, Vol. 18, No. 2, pp. 211-234

-57-

REFERENCES

Bollen, K.A. (1989), Structural equations with latent variables, NY: Wiley

Bollen, K.A. and Long, J.S. (1993), Introduction: Testing structural

equation models, Newbury Park, CA, Sage Publications Inc.

Burkle, T., Ammenwerth, E., Prokosch, H.U. and Dudeck, J. (2001),

“Evaluation of clinical information systems: What can be evaluated

and what cannot”, Journal of Evaluation in Clinical Practice, Vol. 7,

No. 4, pp. 373-385

Chang, M.K. (1998), “Predicting unethical behavior: a comparison of the

theory of reasoned action and the theory of planned behavior”,

Journal of Business Ethics, Vol. 17, pp. 1825-1834

Chin, W.W. and Lee, M.K.O. (2000), “A proposed model and measurement

instrument for the formation of IS satisfaction: The case of end-

user computing satisfaction”, 21st International Conference of

Information Systems, Brisbane, Australia, pp. 553-563

Davis, F.D. (1989), “Perceived Usefulness, Perceived Ease of Use, and

User Acceptance of Information Technology”, MIS Quarterly, Vol.

13, No. 3, pp. 319-340

Davis, S.A., and Bostrom, R.P. (1993), “Training End Users: An

Experimental Investigation of the Roles of the Computer Interface

and Training Methods”, MIS Quarterly, Vol. 17, No. 1, pp. 61-85

DeLone, W.H. and McLean, E.R. (2003), “The DeLone and McLean Model

of Information Systems Success: A Ten-Year Update”, Journal of

Management Information Systems, Vol. 19, No. 4, pp. 9-30

DeLone, W.H., and McLean, E.R. (1992), “Information systems success:

the request for the dependent variable”, Information Systems

Research, Vol. 3, No. 1, pp. 60-95

Deng, X., Doll, W.J., Al-Gahtani, S.S., Larsen, T.J., Pearson, L.M. and

Raghunathan, T.S. (2008), “A cross-cultural analysis of the end-

user computing satisfaction instrument: A multi-group invariance

analysis”, Information & Management, Vol. 45, pp. 211-220

Doll, W.J., and Torkzadeh, G. (1988), “The Measurement of End-User

Computing Satisfaction”, MIS Quarterly, Vol. 12, No. 2, pp. 259-274

-58-

REFERENCES

Doll, W.J., and Torkzadeh, G. (1991), “The Measurement of End-User

Computing Satisfaction: Theoretical and methodological issues”,

MIS Quarterly, Vol. 15, No. 1, pp. 5-10

Edwards P.J., Moloney K.P., Jasco J.A. and Sainfort F., (2008), “Evaluating

usability of a commercial electronic health record: A case study”,

Int. J. Human-Computer Studies, Vol. 66, pp. 718-728

Etenazi-Amoli, J., and Farhoomand, A.F. (1996), “A structural model of

end user computing satisfaction and user performance”,

Information & Management, Vol. 30, pp. 65-73

Gallapher, C.A. (1974), “Perceptions of the value of a management

information system”, Academy of Management Journal, Vol. 17, No.

1, pp. 46-55

Guimaraes, T., Igbaria, M., and Lu, M-te (1992), “The determinants of

DSS Success: An Integrated Model”, Decisions Sciences, Vol. 23,

pp. 409-430

Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (1992),

Multivariate data analysis, with readings, New York: Maxwell

Macmillian International

Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (1995),

Multivariate Data Analysis, 4th ed., Englewood Cliffs, NJ: Prentice-

Hall

Handel, D.A. and Hackman, J.L. (2008), “Implementing electronic health

records in the emergency department”, The Journal of Emergency

Medicine, (forthcoming)

Harrison, A.W. and Rainer, R.K. (1996), “A General Measure of User

Computing Satisfaction”, Computers in Human Behavior, Vol. 12,

No. 1, pp. 79-92

Hayrinen, K., Saranto, K. and Nykanen, P. (2008), “Definition, structure,

content, use and impacts of electronic health records: A review of

the research literature”, International Journal of Medical


-59-

REFERENCES

Igbaria, M. (1990), “End-User Computing Effectiveness: A Structural

Equation Model”, International Journal of Management Science, Vol.

18, No. 6, pp. 637-652

Igbaria, M, Guimaraes, T., and Davis, G.B. (1995), “Testing the

determinants of microcomputer usage via a structural equation

model”, Journal of Management Information Systems, Vol. 11, pp.

87-114

Igbaria, M, and Nachman, S.A. (1990), “Correlates of user satisfaction

with end user computing”, Information & Management, Vol. 19, pp.

73-82

Ives, B., Olson, M.H. and Baroudi, J.J. (1983), “The Measurement of User

Information Satisfaction”, Communications of the ACM, Vol. 26, No.

10, pp. 785-793

Jenkins, A.M., and Ricketts, J.A. (1979), “Development of an instrument

to measure user information satisfaction with management

information systems”, Unpublished working paper, Indiana

University, Bloomington

Jha A.K., Doolan D., Grandt D., Scott T. and Bates D.W., (2008), “The use

of health information technology in seven nations”, International

Journal of Medical Informatics, (forthcoming)

Kluge, E.H.W., (2007), “Secure e-Health: Managing risks to patient health

data”, International Journal of Medical Informatics, Vol. 76, pp.

402-406

Lampsas, P., Vidalis, I., Papanikolaou, C., and Vagelatos, A. (2002),

“Implementation and Integration of Regional Health Care Data

Networks in the Hellenic National Health Service”, Journal of

Medical Internet Research, Vol. 4, No. 3

Larcker, D.F., and Lessig, V.P. (1980), “Perceived usefulness of

information: A psychometric examination”, Decision Science, Vol.

11, No. 1, pp. 121-134

Lawrence, M., and Low, G. (1993), “Exploring Individual User Satisfaction

Within User-Led Development”, MIS Quarterly, Vol. 17, pp. 195-208

-60-

REFERENCES

Mahmood, M.A., Burn, J.M., Gemoets, L.A. and Jacquez, C. (2000),

“Variables affecting information technology end-user satisfaction: a

meta-analysis of the empirical literature”, Int. J. Human-Computer

Studies, Vol. 52, pp. 751-771

Malhotra, N. (1999), Marketing Research: An Applied Operation, Prentice

Hall

Mosley, I.T. (2001), “Computer Management Information Systems and

Computer Production Skills Needed by Industrial Technology

Graduates as Perceived by Universities and Companies”, Journal of

Industrial Technology, Vol. 17, No. 1, pp. 1-6

Nahm, E.S., Vaydia, V., Ho, D., Scharf, B. and Seagull, J. (2007),

“Outcomes assessment of clinical information system

implementation: A practical guide”, Nursing Outlook, Vol. 55, No. 6,

pp. 282-288

Palvia, P.C. (1996), “A model and instrument for measuring small

business user satisfaction with information technology”, Information

& Management, Vol. 31, pp. 151-163

Pitt, L.F., Watson, R.T., and Kavan, C.B. (1995), “Service Quality: A

Measure of Information Systems Effectiveness”, MIS Quarterly, Vol.

19, No. 2, pp. 173-188

Ribiere, V., LaSalle, A.J., Khorramshahgol, R. and Gousty, Y. (1999),

“Hospital Information Systems Quality: A Customer Satisfaction

Assessment Tool”, 32nd Hawaii International Conference on System

Sciences, pp. 1-9

Salmela, H. and Turunen, P. (1997), “Evaluation of information systems in

health care: framework and its application”, Proceedings of the

Fourth European Conference on the Evaluation of Information

Technology, The Delft University Press, pp. 111-121

Smith, T.D. and McMillian, B.F. (2001), “A primer of model fit Indices in

Structural Equation Model”, Paper presented at the Annual Meeting

of the Southwest Educational Research Association, (New Orleans,

LA, February 1-3)

-61-

REFERENCES

Straub, D.W. (1989), “Validating instruments in MIS research”, MIS

Quarterly, Vol. 13, pp. 147-165

Straub, D.W., Boudreau, M.C. and Gefen, D. (2004), “Validation

Guidelines for IS Positivist Research”, Communications of AIS, Vol.

13, pp. 380-427

Thompson, A.M. (1999), “An evaluation of the Selection Process of

Hospital Information Systems”, Journal of Medical Systems, Vol. 14,

No. 5, pp. 245-282

Thompson, R.L., Higgins, C.A., and Howell, J.M. (1991), “Personal

Computing: Toward a Conceptual Model of Utilization”, MIS

Quarterly, Vol. 15, No. 1, pp. 125-143

Torkzadeh, G., and Lee, J. (2003), “Measures of perceived end-user

computing skills”, Information & Management, Vol. 40, pp. 607-615

Uslu A.M. and Stausberg J., (2008), “Value of electronic patient record: An

analysis of the literature”, Journals of Biomedical Informatics, Vol.

41, pp. 675-682

Vaggelatos, A., and Sarivougioukas, I. (2002), Technical Report No.

2002/01/02

Venkatesh, V., Morris, M.G., Davis, G.B. and Davis, F.D. (2003), “User

Acceptance of Information Technology: Toward a Unified View”, MIS

Quarterly, Vol. 27, No. 3, pp. 425-478

Xiao, L. and Dasgupta, S. (2002), “Measurement of user satisfaction with

web-based information systems: an empirical study”, Eighth

Americas Conference on Information System, pp. 1149-1155

Yusof, M.M., Papazafeiropoulou, A., Paul, R.J., and Stergioulas, L.K.

(2008), “Investigating evaluation frameworks for health information

systems”, International Journal of Medical Informatics, Vol. 77, pp.

377-385

Zviran, M. (1990), “Defining the Application Portfolio for an Integrated

Hospital Information System: A Tutorial”, Journal of Medical

Systems, Vol. 14, No. 1 / 2, pp. 31-41

-62-

APPENDIX A: QUESTIONNAIRE

Appendix A: Questionnaire

QUESTIONNAIRE Directions

This questionnaire contains two parts. Part I asks some general information about you. Part II asks

you to rate your agreement or disagreement on a variety of issues. Please rate the statements based

upon the job in relation to the IS of the Hospital.

Thank you very much for your opinion and time. Please be assured this information will remain strictly

confidential and that the questionnaire is anonymous.

Part I 1. Select your sex Male Female 2. Age: ____ 3. Select your education level High School Master’s Degree Bachelor’s Degree Doctoral Degree Else: ________ 4. Position at the hospital Nursing Staff

Doctors

Administrative staff 5. Number of years in the present job: ......year(s) ......month(s)

6. Number of years using the IS: ......year(s) ......month(s)

Part II A) User Background

1 Strongly Disagree

2 Disagree

Somewhat

3 Neutral

4 Agree

Somewhat

5 Strongly Agree

A1.1 The level of my computer experience with packages developed for personal computer (excel, word etc) is satisfactory

1 2 3 4 5

A1.2 The level of my computer experience with information systems is satisfactory

1 2 3 4 5

A1.3 The nature of my previous encounters with information systems was very satisfactory

1 2 3 4 5

A2.1 Adequacy of computer training was provided to me

1 2 3 4 5

A2.2 The training was provided to me by vendors or outside consultants

1 2 3 4 5

A2.3 The training was provided to me by the IS department

1 2 3 4 5

A2.4 Self study was the way of accomplishing my training

1 2 3 4 5

A3.1 Loading, interacting, and executing software packages is easy for me

1 2 3 4 5

A3.2 Formatting and producing useful reports is easy for me

1 2 3 4 5

A3.3 Entering data, formulas, and calculations is easy for me

1 2 3 4 5

-63-


B) System Quality 1

Strongly Disagree

2 Disagree

Somewhat

3 Neutral

4 Agree

Somewhat

5 Strongly Agree

B1.1 The system is friendly to the user 1 2 3 4 5

B1.2 The system is easy to use 1 2 3 4 5

B1.3 I believe that the it is easy to get the system to do what I want to do 1 2 3 4 5

B1.4 My interaction with the system is clear and understandable

1 2 3 4 5

B1.5 Learning to operate the system is easy for me

1 2 3 4 5

B2.1 The operational speed of the system is satisfactory 1 2 3 4 5

B2.2 The system operates at a satisfactory pace

1 2 3 4 5

B2.3 The systems runs very quickly 1 2 3 4 5

B2.4 The speed of the system is satisfactory 1 2 3 4 5

B3.1 I believe that the screen layout of the system are well designed 1 2 3 4 5

B3.2 I believe that the screen colours of the system are pleasant 1 2 3 4 5

B3.3 I believe that the volume of output per screen is suitable

1 2 3 4 5

B3.4 I generally believe that the screen interface is easy to customize 1 2 3 4 5

B4.1 The function of the system to correct a mistake is simple

1 2 3 4 5

B4.2 The system has the ability to correct a incorrect information in a fast way

1 2 3 4 5

B4.3 The methods and policies governing correction and rerun of system outputs that are incorrect are complete

1 2 3 4 5

C) Information Quality C1.1 I believe that the system provides the

precise information I need 1 2 3 4 5

C1.2 I believe that the information content meet my needs 1 2 3 4 5

C1.3 I believe that the systems provides reports that seems to be just about exactly to what I need

1 2 3 4 5

C1.4 The information provided by the system fit my needs 1 2 3 4 5

C1.5 The system provides me the right amount of information for my needs 1 2 3 4 5

C2.1 I believe that the system is accurate 1 2 3 4 5

C2.2 The accuracy of the system is satisfactory 1 2 3 4 5

C2.3 I believe that the system provides accurate information 1 2 3 4 5

C2.4 I believe that the system provides reliable information 1 2 3 4 5

C3.1 I believe that the output is presented in a useful format 1 2 3 4 5

C3.2 I believe that the information provided is clear 1 2 3 4 5

C3.3 The layout of the output is satisfactory 1 2 3 4 5

C3.4 The format of the output is satisfactory 1 2 3 4 5

C3.5 I believe that the information is presented to me with a satisfactory way 1 2 3 4 5

-64-


1

Strongly Disagree

2 Disagree

Somewhat

3 Neutral

4 Agree

Somewhat

5 Strongly Agree

C4.1 I believe that I get the information I need in time 1 2 3 4 5

C4.2 The system provides up-to-date information 1 2 3 4 5

C4.3 I believe that the system provides me with the information in a timely manner 1 2 3 4 5

C4.4 The system provides information that it is to old to be useful 1 2 3 4 5

C4.5 I believe that the system provide some information that it is too late for my needs

1 2 3 4 5

C5.1 The system secures data against physical loss or damage 1 2 3 4 5

C5.2 The system secures data against unauthorized alteration 1 2 3 4 5

C5.3 Generally, the safeguarding of the system from unauthorized access is satisfactory

1 2 3 4 5

D) Service Quality D1.1 I believe that the IS department provides

satisfactory support to all users of the system

1 2 3 4 5

D1.2 I believe that staffs suggestions for future enhancements of the system are responded by IS department cooperatively

1 2 3 4 5

D1.3 I believe that there is availability of IS staff for consultation 1 2 3 4 5

D2.1 I believe that the amount of support provided by the vendor is satisfactory 1 2 3 4 5

D2.2 The vendor is available for assistance with software difficulties 1 2 3 4 5

D2.3 The vendor is available for assistance with hardware difficulties 1 2 3 4 5

Overall Satisfaction Rate your satisfaction with the use of the system

1 2 3 4 5 very dissatisfied neutral very satisfied

Using the system makes me feel:

1 2 3 4 5 extremely dissatisfied

neutral extremely satisfied

All things considered, I am:

1 2 3 4 5 very dissatisfied neutral very satisfied

with using the system.

-65-

APPENDIX B: STATISTICAL ANALYSIS

Appendix B: Statistical Analysis

Factor Analysis

KMO and Bartlett's Test

,704

98,9663

,000

Kaiser-Meyer-Olkin Measure of SamplingAdequacy.

Approx. Chi-SquaredfSig.

Bartlett's Test ofSphericity

Anti-image Matrices

,531 -,254 -,133-,254 ,480 -,207-,133 -,207 ,608,701a -,504 -,235

-,504 ,665a -,383-,235 -,383 ,761a

A1.1A1.2A1.3A1.1A1.2A1.3

Anti-image Covariance

Anti-image Correlation

A1.1 A1.2 A1.3

Measures of Sampling Adequacy(MSA)a.

Communalities

1,000 ,7351,000 ,7831,000 ,680

A1.1A1.2A1.3

Initial Extraction

Extraction Method: Principal Component Analysis.

Total Variance Explained

2,198 73,279 73,279 2,198 73,279 73,279,473 15,767 89,046,329 10,954 100,000

Component123

Total % of Variance Cumulative % Total % of Variance Cumulative %Initial Eigenvalues Extraction Sums of Squared Loadings


Component Matrixa

,858,885,825

A1.1A1.2A1.3

1

Component

Extraction Method: Principal Component Analysis.1 components extracted.a.

Rotated Component Matrixa

Only one component was extracted.The solution cannot be rotated.

a.

-66-


RELIABILITY /VARIABLES=A1.1 A1.2 A1.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.

Reliability Scale: ALL VARIABLES

Case Processing Summary

95 95,05 5,0

100 100,0

ValidExcludeda

Total

CasesN %

Listwise deletion based on allvariables in the procedure.

a.

Reliability Statistics

,806 3

Cronbach'sAlpha N of Items

Factor Analysis


,500

41,5951

,000




Anti-image Matrices

,638 -,384-,384 ,638,500a -,602

-,602 ,500a

A2.2A2.3A2.2A2.3



A2.2 A2.3


Communalities

1,000 ,8011,000 ,801

A2.2A2.3

Initial Extraction


-67-



1,602 80,090 80,090 1,602 80,090 80,090,398 19,910 100,000

Component12



Component Matrixa

,895,895

A2.2A2.3

1

Component




a.

RELIABILITY /VARIABLES=A2.2 A2.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



95 95,05 5,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,748 2


-68-


Factor Analysis


,692

92,2733

,000




Anti-image Matrices

,576 -,262 -,120-,262 ,502 -,227-,120 -,227 ,633,699a -,488 -,198

-,488 ,649a -,403-,198 -,403 ,744a

A3.1A3.2A3.3A3.1A3.2A3.3



A3.1 A3.2 A3.3


Communalities

1,000 ,7051,000 ,7761,000 ,662

A3.1A3.2A3.3

Initial Extraction



2,143 71,430 71,430 2,143 71,430 71,430,510 17,009 88,439,347 11,561 100,000

Component123



Component Matrixa

,840,881,813

A3.1A3.2A3.3

1

Component




a.

-69-


RELIABILITY /VARIABLES=A3.1 A3.2 A3.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



98 98,02 2,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,785 3


-70-


Factor Analysis


,805

243,05610

,000




Anti-image Matrices

,473 -,191 -,128 ,092 ,009-,191 ,296 -,046 -,174 -,066-,128 -,046 ,424 -,106 -,149,092 -,174 -,106 ,430 -,082,009 -,066 -,149 -,082 ,562,753a -,512 -,286 ,204 ,018

-,512 ,763a -,129 -,489 -,161-,286 -,129 ,859a -,248 -,305,204 -,489 -,248 ,787a -,166,018 -,161 -,305 -,166 ,890a

B1.1B1.2B1.3B1.4B1.5B1.1B1.2B1.3B1.4B1.5



B1.1 B1.2 B1.3 B1.4 B1.5


Communalities

1,000 ,5761,000 ,8031,000 ,7311,000 ,6551,000 ,588

B1.1B1.2B1.3B1.4B1.5

Initial Extraction



3,353 67,052 67,052 3,353 67,052 67,052,636 12,714 79,766,471 9,413 89,180,343 6,855 96,034,198 3,966 100,000

Component12345



-71-


Component Matrixa

,759,896,855,810,767

B1.1B1.2B1.3B1.4B1.5

1

Component




a.

RELIABILITY /VARIABLES=B1.1 B1.2 B1.3 B1.4 B1.5 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



94 94,06 6,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,874 5


-72-


Factor Analysis


,714

106,3443

,000




Anti-image Matrices

,573 -,201 -,144-,201 ,467 -,241-,144 -,241 ,512,762a -,389 -,265

-,389 ,679a -,493-,265 -,493 ,712a

B2.1B2.2B2.3B2.1B2.2B2.3



B2.1 B2.2 B2.3


Communalities

1,000 ,7061,000 ,7871,000 ,748

B2.1B2.2B2.3

Initial Extraction



2,241 74,686 74,686 2,241 74,686 74,686,438 14,600 89,286,321 10,714 100,000

Component123



Component Matrixa

,840,887,865

B2.1B2.2B2.3

1

Component




a.

-73-


RELIABILITY /VARIABLES=B2.1 B2.2 B2.3 B2.4 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



94 94,06 6,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,634 4


Factor Analysis


,853

259,2016

,000




Anti-image Matrices

,358 -,123 -,091 -,085-,123 ,362 -,074 -,098-,091 -,074 ,335 -,137-,085 -,098 -,137 ,314,864a -,343 -,262 -,252

-,343 ,865a -,213 -,290-,262 -,213 ,847a -,423-,252 -,290 -,423 ,836a

B3.1B3.2B3.3B3.4B3.1B3.2B3.3B3.4



B3.1 B3.2 B3.3 B3.4


Communalities

1,000 ,7901,000 ,7871,000 ,8031,000 ,819

B3.1B3.2B3.3B3.4

Initial Extraction


-74-



3,199 79,980 79,980 3,199 79,980 79,980,306 7,646 87,625,269 6,737 94,362,226 5,638 100,000

Component1234



Component Matrixa

,889,887,896,905

B3.1B3.2B3.3B3.4

1

Component




a.

RELIABILITY /VARIABLES=B3.1 B3.2 B3.3 B3.4 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



95 95,05 5,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,916 4


-75-


Factor Analysis


,754

217,5363

,000




Anti-image Matrices

,275 -,144 -,082-,144 ,230 -,129-,082 -,129 ,316,761a -,574 -,278

-,574 ,709a -,479-,278 -,479 ,803a

B4.1B4.2B4.3B4.1B4.2B4.3



B4.1 B4.2 B4.3


Communalities

1,000 ,8671,000 ,8971,000 ,847

B4.1B4.2B4.3

Initial Extraction



2,611 87,037 87,037 2,611 87,037 87,037,235 7,838 94,875,154 5,125 100,000

Component123



Component Matrixa

,931,947,920

B4.1B4.2B4.3

1

Component




a.

-76-


RELIABILITY /VARIABLES=B4.1 B4.2 B4.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



95 95,05 5,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,925 3


-77-


Factor Analysis


,847

478,41210

,000




Anti-image Matrices

,325 -,110 -,010 ,013 -,049-,110 ,170 -,083 -,067 ,027-,010 -,083 ,257 -,012 -,056,013 -,067 -,012 ,141 -,104

-,049 ,027 -,056 -,104 ,181,893a -,466 -,036 ,062 -,203

-,466 ,819a -,397 -,436 ,155-,036 -,397 ,914a -,065 -,260,062 -,436 -,065 ,811a -,654

-,203 ,155 -,260 -,654 ,819a

C1.1C1.2C1.3C1.4C1.5C1.1C1.2C1.3C1.4C1.5



C1.1 C1.2 C1.3 C1.4 C1.5


Communalities

1,000 ,7501,000 ,8731,000 ,8211,000 ,8801,000 ,838

C1.1C1.2C1.3C1.4C1.5

Initial Extraction



4,162 83,246 83,246 4,162 83,246 83,246,346 6,919 90,164,244 4,871 95,036,167 3,331 98,367,082 1,633 100,000

Component12345



-78-


Component Matrixa

,866,934,906,938,916

C1.1C1.2C1.3C1.4C1.5

1

Component




a.

RELIABILITY /VARIABLES=C1.1 C1.2 C1.3 C1.4 C1.5 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA. Reliability Scale: ALL VARIABLES


93 93,07 7,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,949 5


-79-


Factor Analysis


,817

358,4246

,000




Anti-image Matrices

,252 -,113 -,014 ,023-,113 ,131 -,079 -,064-,014 -,079 ,211 -,093,023 -,064 -,093 ,331,815a -,624 -,059 ,081

-,624 ,750a -,477 -,305-,059 -,477 ,847a -,352,081 -,305 -,352 ,883a

C2.1C2.2C2.3C2.4C2.1C2.2C2.3C2.4



C2.1 C2.2 C2.3 C2.4


Communalities

1,000 ,8011,000 ,9241,000 ,8721,000 ,775

C2.1C2.2C2.3C2.4

Initial Extraction



3,372 84,305 84,305 3,372 84,305 84,305,352 8,806 93,111,182 4,560 97,671,093 2,329 100,000

Component1234



Component Matrixa

,895,961,934,880

C2.1C2.2C2.3C2.4

1

Component


-80-




a.

RELIABILITY /VARIABLES=C2.1 C2.2 C2.3 C2.4 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



95 95,05 5,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,937 4


-81-


Factor Analysis


,870

563,76010

,000




Anti-image Matrices

,216 -,092 -,029 -,012 -,025-,092 ,172 -,071 -,039 ,017-,029 -,071 ,174 -,006 -,059-,012 -,039 -,006 ,163 -,099-,025 ,017 -,059 -,099 ,154,908a -,475 -,152 -,066 -,135

-,475 ,858a -,411 -,233 ,104-,152 -,411 ,897a -,035 -,361-,066 -,233 -,035 ,860a -,626-,135 ,104 -,361 -,626 ,834a

C3.1C3.2C3.3C3.4C3.5C3.1C3.2C3.3C3.4C3.5



C3.1 C3.2 C3.3 C3.4 C3.5


Communalities

1,000 ,8431,000 ,8671,000 ,8811,000 ,8661,000 ,865

C3.1C3.2C3.3C3.4C3.5

Initial Extraction



4,322 86,448 86,448 4,322 86,448 86,448,292 5,847 92,295,165 3,299 95,594,134 2,689 98,282,086 1,718 100,000

Component12345



-82-


Component Matrixa

,918,931,939,931,930

C3.1C3.2C3.3C3.4C3.5

1

Component




a.

RELIABILITY /VARIABLES=C3.1 C3.2 C3.3 C3.4 C3.5 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



97 97,03 3,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,960 5


-83-


Factor Analysis


,695

249,90310

,000




Anti-image Matrices

,264 -,144 -,049 -,024 -,027-,144 ,201 -,131 ,005 ,032-,049 -,131 ,314 ,016 -,056-,024 ,005 ,016 ,638 -,370-,027 ,032 -,056 -,370 ,620,758a -,624 -,170 -,058 -,066

-,624 ,678a -,522 ,014 ,090-,170 -,522 ,801a ,037 -,127-,058 ,014 ,037 ,531a -,587-,066 ,090 -,127 -,587 ,542a

C4.1C4.2C4.3C4.4C4.5C4.1C4.2C4.3C4.4C4.5



C4.1 C4.2 C4.3 C4.4 C4.5


Communalities

1,000 ,8631,000 ,9121,000 ,8411,000 ,8031,000 ,798

C4.1C4.2C4.3C4.4C4.5

Initial Extraction



Initial Eigenvalues Extraction Sums of Squared Loadings

Rotation Sums of Squared Loadings

Compo nent

Total % of Variance

Cumulative %

Total % of Variance

Cumulative %

Total % of Variance

Cumulative %

1 2 3 4 5

2,714 1,504 ,402 ,248 ,132

54,276 30,078

8,038 4,968 2,640

54,276 84,354 92,392 97,360

100,000

2,714 1,504

54,276 30,078

54,276 84,354

2,609 1,609

52,178 32,176

52,178 84,354


-84-


Component Matrixa

,913 -,171,921 -,251,899 -,181,300 ,845,364 ,816

C4.1C4.2C4.3C4.4C4.5

1 2Component



,923 ,106,955 ,031,912 ,091,038 ,896,108 ,887

C4.1C4.2C4.3C4.4C4.5

1 2Component

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

Rotation converged in 3 iterations.a.

Component Transformation Matrix

,956 ,294-,294 ,956

Component12

1 2

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

RELIABILITY /VARIABLES=C4.1 C4.2 C4.3 C4.4 C4.5 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



89 89,011 11,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,740 5


-85-


Factor Analysis


,706

203,8133

,000




Anti-image Matrices

,482 -,106 -,051-,106 ,206 -,165-,051 -,165 ,227,875a -,338 -,153

-,338 ,647a -,763-,153 -,763 ,669a

C5.1C5.2C5.3C5.1C5.2C5.3



C5.1 C5.2 C5.3


Communalities

1,000 ,7441,000 ,8961,000 ,872

C5.1C5.2C5.3

Initial Extraction



2,512 83,739 83,739 2,512 83,739 83,739,366 12,190 95,930,122 4,070 100,000

Component123



Component Matrixa

,863,947,934

C5.1C5.2C5.3

1

Component




a.

-86-


RELIABILITY /VARIABLES=C5.1 C5.2 C5.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



96 96,04 4,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,901 3


Factor Analysis


,737

192,0903

,000




Anti-image Matrices

,292 -,181 -,099-,181 ,275 -,125-,099 -,125 ,428,710a -,638 -,279

-,638 ,693a -,363-,279 -,363 ,833a

D1.1D1.2D1.3D1.1D1.2D1.3



D1.1 D1.2 D1.3


-87-


Communalities

1,000 ,8571,000 ,8721,000 ,786

D1.1D1.2D1.3

Initial Extraction



2,515 83,825 83,825 2,515 83,825 83,825,313 10,424 94,249,173 5,751 100,000

Component123



Component Matrixa

,926,934,886

D1.1D1.2D1.3

1

Component




a.

RELIABILITY /VARIABLES=D1.1 D1.2 D1.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA. Reliability Scale: ALL VARIABLES


99 99,01 1,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,903 3


-88-


Factor Analysis


,722

140,9653

,000




Anti-image Matrices

,378 -,226 -,155-,226 ,396 -,128-,155 -,128 ,529,684a -,584 -,346

-,584 ,698a -,279-,346 -,279 ,807a

D2.1D2.2D2.3D2.1D2.2D2.3



D2.1 D2.2 D2.3


Communalities

1,000 ,8251,000 ,8091,000 ,728

D2.1D2.2D2.3

Initial Extraction



2,361 78,710 78,710 2,361 78,710 78,710,395 13,162 91,872,244 8,128 100,000

Component123



Component Matrixa

,908,900,853

D2.1D2.2D2.3

1

Component




a.

-89-


RELIABILITY /VARIABLES=D2.1 D2.2 D2.3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



98 98,02 2,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,865 3


-90-


Factor Analysis


,754

279,8163

,000




Anti-image Matrices

,198 -,115 -,050-,115 ,159 -,099-,050 -,099 ,262,755a -,650 -,220

-,650 ,697a -,488-,220 -,488 ,828a

SAT1SAT2SAT3SAT1SAT2SAT3



SAT1 SAT2 SAT3


Communalities

1,000 ,9001,000 ,9291,000 ,873

SAT1SAT2SAT3

Initial Extraction



2,702 90,051 90,051 2,702 90,051 90,051,195 6,503 96,553,103 3,447 100,000

Component123



Component Matrixa

,949,964,934

SAT1SAT2SAT3

1

Component




a.

-91-


RELIABILITY /VARIABLES=SAT1 SAT2 SAT3 /SCALE('ALL VARIABLES') ALL/MODEL=ALPHA.



99 99,01 1,0

100 100,0

ValidExcludeda

Total

CasesN %


a.


,944 3


-92-

Date post:	16-Oct-2021
Category:	Documents
Upload:	others
View:	6 times
Download:	0 times

“Hospital Information System Evaluation”

Documents