90
Master Thesis
Computer Science
Thesis no: MCS:2011:21
Sept. 2011
A Comparative Evaluation of Usabilityfor the iPhone and iPad
Muhammad Azam
Luqman Ahmad
School of Computing
Blekinge Institute of Technology
SE-371 79 Karlskrona
Sweden
1
This thesis is submitted to the School of Computing at Blekinge Institute of Technology in partial
ful�llment of the requirements for the degree of Master of Science in Computer Science. The
thesis is equivalent to 20 weeks of full time studies.
Contact Information:
Author(s):
Muhammad Azam
Address: Gyllenstjärnas Väg 18, 20
Post code 37140 Karlskrona,
Sweden
Email: [email protected]
Luqman Ahmad
Address: Gyllenstjärnas Väg 18, 20
Post code 37140 Karlskrona,
Sweden
E-mail: [email protected]
University advisor:
Dr. Veronica Sundstedt
School of Computing
School of Computing
Blekinge Institute of Technology Internet : www.bth.se/com
SE-371 79 Karlskrona Phone : +46 455 38 50 00
Sweden Fax : +46 455 38 50 57
2
ACKNOWLEDGMENTS
In the name of Allah, the Most Bene�cent, the Most Merciful.
First and foremost we would like to thank Allah the Almighty, Who blessed us courage
and devotion for this study.
We heartily thank our supervisor Dr. Veronica Sundstedt, who encouraged us and
gave guidance throughout this research work. Her constructive suggestions, directions
and invaluable advice made us capable to complete this thesis. We would also like to
express our appreciation to those who participated and dedicated their valuable time for
these studies, without their participation these studies would not have been feasible.
Last but not least, we express our gratitude to our parents and family members, for
providing us untiring support and prayers during this research work.
3
Abstract
Many everyday systems and products seem to be designed with little regard to
usability. This leads to the frustration, wasted time and errors. So the usability of
the product is important for its survival in the market.
In many previous studies the usability evaluation of the iPhone and iPad carried
out individually and very little work has been done on the comparative usability
evaluation. However, there was not any study conducted on the comparative us-
ability evaluation and measuring the performance of the iPhone versus iPad in a
controlled environment.
In this research work, the authors performed the comparative usability evaluation
and measured the performances of the iPhone and iPad on the selected applications
by considering the young users as well as the elderly users. Another objective of this
study is to identify the usability issues in performances of the iPhone and iPad.
A survey and experiment techniques were used to achieve the de�ned objectives.
The survey questionnaire consisted of 42 statements that presented the di�erent
usability aspects. The objectives of the survey study were to validate the identi�ed
issues from the literature study, identify new issues and measure the signi�cant
di�erence in user opinions for the iPhone and iPad. However, the experiment studies
helped to measure the performance signi�cances between the devices against the
three user groups (novice user, experienced user, elderly user) and among the groups
over the devices. Further, objective was to measure the satisfaction level of the
participated users against the iPhone and iPad.
The experiment was performed in a controlled environment. Total six tasks (two
tasks per application) were de�ned and each participant performed the same task on
both devices. Generally the authors found that the participants performed better
on the iPad with lower error rates as compare to the iPhone.
Keywords: Usability, Usability Evaluation, Touch Screen, Smart-phone, iPhone,
iPad, performance.
4
Contents
1 INTRODUCTION 10
1.1 Aim and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Study Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 BACKGROUND 14
2.1 What is Usability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Usability Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Performance Measuring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4 Usability Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 Touch Screen and Smart Phones . . . . . . . . . . . . . . . . . . . . . . . . 17
2.6 The iPhone and the iPad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.7 Features of the iPhone and iPad . . . . . . . . . . . . . . . . . . . . . . . 18
2.7.1 Multi-touching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.7.2 Operating System (iOS 4) . . . . . . . . . . . . . . . . . . . . . . . 19
2.7.3 Applications (Apps) Store . . . . . . . . . . . . . . . . . . . . . . . 19
2.8 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.9 Issues in the iPhone and iPad . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.9.1 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.9.2 A�ordance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.9.3 Small Search Box and Missing Information . . . . . . . . . . . . . . 21
2.9.4 Invisible Control and Missing Instructions . . . . . . . . . . . . . . 22
3 RESEARCH METHODOLOGY 23
3.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Survey Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3.1 Participant Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3.2 Variables Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3.3 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3.4 Task De�nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3.5 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.6 Experimental Material . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.7 Post Test Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.8 Pilot Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Evaluation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4.1 Mann Whitney Test . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5
3.4.2 Normality Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.3 Analysis of Variances (ANOVA) . . . . . . . . . . . . . . . . . . . . 34
3.4.4 Unpaired T-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.5 Paired T-test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.6 Wilcoxon Matched Paired Signed Ranks Test . . . . . . . . . . . . 35
3.4.7 Error Bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.5 Validity Threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.5.1 Internal Validity Threats . . . . . . . . . . . . . . . . . . . . . . . . 36
3.5.2 External Validity Threats . . . . . . . . . . . . . . . . . . . . . . . 36
3.5.3 Constructive Validity Threats . . . . . . . . . . . . . . . . . . . . . 36
3.5.4 Statistical Conclusion Validity Threats . . . . . . . . . . . . . . . . 36
4 STATISTICAL ANALYSIS, RESULTS ANDDISCUSSIONS 37
4.1 Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.2 Demographics Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3 Usability and Preference Testing . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3.1 System Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.3.2 Touch Screen Gestures Results . . . . . . . . . . . . . . . . . . . . . 41
4.3.3 Keypad Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.4 Applications Results . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.5 Participant Comments . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.4 Measuring Usability Attributes . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4.1 Task Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4.2 Accuracy Measurement . . . . . . . . . . . . . . . . . . . . . . . . 61
4.4.3 Satisfaction Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.4.4 Participant Comments . . . . . . . . . . . . . . . . . . . . . . . . . 68
5 CONCLUSIONS AND FUTURE WORK 70
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6 APPENDICES 73
6.1 Appendix A Survey Questionnaire . . . . . . . . . . . . . . . . . . 73
6.2 Appendix B Survey Questionnaire Results . . . . . . . . . . . . . 76
6.3 Appendix C Survey Questionnaire p-values . . . . . . . . . . . . . 78
6.4 Appendix D Post Test Questionnaire . . . . . . . . . . . . . . . . 80
6.5 Appendix E Task Time on the iPhone and iPad . . . . . . . . . . 82
6
List of Figures
1 The Overall Study Process. . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Shackel's Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . . . 14
3 Nielsen's Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . . . 15
4 ISO 9241-11 Usability De�nition [21]. . . . . . . . . . . . . . . . . . . . . . 16
5 Di�erent Consistency Issues in SMS and Mobile Voip Applications. . . . . 20
6 A�ordance Issues in the Weather Applications of the iPhone and iPad. . . 21
7 Small Search Box Issues in the Applications of the iPhone and iPad. . . . . 21
8 Invisible Control and Missing Instructions Issues in the Twitter and Face-
book Application on the iPhone and iPad. . . . . . . . . . . . . . . . . . . 22
9 Presentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
10 System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
11 Touch Screen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
12 Keypad. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
13 Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
14 Participants Selection for the Experiment. . . . . . . . . . . . . . . . . . . 27
15 Laboratory Setup for User Trials. . . . . . . . . . . . . . . . . . . . . . . . 32
16 Participant Group Levels Comparison Across the Device. . . . . . . . . . . 34
17 Device Levels Comparison Across the Participant. . . . . . . . . . . . . . . 35
18 Percentage Graph Results of the Degree of Agreement Across Each System
Statement on the iPhone. St. Represents Strongly. . . . . . . . . . . . . . 41
19 Percentage Results of the Degree of Agreement Across Each System State-
ment of the iPad. St. Represents Strongly. . . . . . . . . . . . . . . . . . . 41
20 Graphs Show Percentage Results of the Degree of Agreement Across the
Each Statement of the Touch Screen Gestures on the iPhone and iPad
Respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
21 Graph Shows Percentage Results of the Degree of Agreement Across the
Each Statement of the Keypad on the iPhone. . . . . . . . . . . . . . . . 45
22 Graph Shows Percentage Results of the Degree of Agreement Across the
Each Statement of the Keypad on the iPad. . . . . . . . . . . . . . . . . . 46
23 Percentage Graph Results of the Degree of Agreement Across Each Appli-
cations Statement on the iPhone. . . . . . . . . . . . . . . . . . . . . . . . 49
24 Percentage Graph Results of the Degree of Agreement Across the Each
Statement of the Applications on the iPad. . . . . . . . . . . . . . . . . . 49
25 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task Facebook Login. . . . . . . . . . . . 54
26 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task Send a Message. . . . . . . . . . . . 55
7
27 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task Location Identi�cation. . . . . . . . 56
28 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task Location Close View. . . . . . . . . 58
29 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task New Note. . . . . . . . . . . . . . . 59
30 Device Levels Comparison Graph of the Performance Measurement of the
Participant Group Levels for the Task Evernote Logout. . . . . . . . . . . 60
31 Novice User Mistakes Results Graph on the iPhone and iPad. . . . . . . . 62
32 Experienced User Mistakes Results Graph on the iPhone and iPad. . . . . 63
33 Elderly User Mistakes Results Graph on the iPhone and iPad. . . . . . . . 63
34 The Graph Presents the Median Comparison of the Degree of Agreement
Against Each Question by Three User Groups on the iPhone. . . . . . . . . 67
35 The Graph Presents the Median Comparison of the Degree of Agreement
Against Each Question by Three User Groups on the iPad. . . . . . . . . . 68
8
List of Tables
1 iPhone and iPad Speci�cation Comparison. The Bold Values Show the
Di�erences in Speci�cation. . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Usability Issues on the iPhone and iPad. . . . . . . . . . . . . . . . . . . . 22
3 Advantages and Disadvantages of Qualitative Approach [52] . . . . . . . . . 23
4 Results of Surveyed Participants Comments Against the Di�erent Features
of the iPhone and iPad. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
5 Results from the ANOVA Tests for Estimating the Signi�cance Among the
Group Levels and Between the Device Levels Across Each Task. The p-
value Shown the Signi�cance Among the Mean Values of Each Participant
Group Against the Each Task. However, the Bold (p-values) Values Shown
the Signi�cant Di�erences Among the Groups and Between the Devices. . . 51
6 Participant Group Levels Comparison Results. The p-values Showed the
Di�erences Between the Participant Group Levels on the iPhone and iPad.
However, the Bold p-values Shown that there was a Signi�cant Di�erence
Between the Group Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
7 Device Levels Means Comparisons Results. The p-value Showed the Dif-
ferences Between the Means of the iPhone versus iPad Against each User
Group. However, the Bold p-values Shown the Signi�cant Di�erence. . . . 52
8 Means Results and Di�erences of Each Type of Mistake Made by Each
User Group on the iPhone and iPad. However, the Bold p-values Shown
the Signi�cant Di�erence. . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
9 Post Test Questionnaires Results in the Form of the Median, the Standard
Deviation and the Di�erences Between the Users Opinions Against Each
Question. However, the Bold p-values Shown the Signi�cant Di�erence. Q,
iP and id Represented the Question, iPhone and iPad Respectively. . . . . 64
10 Frequency and Percentage Results of the User Comments on the Di�erent
Aspects of Both the Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . 68
9
Chapter 1
1 INTRODUCTION
Human Computer Interaction (HCI) is concerned with the joint performance of tasks by
the human and machines [1]. It has become valuable due to the fact that it involves the
design, implementation and evaluation of computer interactive systems that human beings
use in di�erent contexts to accomplish their tasks. The main goal of HCI is to produce
computer systems which are easy to use, well functional, e�cient, e�ective, and that are in
accordance with the requirements of the user [2]. The HCI goal would be ensured with the
usability of the product. According to the Webster dictionary, the usability originate from
the word �usable� which means �capable of being used � or �convenient and practicable for
use�.
We are living in a mobile society [3]. New information and communication technologies
(ICTs) have become a part of people's everyday lives. The mobile technologies are devel-
oping very quickly [5] and the trend is changing from conventional desktop computing to
the mobile computing. It facilitates the users to access and use their services anywhere
anytime [4] e.g. doing trading, shopping, banking and bill payment etc [6]. So in the
last few years, the developers redirect their design e�ort from a desktop perspective to a
small screen approach [7]. Under the constraint of small size, the usability of the mobiles
devices and their applications is a key factor for the success of mobile computing [8].
The mobile phone technologies have an impact on all age groups in our society. Studies
illustrate that young people are more active users of technology [9, 10]. They are the
�rst users who adopt the new technology and develop an actual mobile communication
culture [10]. On the other hand, the elderly people are also very keen to use the new
mobile technology but because of the small size, invisibility of text and complexity of the
functions, they avoid to use it [11]. The elderly population is growing rapidly in almost
all the economically developed countries [12]. It is noticed that elderly people take more
interest in games as compared to younger but because of the complexities and di�culties
of the user interfaces they cannot take part more e�ectively [13]. Earlier studies presented
that most of the services are designed by considering the young population and mistakenly
the elderly people are also considered as a part of the same age group [14].
In the mobile phone multi-touch technology, the iPhone, iPod Touch, and iPad are
sophisticated devices, which provide advanced and sensitive user interactions based on
gestures [15]. The iPhone and iPad support millions of the applications that designed and
developed by the Apple and the third party. It has been considered that the Evernote
is used by millions of the people in the world to remember their important notes and
10
synchronize them with their PCs [38]. Meanwhile, in the social network sites the Facebook
is one of the leading applications having more than 500 million active users in the world
[69]. Only in the Sweden more than 4 million people are using the facebook [71]. Similarly,
the Google web application (search engine) is also a famous site having the large number
of users [70]. The simplicity and consistency of the devices motivated the authors to
conduct the comparative usability research studies of the iPhone and iPad.
In order to �nd out whether the iPhone and iPad are performing according to user
expectations or not, the authors performed the survey and empirical usability evaluation
studies. The authors selected three frequently used applications (Facebook, Google Mobile
Application, Evernote) and measured the usability performances of three user groups
(novice user, experienced user, elderly user) as well as the iPhone and iPad. The age
range for the young people is 22-34 and for the elderly people is 60-75.
1.1 Aim and Objectives
The aim of this research work is to evaluate and compare usability performance of the
iPhone and iPad on three selected applications considering the elderly people as well as
the young people. To achieve this goal the following objectives are de�ned:
� Identifying usability issues regarding the iPhone and iPad.
� Investigating usability performance di�erences of the same applications on the iPhone
and iPad.
� Investigating the performance di�erences between di�erent user groups using the
same applications and same devices.
1.2 Research Questions
1. What usability issues exist in the iPhone and iPad?
2. How does the performance di�er while using the same application on the iPhone
and iPad?
3. How does the performance di�er using the same application on the same device by
di�erent user groups?
11
1.3 Study Process
The study process presented a hierarchical �ow that was carried out during the the whole
research work.
Figure 1: The Overall Study Process.
1.4 Thesis Structure
Chapter 1 introduces the thesis and de�nes the research domain. In Section 1.1 aim and
objectives are given. In Section 1.2 research questions are given. In Section 1.3 the overall
research process is presented in the form of diagram .
Chapter 2 provides background knowledge about the usability. In Section 2.1 usability
and its di�erent attributes are discussed that de�ned by di�erent authors. Section 2.2
describes how to measure the usability. Section 2.3 describes the performance measuring.
Section 2.4 describes the usability evaluation. Section 2.5 describes the touch screen and
smart phones. Section 2.6 describes the iPhone and iPad. Section 2.7 is about the main
features of the iPhone and iPad. Section 2.8 presents the related work and Section 2.9
presents the usability issues concerning the iPhone and iPad.
Chapter 3 is about the research methodology adopted in the thesis. Section 3.1 presents
12
a literature review. Section 3.2 presents the survey design . Section 3.3 consists of the
experiment design . Section 3.4 presents the evaluation methods and Section 3.5 presents
the validity threats in research.
Chapter 4 presents the survey and experimental analysis results and discussion. Sec-
tion 4.1 presents the survey. Section 4.2 describes the demographics results of the survey.
Section 4.3 presents the usability and preference testing of the survey data. Section 4.4
describes the participants comments regarding the usability of the iPhone and iPad in the
survey questionnaire. Section 4.5 presents the measurement of usability attributes.
Chapter 5 contains conclusions and future work of the thesis. Section 5.1 presents the
conclusions of the research study and Section 5.2 presents the future work.
Chapter 6 contains the Appendices. In Section 6.1 Appendix A represents the survey
questionnaire. In the Section 6.2 Appendix B survey Questionnaire results are given. In
Section 6.3 Appendix C contains the p-values of the survey results. In the Section 6.4
Appendix D presents the post test questionnaire. Section 6.5 contains the Appendix E
that presents the experimental data i.e task time on both devices.
13
Chapter 2
2 BACKGROUND
2.1 What is Usability?
Usability in itself is not a new topic [17]. It is considered as a core term HCI [18] and has
become an important part in the design of products, software applications and devices.
However the user centered design and goal directed interaction design has increased us-
ability of products [17]. The role of usability is to make the system e�ective and easy to
use [19]. There are many de�nitions of the term usability. According to ISO 9241-11 the
term usability can be de�ned as �The extent to which a product can be used by speci�ed
users to achieve speci�ed goals with e�ectiveness, e�ciency and satisfaction in a speci�ed
context of use� [20].
Usability is the composition of the entities [21] e.g. e�ectiveness, e�ciency, learnability,
memorability etc. Some times these entities are de�ned as dimensions, components, scales
or factors of usability. All these forms have the same meaning, so the commonly used
term for them is usability attributes [21].
Shackel [19] is one of the famous authors who realized the importance of usability and
de�ned it as a relative property of a system. This property makes the relationship between
users and the systems; i.e. the subjective usability measure. The other side relates to
objective measures of interaction i.e. how easily and e�ciently users could achieve their
goals . Shackel de�ned the following attributes to measure the usability.
� E�ectiveness: how e�ciently and with minimum errors the user completes his
goal.
� Learnability: how much time the user required learning and how long this could
remember it.
� Flexibility: adaptation to change in tasks.
� Attitude: user satisfaction with the system.
Figure 2: Shackel's Usability De�nition [21].
14
Nielsen [22] is another usability expert, who de�ned usability by considering the following
attributes.
� Learnability: the system should be simple and easy to learn.
� E�ciency: the system should be capable to perform the users work within speci�ed
time.
� Memorability: it should be easy to remember, so that the user can easily use it
after some period of not using the system, without having to learn all again.
� Errors: the system should have low error rate and provide help to the user while
he/she make errors.
� Satisfaction: the system should be attractive, so that the user feels comfort and
satisfaction while using it.
Figure 3: Nielsen's Usability De�nition [21].
Since from last 15 years, the International Standard Organization (ISO) has developed
di�erent standards for HCI and usability. The functionality of these standards is to
make the system simple and consistent. The ISO 9241-11 decompose usability into three
attributes [23] that are as discussed below.
� E�ectiveness: the accuracy and time for speci�ed goal achievement by speci�ed
users.
� E�ciency: utilization of resources for completion of speci�ed tasks.
� Satisfaction: the user must be feeling comfort while using the system.
The software engineering community has also associated usability with interface design.
The ISO 9126-1 de�ned usability as �quality in use�. This de�nition has the same meaning
as the ISO 9241-11 de�ne usability but it is used in speci�c context. For the measurement
of usability, ISO 9126-1 speci�ed the following usability attributes [24].
� Understandability: the software product must have the capability that the user
can easily understand and utilize it for the speci�ed task.
15
Figure 4: ISO 9241-11 Usability De�nition [21].
� Learnability: the software system should have the capability that the user should
learn it easily.
� Operability: the software system must have the capability that user could easily
operate and control it.
� Attractiveness: the software system must have capability to attract the user.
The existing usability studies on mobile applications de�ne the nine usability attributes
such as: learnability, e�ciency, memorability, errors, user satisfaction, e�ectiveness, sim-
plicity, comprehensibility and learning performance. Selection of suitable attributes for
evaluation of mobile applications depends on the nature of the applications and the ob-
jectives of the usability study [25].
2.2 Usability Measurement
Many past studies presented that the usability is dependent on the context of use [18].
This context of use is formed by the user, task, equipment and environment [26]. Most
of the studies are conducted in controlled environment, where the user performed some
prede�ned tasks using the speci�ed system. The usability can be assessed by consider-
ing several measurable parameters. These parameters can be divided into two groups:
subjective preference measures and objective performance measures.
Subjective preference measures are being assessed by considering what the user likes
and dislikes about the system. It can be measured by using for example a questionnaires
[27]. An advantage of a questionnaires is that they can provide the evaluator with feedback
from the user's point of view. Another bene�t of the questionnaires is to provide the
comparable measures across the systems that being evaluated [28].
The objective performance measures relate to the e�ectiveness of system (e.g. time,
errors, and number of activities). Objective performance measures are being assessed by
giving the speci�ed tasks under a speci�ed time limit. Sometimes it can be measured by
counting the number of tasks per unit time [27]. The e�ciency measurement of the system
is also related to the total time on task, usages of the resources and mental resources used
to manipulate the system interfaces.
16
2.3 Performance Measuring
Performance is the degree to which the system accomplishes its designated function under
certain constraints [29]. In this study, the authors followed the usability attributes that
de�ned by ISO 9241-11 i.e. e�ectiveness, e�ciency and satisfaction. To test the usability
attributes following metrics were measured.
Task Time: How long it took a participant to complete a task in seconds [30].
Errors: Errors are an unintentional action, mistakes, slips and omissions, a user makes
while performing a task [30].
Satisfaction: It is the reaction of the participants about the overall performance of the
system.
2.4 Usability Evaluation
Usability evaluation is �the systematic process of collecting data, in ordered to have a
better understanding of the users and how the user groups use the product to perform a
speci�c task under speci�ed conditions� [31]. It has been required in di�erent stages of
the system development process. Its goal is to provide feedback that helps to improve
the quality and functionality of the system [32]. Zhang [33] de�ned three approaches
of usability evaluation methods such as: testing (e.g. Coaching method, performance
measurement, thinking aloud protocol), inspection (e.g. heuristic evaluation) and inquiry
(e.g. interviews, �eld observation, questionnaires). In the usability testing approach,
speci�ed users perform speci�c tasks using the system or prototype [21]. The usability
evaluator examines, how easily and e�ciently the users perform their tasks. The usability
inspection approach involves the usability experts or professional to examine whether the
system or each part of the system follows usability principles. In the usability inquiry
approach the evaluator collect information about user perceptions and understanding of
the system through interviews, surveys and verbal discussions [21].
2.5 Touch Screen and Smart Phones
Touch screen user interfaces have become more attractive in electronic devices [34]. There
are two types of touch screens: capacitive and resistive. Resistive touch screens are
made of a number of metallic and electronic conductive layers, separated with a small
gap. When a user tap on the touching surface, both layers make a connection and cause
the electric current that activate the touching event. Capacitive touch screens work by
sensing conductive object e.g. �nger. They allow the multi-touch functionality that can be
performed by using multiple bare �ngers [35]. The touch screen input method is becoming
17
popular in the smart-phones and other mobiles devices such as PDAs and tablet laptops
[36]. In 1993, IBM presented the �rst smart-phone and its name was Simon. The main
features of Simon were a calendar, a calculator, a world clock, and an email client. It had
a touch screen interface that let the users write text by using an on-screen keyboard or
a stylus (i.e. stick like a pen). There is no proper de�nition of the term smart phone.
However, it is considered as a mobile phone that has a greater computing power than the
normal cell phones and having the features like the PCs. After launching the iPhone, the
user interface design had great impact on the smart-phones with multi-touch technology
[37].
2.6 The iPhone and the iPad
Apple designs, manufactures and markets a range of digital products. These products
contain personal computers, mobiles devices, music players, related software and third-
party applications. In the past few years, the company has launched products such as the
iPhone, iPod, and iPad [38]. The iPhone and iPad are considered as revolutionary Apple
products [39] and both have some similar features and look like the smaller and larger
version of each other. However in some perspectives they have some di�erences such as
RAM, size, resolution and weight [40]. These similarities and di�erences are presented in
the Table 1.
Feature iPhone 4 iPad
Chip A4 A4
OS iOS 4 iOS 4
Processor 1 GHz ARM Cortex-A8 1 GHz ARM Cortex-A8
RAM 512 MB 256 MB
Connection Wi-Fi Wi-Fi
Size 4.5 x 2.31 x0.37 inch 9.56 X 7.47 X 0.52 inch
Resolution 960 X 640 pixel 1024 X 768 Pixel
Weight 137g 730g
Table 1: iPhone and iPad Speci�cation Comparison. The Bold Values Show the Di�er-ences in Speci�cation.
2.7 Features of the iPhone and iPad
The main features of the iPhone and iPad are described below:
2.7.1 Multi-touching
The multi-touch technology is used as an input method for the iPhone and iPad. It is
de�ned as the technology where the user can interact with system with multiple inputs
at a time e.g. two or more �ngers for a single task. The users can use their �ngers for
18
typing email, enlarging text, swapping through photos and for using zooming functions.
All input functions are performed by using the �ngers at touch screen interfaces.
2.7.2 Operating System (iOS 4)
The iOS 4 is an operating system that is being used by Apple for the iPhone 4 and iPad.
It is considered as a simple operating system. The user can perform various tasks (e.g.
writing email and zooming text) simply using the �ngers on the touch screen interface.
The user can use millions of di�erent built-in applications with a few �nger gestures such
as tap, drag, swipe and pinch [38].
2.7.3 Applications (Apps) Store
Apps Store is a database that is especially designed for Apple's products. It contains
business applications, education applications, entertainment applications, social network-
ing applications, news applications and so on. The applications are designed by Apple's
designers and developers as well as by a third party. Some applications are free but for
some applications you will have to pay [38].
2.8 Related Work
Tsung [41] performed an age based usability study on the iPhone. Three age groups
i.e. college-level youth, middle-aged professionals and senior citizens were taken. The
conclusion of this study was that the system should not only consider the �glamorous
animation and fancy utilities� but all human physical conditions e.g. hearing and eye
sight that will make the system more senior friendly.
Chaparro [42] evaluated the keyboard performance of the Netbook and iPad. They
measured the performance on three di�erent style keyboards. The iPad soft touch key-
board (portrait as well as in landscape) and the Acer Netbook physical keyboard. The
attributes and their metrics are given below.
� Performance (task time and errors)
� Perceived Mental Workload
� Satisfaction
The study showed that there was a slight di�erence in the performance of the Netbook
physical keyboard and the iPad landscape and no di�erence in the performance of the
iPad portrait and landscape keyboards. The satisfaction level of the iPad landscape was
slightly higher than the physical keyboard and portrait was slightly lower than the physical
keyboard. The preference level of the landscape iPad keyboard and physical keyboard was
same the while portrait keyboard was less preferred by the participants .
19
In another study, six sta� members at Curtin University were supplied with the iPad
to test usability for 100 days. The iPad was handed over to each sta� member and at the
end of the third month these participants were interviewed. The research showed that
there was no problem for inexperienced users using the iPad. The consistency issues were
noted like the other studies and also some common functionalities were identi�ed [43].
The Nielsen and Norm group (NNGroup) [44] conducted a usability study of the iPad
applications and contents, a few weeks after Apple launched the device. They tested seven
users; all had at least three months iPhone experience and only one of them was an iPad
user. In this study they assessed the interface of applications, websites and speci�c iPad
applications, touch screen, changing orientation, and di�erent gestures. They criticized
and pointed out that the applications are designed without considering the importance of
usability e.g. read tab asymmetry, zoom the map or zoom the page, use of proper mental
models, small target area, accidental tapping and the back button etc. They highlighted
many usability issues e.g. inconsistencies and accidental actions etc.
2.9 Issues in the iPhone and iPad
From the literature [44], the authors found some usability issues that are discussed below:
2.9.1 Consistency
The consistency issue remain almost all the time with the iPhone and iPad applications.
Figure 5 shows the inconsistency in the button, keyboard appearance and writing envi-
ronment of the messages of SMS and MobileVoip application of the iPhone.
Figure 5: Di�erent Consistency Issues in SMS and Mobile Voip Applications.
20
2.9.2 A�ordance
The other issue in the iPhone as well as the iPad application was a�ordance. �Users don't
know that something is touchable unless it looks so� [44]. In the weather application of
the iPhone it is very di�cult to add new a location as the option lacks the a�ordance.
Figure 6 shows this a�ordance problem in the weather application of the iPhone. The �i�
at the right bottom side of the application is used for adding the new location but the
option is not very clear as well as not seems to be tappable.
Figure 6: A�ordance Issues in the Weather Applications of the iPhone and iPad.
2.9.3 Small Search Box and Missing Information
Figure 7 shows the small search box and missing search button at the bottom of the Apps
Store application of iPad. As there is a big screen on the iPad the search box needs to be
more clear and also users expect the search button at the bottom of the application.
Figure 7: Small Search Box Issues in the Applications of the iPhone and iPad.
21
2.9.4 Invisible Control and Missing Instructions
Figure 8 shows the invisible control and missing instructions in the iPhone and iPad
applications. In Figure 8 at the right side when the message is too long it create a scroll
bar but it remain invisible until a someone tap on it. Similarly in twitter there are four
pages that can swipe forward and backward but there are no instructions to move them.
Another miss leading button at the top of the twitter application that seems like a setting
button but in fact it is used for reply option.
Figure 8: Invisible Control and Missing Instructions Issues in the Twitter and FacebookApplication on the iPhone and iPad.
Table given bellows describe the list of other usability issues in the iPhone and iPad.
Usability Issues Detail
Inconsistency in interaction Nothing happens
Enlarging the picture
Hyper-linking to a more detail page about that item
Replace the picture with new picture
Popping up a set of navigation choices
Accidental touch Touch something that one did not mean to
False back button Button that look like back button, but when pressed
it lead to new page
Lack of a�ordance Tappable elements look like it not supposed to touch
and untappable look like tappable
Hyperlinks Missing of important hyperlinks like wired magazine
using back button instead of hyperlinks
Changing orientation Application looks di�erent in portrait and landscape
Gestures Di�cult to remember gestures
Multiple panels Small font size and crowded contents
Auto-correct Should turn into autocomplete
Table 2: Usability Issues on the iPhone and iPad.
22
Chapter 3
3 RESEARCH METHODOLOGY
The research methodology is the systematic way to solve the research problem. Commonly
two main approaches are used for the research studies such as quantitative and qualitative
[45]. However, Creswell [46] has also introduced a third type of the research approach
called a mixed method approach i.e. the combination of quantitative and qualitative
approaches.
The quantitative research approach is based on the measurement of quantity or amount.
It is used when one begins with a hypothesis and tests for con�rmation and dis-con�rmation
of that hypothesis [47]. It is applicable to phenomena that can be expressed in terms of
quantity [48]. Generally, the survey and experiment are used in the quantitative approach
to inquiry the hypothetical issues and participants of the studies [46]. Some advantages
of the quantitative approach [49] are listed below:
1. Data can be gathered from large user groups in a short period of time
2. Large sample data can be generalized to the entire population
3. Quantitative research is repeatable
The qualitative research approach is concerned with subjective assessment of attitudes,
opinions and behaviors [48]. It is an interpretative approach which uses the meaning of
phenomena as understood by the participants of the studies [50]. Generally, open ended
questions are used in qualitative research [51]. Some of the advantages and disadvantages
are listed in Table 3.
Advantages of Qualitative Approach Disadvantages of Qualitative Approach
It goes deep and takes into account all details in Finding cannot be generalized because they
the environment are particular environment dependent
No need to recruit a large number of participants The �nding can be biased according to the
Flexibility in time and place understanding of the researcher
No need to conduct interviews of all participants
at same time
Table 3: Advantages and Disadvantages of Qualitative Approach [52] .
In this research, the authors adopted the mixed research approach. The quantitative
research approach was used to perform the experiment and the qualitative research ap-
proach was used to conduct the literature review and analyze the open ended questions
and participants comments. These research studies were carried out in multiple steps such
as a literature review, survey design, experiment design, pilot tests studies, experiment
23
execution, selection of suitable evaluation methods, analysis of gathered data, results,
conclusions and future work.
3.1 Literature Review
In the �rst step a literature review was done. The objectives of a literature review was
to understand the usability, usability attributes, usability issues in smart phone, and the
usability issues in the iPhone and iPad. Another objective of a literature review was
acquiring the understandability of the di�erent statistical usability evaluation methods.
The authors utilized the all available information resources to carried out this re-
search work. The mentioned keywords are used to �nd out the related data from all
available databases. However some of the authentic knowledge resources were listed as
BTH Library's books, IEEE, ACM digital library, Engineering Village, Google Scholar,
Springer-link, Scopus, and Ebrary.
3.2 Survey Design
To �nd usability issues the authors used the survey technique. The objectives of the
surveys were:
1. To validate the issues regarding the usability of the iPhone and iPad that authors
�nd out during the literature review.
2. To get the input for the experimental design.
3. To �nd out the users point of views about the usability of the iPhone and iPad.
A survey is an e�ective way to gather information from a large number of the population
in a short period of time at relatively low cost. It is considered that the respondents are
less likely try to please the researcher and provide the social acceptable responses. The
survey technique gives time to the respondents to think about the question or statement
before answering and do not bind them to complete it in one setting [53].
The seven point Likert scale was used in a survey questionnaire that helps the users
to express their degree of agreement with a statement. The numbers (1-7) were used for
strongly disagree, disagree, near to disagree, neutral, near to agree, agree and strongly
agree respectively for each statement.
In order to collect feedback from the users about the iPhone and the iPad, the authors
divided the survey into demographics, systems (iPhone, iPad) information, applications
information, touch screen features and keypad functionality. Each part contained several
related statements, as well as a space for own comments. The detailed survey questionnaire
is given in Appendix A. However, here the small piece of each section of questionnaire
was presented and structured as follows.
24
1. Presentation This section presented the users demographics (age, sex, quali�cation,
using experience), purpose of the survey study, explain the questionnaire and provide the
general information. Following is the presentation section of the survey questionnaire.
Survey Questionnaire for Evaluating the Usability of the systems(iPhone/iPad)
Participant Number: ______________ Sex: ___________________Home Country: _____________________ Age: ________________Education level: 2 BSc 2 MSc 2 PhD 2 Other __________________This questionnaire gives you an opportunity to describe your experiences of the system.Your responses will help us to evaluate the usability of the system.To as great degree as possible, consider all the tasks that you have been performing withthe system while you answer these questions. Please read each statement and indicatehow strongly you disagree or agree with the statement by marking (X) in the circle. If astatement does not apply to you, leave it empty or use the word (N/A) at the commentsline.Please write comments to elaborate on your answers (if you have any).As you complete the questionnaire, please do not hesitate to ask any questions.
Thank you.
Figure 9: Presentation.
2. System (iPhone/ iPad) This section presented the overall impression of the de-
vices. The objective of this section was to obtain the feedback about the system design,
graphics, capabilities and somehow about the ease of use. A piece of the system section
of a questionnaire is presented in Figure 9 below.
Figure 10: System.
3. Touch Screen This section presented the usability of the touch screen gestures.
Figure 10 shows a few statements of such section.
25
Figure 11: Touch Screen.
4. Keypad This section presented to gain the user opinions about the performance
of the keypad. The statements contained the di�erent keypad aspects such small size of
keys, landscape style and single character keys. The Figure 10 shows a few statements
regarding the keypad section.
Figure 12: Keypad.
5. Applications This section presented the di�erent usability features related to the
applications simplicity, ease to learn, consistency, visibility of the text and recoverability
of the mistakes. A small piece of this section presented in the Figure 12.
Figure 13: Applications.
26
3.3 Experimental Design
Experimental design is the process of planning a study to meet speci�ed objectives. Proper
planning of an experiment is important in order to ensure that the right type of data,
a su�cient sample size and resources are available to answer the research questions of
interest as clearly and e�ciently as possible [54, 55]. The following aspects provide further
insight into the experimental design.
3.3.1 Participant Selection
The authors selected the three user groups such as young novice users (20), young expe-
rience users (20) and elderly users (20). All the young users were bth students and well
known with the English language. In the elderly users 12 were bth employees; �ve were
the Karlskrona public library employees and three were Ronneby public library employees.
In the beginning, the authors were interested to divide the elderly people into two groups
like the young people (novice, experienced) and also visited the old people apartments in
Karlskrona and Ronneby. However, after meeting with the management of the old people
apartments the authors realized that it is di�cult to collect reliable data from the elderly
people because of language communication problems. So the authors decided to make a
single group of the elderly people and involved all those people who would speak, write,
read and understand the English language. Figure 13 shows the participants selection for
the experiment.
Figure 14: Participants Selection for the Experiment.
Novice Participant The person who was never used the smart phone, touch screen
systems or the internet on small devices.
27
Experienced Participant The authors considered the person as an experienced par-
ticipant who was using the smart phone or the iPhone more than three months.
Elderly Participant The person who was at least using a computer system or smart
phone or having the experience about the touch screen system.
3.3.2 Variables Selection
A variable is any character or attribute that can vary across people or situation or thing
[56]. There are two basic types of variables; independent variable and dependent variable.
The independent variable is one which the experimenter controls and manipulates to see
the e�ect on the treatments (dependent variable). On the other hand, the dependent
variable is the responses of the independent variable [56].
In the experiment, the authors used two types of the variables, independent variable
(factor) i.e. �Participant group� and dependent variable i.e. �Device�. The �Participant
group� variable has three levels such as novice user, experienced user and elderly user.
Similarly, the �Device� variable also has two levels such as the iPhone and iPad.
3.3.3 Hypotheses
The �rst step in the research process is the formulation of hypothesis about a speci�c
issue [57]. It is a precise problem statement that can be directly tested through empirical
investigation [58].
In comparative usability study of the iPhone and iPad across the three groups of par-
ticipants, the authors were interested to measure the performance di�erences between the
device levels as well as the di�erences among the participant group levels. The hypotheses
statements were de�ned as as follows:
H01: There is no di�erence in group levels over the device variable.
Ha1: There is a di�erence in group levels over the device variable.
H02: There is no di�erence in device levels across the group variable
Ha2: There is a di�erence in device levels across the group variable.
Here H0 and Ha represent the null hypotheses and alternative hypotheses respectively.
3.3.4 Task De�nition
For the execution of usability tests, six tasks were de�ned. Each task and its description
were designed on a separate paper. The presentation of the tasks was as follows:
28
Task 1: Login to Facebook.
� Email address: [email protected]
� Password: usbtest2k11
Description:
� Tap on Facebook
� Tap on the First Text box �Email� and enter the email address
� Tap on the �password� text box and enter the password
� Tap on �Go� at the keypad
� To use capital text tap on
� To use numbers tap on
29
Task 2: Compose a new message on Facebook and send it.
� To: [email protected]
� TEXT: This is a usability test message for the iPhone
� Send
Description:
� Tap on the Facebook Home button
� Tap on messages
� Tap on create message at the top right corner of the Facebook application
� Enter email address to whom you want to send a text , i.e. Given above
� Tap just below the line and Write a short message
� Tap on Send Button at the top of application
Google Map
Task 3: using the Google mobile application �nd the location of Blekinge
Tekniska Högskola and tap on the sign to con�rm the location.
Description:
� Tap on Google mobile application
� Tap on Apps at middle of bottom
� Tap on Maps
� Write Blekinge Tekniska Högskola
� Tap on search button
� Tap on the sign to con�rm the location
30
Task 4: Make a close view of Blekinge Tekniska Högskola building with zoom
in function.
Description:
� Tap on Layers at the bottom left corner
� Select the satellite from list
� Tap on search box
� Write Blekinge Tekniska Högskola
� Tap on sign for zoom in
Evernote
Task 5: Create a new note in the Evernote application
� User name: azamjarral
� Password: aS5AK9
� Title: First Note
� Text: My �rst note in the Evernote application
� Save
Description:
� Open Evernote application
� Tap on Sign in button
� Tap on text box username and enter username
� Tap on the text box password and enter password
� Tap on Sign in
� Tap on New Note at the bottom left corner. (For iPhone at the middle of Facebook
application)
� Tap on the Save button at the right top corner of the application
31
Task 6: Logout from Evernote
Description:
� Open Evernote application
� Tap on the setting icon
� Find the Logout option
� Tap on Logout
3.3.5 Procedure
The authors booked a silent room in BTH that was used as a usability laboratory. To
provide the same environment to all the participants, the authors �xed the devices and
chairs. Before starting the experiment, all the tasks were arranged at a separate table. it
was conducted with a single user at a time. The experimental setup is shown in Figure
14.
Figure 15: Laboratory Setup for User Trials.
3.3.6 Experimental Material
The experimental material contained the iPhone, iPad, documents for the tasks descrip-
tion, demonstration and post test questionnaire.
32
3.3.7 Post Test Questionnaire
To know the participants usability experience abut the systems, a post-test questionnaire
was designed. It contained close ended as well as open ended questions. The close ended
questions were designed by using seven points Likert Scale and open ended questions were
also part of the same questionnaire but they were in the form of participants comments.
The close ended questions limit the respondent to the set of alternatives being o�ered,
however the respondent could easily respond the maximum questions within the limited
time. On the other hand the open ended questions help to explore the respondent and
thus avoiding the bias which may occur in the case of close ended questions [59]. The
detailed questionnaire is given in Appendix D.
3.3.8 Pilot Studies
Three pilot studies were conducted i.e. an experiment, a survey and a post test question-
naire. Before starting the experiment pilot test were conducted to assure that the tasks
were properly de�ned and the participants would have no di�culty in understanding every
step in each task. Three tests were conducted, one for each group and improvements were
made on the basis of their feedback. At the start the authors decided to just mention the
task e.g. login to Facebook, however after the pilot test, the authors realized that it was
not easy for the novice users as well as for elderly user to successfully complete the given
tasks. Then the authors decided to write down the step by step description of each task
with the symbolic representation wherever required. In this way the ambiguities were
removed in the experimental tasks.
Similarly the pilot studies were also conducted for the survey questionnaire and post
test questionnaire. After those studies, the changes were made in both questionnaires
according to the requirement of the users.
3.4 Evaluation Methods
In order to evaluate the survey and experimental data, the authors use multiple statistical
test methods. The detail of each method is given below.
3.4.1 Mann Whitney Test
The Mann Whitney test is a nonparametric test that can be used for comparing two
unequal sample size and unpaired data [60, 61]. The key result is the �p� value that shows
the signi�cant di�erence between both samples [62]. In this research, the survey data was
unpaired, unequal samples size and consist on rating scale that is why the Mann Whitney
test used to measure the signi�cance between the iPhone and iPad.
33
3.4.2 Normality Test
After obtaining the data from the experiment, it was important to test the normality
of the data. The authors read three test techniques: Shapiro Wilk Test, Kolmogorov-
Smirnov (KS) Test, and Histogram. The KS test is used for more than 50 samples, so
the authors did not use this method. On the experimental data, the Shapiro Wilk Test
and Histogram techniques could be applied to check the normality. however the authors
used �Shapiro Wilk test� online software that directly show whether the data is normally
distributed or not [63].
3.4.3 Analysis of Variances (ANOVA)
ANOVA is the most powerful and robust parametric method used to measure the signif-
icance. It has some assumptions such as the homogeneity of variance or normally distri-
bution of the data. The ANOVA is used to measure the signi�cant di�erence between
two or more than two groups [64]. In the experiment, there were three user groups and
two devices. The purpose experiment was to measure the di�erences among the groups
and between the devices. So to analyze the data, the authors used two way ANOVA
with replication. Further to see where the signi�cant di�erences between the devices and
among the groups were, the unpaired t-test and paired t-test were used respectively.
3.4.4 Unpaired T-test
The unpaired t-test is used to see where the signi�cant di�erences among groups across
the each device were. The comparison of the groups was made as follows.
Figure 16: Participant Group Levels Comparison Across the Device.
3.4.5 Paired T-test
The paired t-test used to see where the signi�cant di�erences between the devices across
the each group were. The comparison between devices across groups was as follows.
34
Figure 17: Device Levels Comparison Across the Participant.
3.4.6 Wilcoxon Matched Paired Signed Ranks Test
The Wilcoxon Matched Paired Signed Ranks test is a nonparametric test that is an
alternative to the paired t-test. It should be used if the distribution of di�erences between
pairs may be non-normally distributed [65]. The authors applied this method to �nd out
the signi�cance between the post-test questionnaires of the iPhone and iPad.
3.4.7 Error Bars
The error bars are used to estimate the variation or distribution of a statistical data in
the graphs. They can represent the standard deviation, the standard error, the con�dence
interval, or some other measure of error or uncertainty. Here the authors only discussed the
standard deviation and standard error. The standard deviation is a measure of variation
in data about their mean and standard error is a standard deviation of the estimated
statistical data such as mean, and regression coe�cient etc. The error bars do not let you
decide whether the di�erence between the mean is statistically signi�cant or not when
the two samples error bars do overlap or not [66]. It indicates how closely the population
mean is likely to be estimated by the sample mean.
3.5 Validity Threats
Validity refers to the conceptual and scienti�c soundness of a research study and investi-
gation [67]. There are many potential issues that may a�ect the reliability of the research.
Four types of validity threats discussed in the literature [46] are as follows:
� Internal validity threats
� External validity threats
� Constructive validity threats
� Statistical conclusion validity threats
35
3.5.1 Internal Validity Threats
Internal validity threats refer to the research design that causes the interferences of the
researcher's ability to draw the correct inference from the gathered data [46]. They might
be the inadequate procedures, wrong participant's selection, and technical skill of the
researchers, ambiguities in the experiment, post-test questionnaire and survey design and
improper instrumentation of the experiment [67, 68]. To overcome such types of issues
the authors studied the core concepts of survey design, experiment design and ongoing
research studies about usability of the iPhone and iPad. In the survey and post test
questionnaire, the authors obtained the demographics that help to minimize the wrong
selection of the participants. Pilot tests were conducted for the survey questionnaire,
experiments and post-test questionnaire which helped to �nd out the ambiguities in all
of them. Furthermore the authors also used the counterbalancing technique to overcome
the bias.
3.5.2 External Validity Threats
External validity threats relate to the generalizability of the results of a research study [67].
They might be the sample characteristics, stimulus characteristics and settings, reactivity
of experimental arrangements, multiple treatments interference, novelty e�ect, reactivity
of assessment and timing of measurement [67]. To minimize the external validity threats,
the authors requested all of the participants to report their own experiences because the
purpose of the study is to evaluate the performances of devices not to the participants.
However there might be external validity risks, as people really do not report their personal
reality because they want to see themselves in a good light [68].
3.5.3 Constructive Validity Threats
Constructive validity threats relate to the selection of de�nitions and the measures of the
variables in a study [46]. Authentic data sources such as IEEE, ACM Digital Library,
ScienceDirect provide the correct de�nitions and measures of the variables. The authors
try to use the authentic data sources to minimize such types of validity threats.
3.5.4 Statistical Conclusion Validity Threats
Statistical conclusion validity threats relate to the statistical conclusion of a study. These
threats might be the cause of the inadequate selection of a statistical power such as
sampling and suitable methods [67, 68]. To reduce such type of validity threats, the
authors studies di�erent statistical methods in detail.
36
Chapter 4
4 STATISTICAL ANALYSIS, RESULTS AND DISCUSSIONS
This chapter presents the analysis, results and discussions of the results about the collected
data of the studies. There are two main sections of this chapter; the �rst one presents the
results and discussions on the results of the survey data and the second one presents the
results and discussions on the results of the experimental data.
4.1 Survey
In order to evaluate the survey data, the authors clustered the 42 survey statements into
subsections such as system results, touch screen results, keypad results and applications
results. The results of the survey responses are given in the Appendix B and the p values
are given in the Appendix C.
4.2 Demographics Results
This section of the survey was presented the demographics of the respondents. The
authors collected feedback from 14 (8M, 6F, age = 21-35 years) iPhone and 10 (7M, 3F,
age = 23-47 years) iPad users through the survey. 69% of the respondents had been using
the iPhone for more than six months. 8% of the respondents had been using the iPhone
for four to six months and the remaining 23% participants had been using it for the last
one to three months. On the other hand, the iPad responses showed that 20% of the
respondents were using the iPad for more than six months. 30% of the respondents had
been using the iPad for four to six months. 30% of the respondents had been using it for
the last one to three months and 20% of the respondents had been using the iPad for less
than one month.
4.3 Usability and Preference Testing
4.3.1 System Results
The �rst section of the usability testing questionnaire named �system� contained 15 state-
ments. Each statement was related to the overall satisfaction about the iPhone and iPad.
The results and signi�cances between the users opinions against each statement regarding
the iPhone and iPad are:
The system is easy to use for opening applications.7% (1) of the iPhone participants
disagreed with a statement and they considered it di�cult for opening applications. 21%
(3) were near to agree, 14% (2) were agree and 57% (8) were strongly agree that the
37
iPhone was easy to use in opening application. On the other hand, the iPad participant's
responses results presented that 10% (1) was near to agree, 70% (7) were agree and 20% (2)
were strongly agree with the above statement. However the comparative study regarding
this statement presented that there was no signi�cant di�erence (p = 0.183 > 0.05)
between the user views for the iPhone and iPad.
The system is easy to use in closing applications. The iPhone results regarding this
statement showed that 8% (1) of the respondent was disagree and he thought it is not easy
to use in closing applications; 25% (3) participants were neutral; 8% (1) participant was
near to agree, 25% (3) were agree and 33% (4) were strongly agree. The iPad participants
responses results presented as: 11% (1) was disagree, 11% (1) was near to disagree; 11%
(1) was neutral; 44% (4) were agree and 22% (2) were strongly agree. The comparative
study of the user's opinions concerning the iPhone and iPad presented that there was no
signi�cance (p = 0.404 > 0.05).
The system is easy to use in �nding help. The iPhone participant's responses results
concerning this statement were presented as: 14% (2) were disagree, 7% (1) was near to
disagree; 43% (6) were neutral; 14% (2) were near to agree, 7% (1) was agree and 14%
(2) were strongly agree. Similarly, the iPad participants responses results presented as:
10% (1) was disagree, 20% (2) were near to disagree; 40% (4) were neutral; 20% (2) were
near to agree and 10% (1) was agree. The comparative study of the user's views showed
that there was no signi�cant di�erence (p = 0.083 > 0.05) between the iPhone and iPad.
I can easily use the system in the sun light. The iPhone participants responses results
regarding this statement were presented as: 7% (1) was near to disagree; 21% (3) were
neutral; 36 % (5) were near to agree, 21% (3) were agree and 14% (2) were strongly
agree. The iPad responses results presented as: 20% (2) were near to disagree; 40% (4)
were neutral; 30% (3) were near to agree and 10% (1) was agree. The comparative study
concerning this statement showed that there is a signi�cant di�erence (p = 0.011 < 0.05)
in user's views regarding the iPhone and iPad. The statistical result showed that the users
preferred the use of iPhone in sun light than the iPad.
It is easy to �nd a new application in the Apple store. The iPhone responses results
showed that 7% (1) was near to disagree; 14% (2) were neutral; 14% (2) were near to
agree, 7% (1) was agree and 57% (8) were strong agree. Similarly the iPad responses
results regarding this statement showed that 10% (1) was disagree, 10% (1) was near to
disagree; 30% (3) were neutral; 30% (3) were near to agree, 10% (1) was agree and 10%
(1) was strongly agree. Comparative statistical result showed that there was a signi�cant
di�erence (p = 0.018 < 0.05). The users responses result showed that they would easily
use the iPad for �nding the new applications on the Apple store comparing the iPhone.
The layout of the applications on the system screen is clear. The results of the iPhone
respondents showed that 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near to
agree, 21% (3) were agree and 36% (5) were strongly agree. Similarly, the iPad participants
38
responses results were presented as: 10% (1) was neutral; 60% (6) were agree and 30%
(3) were strongly agree across above statement. The comparison result showed that there
was no signi�cance (p = 0.322 > 0.05) in users views for the iPhone and iPad.
The graphics of the system are appealing for all age groups. Responses results regarding
the iPhone presented that 7% (1) was disagree; 7% (1) was neutral; 21% (3) were near
to agree, 29% (4) were agree and 36% (5) were strongly agree. Similarly the iPad results
presented that 11% (1) was near to agree, 56% (5) agree and 33% (3) were strongly agree.
The comparison result showed that there was no signi�cant di�erence (p = 0.341 > 0.05)
between the participants opinions about the iPhone and iPad.
I can easily change the background colour of the interface. The iPhone participant's
responses results presented as: 14% (2) were disagree, 14% (2) were near to disagree;
21% (3) were neutral; 14% (2) were near to agree and 36% (5) were strongly agree. On
the other side, the iPad participant's responses results showed that 10% (1) was near
to disagree; 30% (3) were neutral; 20% (2) were near to agree, 30% (3) were agree and
10% (1) was strongly agree. However the comparative study showed that there was no
signi�cance (p = 0.489 > 0.05) between the participants opinions regarding the iPhone
and iPad.
I think the system is designed for all age groups. The iPhone results showed that 7%
(1) was strongly disagree, 7% (1) was disagree; 21% (3) were neutral; 14% (2) were near
to agree, 21% (3) were agree and 29% (4) were strongly agree. Alike, the iPad results
presented that 20% (2) were near to disagree; 10% (1) was neutral; 10% (1) was near
to agree, 30% (3) were agree and 30% (3) were strongly agree. There was no signi�cant
di�erence(p = 0.467 > 0.05) in the user's opinions for the iPhone and iPad.
I like the �nishing of the interface. The iPhone responses results presented that 7%
(1) was disagree; 7 % (1) was neutral; 21% (3) were near to agree, 21% (3) were agree
and 43% (6) were strongly agree. Similarly the iPad results presented that 20% (2) were
neutral; 20% (2) were near to agree, 20% (2) were agree and 30% (3) were strongly agree.
The comparative study showed that there was no signi�cant di�erence (p = 0.382 > 0.05)
in the participants' views for the iPhone and iPad.
The interface of the system is pleasant. The results of the iPhone responses: 7% (1)
was disagree; 14% (2) were neutral; 21% (3) were near to agree, 29% (4) were agree and
29% (4) were strongly agree. Similarly, the results of the iPad responses: 10% (1) was
neutral; 40% (4) were near to agree, 20% (2) were agree and 30% (3) were strongly agree.
The comparative study showed that there was no signi�cant di�erence(p = 0.467 > 0.05)
between the users outlook for the iPhone and iPad.
It is easy to install the required application. The iPhone responses results showed
that 7% (1) was disagree; 7% (1) was neutral; 14% (2) were near to agree, 29% (4) were
agree and 43% (6) were strongly agree. Similarly, the iPad responses results showed that
11% (1) was near to disagree; 11% (1) was neutral; 33% (3) were near to agree, 22% (2)
39
were agree and 22% (2) were strongly agree. The comparison result showed there was no
signi�cant di�erence (p = 0.174 > 0.05).
It is simple to uninstall the application. The iPhone responses results showed that 7%
(1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral; 7% (1) was near to
agree, 36% (5) were agree and 36% (5) were strongly agree. On the other side, the iPad
responses results showed that 11% (1) was near to disagree; 11% (1) was neutral; 11% (1)
was near to agree, 44% (4) were agree and 22% (2) were strongly agree. The comparative
study showed that there was no signi�cance(p = 0.279 > 0.05) in the iPhone and iPad
users views.
The system has all the functions and capabilities I expect it to have. The iPhone
responses results showed that 29% (4) were disagree, 7% (1) was near to disagree; 7% (1)
was neutral; 29% (4) were near to agree, 21% (3) were agree and 7% (1) was strongly
agree. Similarly, the iPad responses results showed that 30% (3) were disagree, 10% (1)
was near to disagree; 20% (2) were neutral; 20% (2) were near to agree and 20% (2) were
agree. The comparison of the users point of views about the iPhone and iPad was showed
no signi�cance(p = 0.265 > 0.05).
I like using the interface of the system. For such statement, the iPhone responses
results showed that 7% (1) was strongly disagree, 7% (1) was near to disagree; 14% (2)
were neutral; 7% (1) was near to agree, 43% (6) were agree and 21% (3) were strongly
agree. The iPad responses results showed that 13% (1) was near to disagree; 13% (1)
was neutral; 50% (4) were near to agree and 25% (2) was strongly agree. Similarly the
comparative study about the user´s point of views regarding the iPhone and iPad showed
no signi�cance (p = 0.233 > 0.05). Figures 17 and 18 showed the results graphs of the
users opinions across the each statement regarding the iPhone and iPad respectively.
40
Figure 18: Percentage Graph Results of the Degree of Agreement Across Each SystemStatement on the iPhone. St. Represents Strongly.
Figure 19: Percentage Results of the Degree of Agreement Across Each System Statementof the iPad. St. Represents Strongly.
4.3.2 Touch Screen Gestures Results
The second section of the usability testing questionnaire named �Touch Screen Gestures�
contained 16-21 statements. Each statement presented di�erent usability features such as
41
zooming functionality, gestures memorability and learnability. The results obtained from
the user responses for each statement presented as follows.
I can easily use the text �zoom in� and �zoom out� signs (gestures). The results of
the iPad responses presented that 7% (1) was strongly disagree; 7% (1) was neutral; 7%
(1) was near to agree, 14% (2) was agree and 64% (9) were strongly agree. The iPad
results showed that 10% (1) was near to disagree; 10% (1) was neutral; 40% (4) were
near to agree, 20% (2) were agree and 20% (2) were strongly agree. The analysis of
the relationship showed that there was signi�cant di�erence (p = 0.027 < 0.05). For this
statement the users views result showed that they could easily performed the zoom in/out
gestures using the iPad than the iPhone.
I like the gestures for selecting and zooming the text. The results of the iPhone re-
sponses showed that 14% (2) were near to disagree; 7% (1) was near to agree, 50% (7) were
agree and 29% (4) were strongly agree. Similarly, the results of the iPad responses showed
that 20% (2) was near to disagree; 20% (2) were neutral; 10% (1) was near to agree, 40%
(4) were agree and 10% (1) was strongly agree. The comparative study showed that there
was no signi�cance(p = 0.108 > 0.05) in users opinions for the iPhone and iPad.
I think the gestures are easy to learn. The iPhone results showed that 7% (1) was
disagree, 7% (1) was near to disagree; 7% (1) was neutral; 43% (6) were near to agree, 7%
(1) was agree and 29% (4) were strongly agree. Similarly, the iPad results showed that
20% (2) were near to disagree; 10% (1) was neutral; 20% (2) were near to agree, 50% (5)
were agree. There was no signi�cance (p = 0.424 > 0.05) in users opinions for the iPhone
and iPad.
I think the gestures are easy to remember. The iPhone results showed that 7% (1) was
near to disagree; 14% (2) were neutral; 29% (4) were near to agree, 21% (3) were agree
and 29% (4) were strongly agree. On the other side, the iPad results showed that 40% (4)
were neutral; 20% (2) were near to agree, 30% (3) were agree and 10% (1) was strongly
agree. The comparative study showed that there was no signi�cance(p = 0.214 > 0.05)
between users views for the iPhone and iPad.
I can easily change the location of the contents (icons) on the interface. The iPhone
results showed that 7% (1) was disagree, 7% (1) was near to disagree; 7% (1) was neutral;
7% (1) was near to agree, 29% (4) were agree and 43% (6) were strongly agree. On the
other hand, the iPad results showed that 10% (1) was near to disagree; 20% (2) were
neutral; 50% (5) were near to agree and 20% (2) were agree. There was a signi�cance(p =
0.015 < 0.05) in users opinions for the iPhone and iPad. The results showed that the user
could easily change the contents location using the iPad as compared to iPhone.
I can easily select the written text for deletion. The iPhone results presented that 7%
(1) was strongly disagree, 14% (2) were disagree, 14% (2) were near to disagree; 14%
(2) neutral; 21% (3) were near to agree, 14% (2) were agree and 14% (2) were strongly
agree. Similarly, the iPad results showed that 30% (3) were disagree, 20% (2) were near
42
to disagree; 10% (1) was neutral; 30% (3) were near to agree and 10% (1) was agree. The
analysis relationship result showed that there was a no signi�cance (p = 0.198 > 0.05).
In Figure 19 presented the graphs results of the users opinions against each statement on
both the devices.
Figure 20: Graphs Show Percentage Results of the Degree of Agreement Across the EachStatement of the Touch Screen Gestures on the iPhone and iPad Respectively.
4.3.3 Keypad Results
The third section of the survey questionnaire named �keypad� contained eight statements.
Each statement assessed the performance of the keypad of both devices. The results
obtained from the user's responses for the each statement are presented as follows.
I enjoy using the keypad while writing Emails. The results of the iPhone responses
showed that 7% (1) was strongly disagree, 21% (3) were near to disagree; 14% (2) were
neutral; 14% (2) were near to agree, 7% (1) was agree and 36% (5) were strongly agree.
The results of the iPad responses showed that 10% (1) was strongly disagree, 10% (1) was
disagree; 10% (1) was neutral; 20% (2) were near to agree, 40% (4) were agree and 10%
(1) was strongly agree. The comparative study of the users opinion showed that there
was no signi�cance(p = 0.446 > 0.05) for the iPhone and iPad.
I like the keypad in landscape style while writing Emails. The results of the iPhone
responses showed that 14% (2) were disagree; 21% (3) were neutral; 14% (2) were near
to agree, 29% (4) were agree and 21% (3) were strongly agree. Similarly, the iPad results
43
showed that 20% (2) were strongly disagree, 10% (1) was near to disagree; 30% (3) were
neutral; 20% (2) were near to agree and 20% (2) were strongly agree. The comparative
study of the user's responses showed that there was no signi�cance(p = 0.143 > 0.05)
regarding the iPhone and iPad.
It is easy to use the special characters (e.g., `' _ *). The iPhone user's responses
results showed that 7% (1) was strongly disagree, 21% (3) were near to disagree; 7% (1)
was neutral; 14% (2) were near to agree, 29% (4) were agree and 21% (3) were strongly
agree. On the other side, the iPad results showed that 22% (2) were disagree; 22% (2)
were neutral; 11% (1) were near to agree and 44% (4) were strongly agree. There was no
signi�cant di�erence (p = 0.206 > 0.05) between the user's opinions for the iPhone and
iPad.
The arrangement of alphabets on the keypad is in accordance to my choice. The iPhone
results showed that 14% (2) were strongly disagree, 7% (1) was near to disagree; 21% (3)
were near to agree, 14% (2) were agree and 43% (6) were strongly agree. The results of
the iPad showed: 10% (1) was strongly disagree, 10% (1) was disagree, 10% (1) was near
to disagree; 10% (1) was neutral; 30% (3) were near to agree and 30% (3) were agree.
The comparison between the user's point of views for the iPhone and iPad was showed
no signi�cance (p = 0.080 > 0.05).
Small keys can cause typing mistakes. The iPhone results showed that 7% (1) was
strongly disagree, 14% (2) were disagree, 7% (1) was near to disagree; 14% (2) were
neutral; 21% (3) were near to agree, 21% (3) were agree and 14% (2) were strongly
agree. Similarly the iPad results showed that 10% (1) was disagree, 20% (2) were near
to disagree; 50% (5) were neutral; 10% (1) was agree and 10% (1) was strongly agree.
The comparison of the user opinions about the iPhone and iPad showed that there was
no signi�cance (p = 0.183 > 0.05).
I can easily �nd the number (e.g. 1, 2, 3...) on the keypad. The iPhone results showed
that 8% (1) was disagree; 15% (2) were neutral; 15% (2) were near to agree, 8% (1) was
agree and 54% (7) were strongly agree. On the other side, the iPad results showed that
20% (2) were near to disagree; 30% (3) were near to agreed, 30% (3) were agreed and
20% (2) were strongly agreed. The relationship between the iPhone and iPad responses
showed that there was no signi�cance(p = 0.129 > 0.05).
The single character keypad improves my writing performance. The iPhone results
showed that 7% (1) was strongly disagree, 7% (1) was disagree, 14% (2) were near to
disagree; 7% (1) was neutral; 29% (4) were near to agree, 14% (2) were agree and 21%
(3) were strongly agree. Similarly, the iPad results showed that 10% (1) was strongly
disagree, 10% (1) was near to disagree; 40% (4) were neutral; 30% (3) were near to agree
and 10% (1) was agree. The comparative study showed that there was no signi�cance(p =
0.143 > 0.05) between user's opinions for the both devices.
I can easily move the cursor forward and backward in the editor while writing text.
44
The iPhone results showed that 14% (2) were strongly disagree, 7% (1) was disagree, 36%
(5) were near to disagree; 7% (1) was neutral; 14% (2) were near to agree, 14% (2) were
agree and 7% (1) was strongly agree. Alike, the iPad results showed that 22% (2) were
disagree, 22% (2) were near to disagree; 22% (2) were neutral and 33% (3) were near to
agree. The comparative study showed that there was no signi�cance (p = 0.500 > 0.05)
in user's opinions for the both devices. The Figure 20 represented the graphs results of
users opinions against each statement of the Keypad on the iPhone and iPad respectively.
Figure 21: Graph Shows Percentage Results of the Degree of Agreement Across the EachStatement of the Keypad on the iPhone.
45
Figure 22: Graph Shows Percentage Results of the Degree of Agreement Across the EachStatement of the Keypad on the iPad.
4.3.4 Applications Results
The fourth section of the questionnaire named �Applications� contained 13 statements.
The results of each statement for the both devices are presented as follows.
The applications (e.g. Evernote, calendar) are simple. The iPhone results presented
that 29% (4) were near to disagree; 7% (1) was neutral; 29% (4) were near to agree, 14%
(2) were agree and 21% were strongly agree. On the other side, the iPad results showed
that 11% (1) was near to disagree; 44% (4) were near to agree, 33% (3) were agree and
11% (1) was strongly agree. The relationship between the users opinions regarding the
iPhone and iPad was not signi�cant (p = 0.385 > 0.05).
The applications (e.g. Facebook, calendar) work very well in portrait style. The iPhone
results showed that 7% (1) was disagree, 7% (1) was near to disagree; 21% (3) were
near to agree, 36% (5) were agree and 29% (4) were strongly agree. Similarly, the iPad
results showed that 44% (4) were near to agree, 22% (2) were agree and 33% (3) were
strongly agree. The comparative of users views showed that there was no signi�cant
di�erence(p = 0.477 > 0.05) for the iPhone and iPad.
I think the applications can easily change from portrait style to landscape style. The
iPhone results showed that 7% (1) was disagree; 14% (2) were neutral; 21% (3) were near
to agree, 29% (4) were agree and 29% (4) were strongly agree. On the other side, the
iPad results presented that 11% (1) was near to disagree; 11% (1) was neutral; 44% (4)
were near to agree, 22% (2) were agree and 11% (1) was strongly agree. The relationship
46
within the users views showed that there was no signi�cance(p = 0.201 > 0.05) for the
iPhone and iPad.
I can easily understand the symbols in the applications. The iPhone results showed
that 14% (2) were near to disagree; 14% (2) were neutral; 29% (4) were near to agree,
29% (4) were agree and 14% (2) were strongly agree. The iPad results showed that 30%
(3) were near to disagree; 20% (2) were neutral; 20% (2) were near to agree and 30% (3)
were agree. Similarly, the comparative study for the users opinions about the iPhone and
iPad showed that there was no signi�cance (p = 0.098 > 0.05).
The navigation on the applications is simple. The iPhone results showed that 15% (2)
were neutral; 23% (3) were near to agree, 23% (3) were agree and 38% (5) were strongly
agree. On the other hand, the iPad results showed that 10% (1) was near to disagree;
30% (3) were neutral; 20% (2) near to agree and 40% (4) were agree. The comparison
showed there was a signi�cance (p = 0.048 < 0.05). Result of the users opinions shown
that they preferred the simplicity of navigation on the applications on the iPhone than
on the iPad.
The navigation labels are clear and concise. The iPhone results presented that 7% (1)
was near to disagree; 14% (2) were neutral; 29% (4) were near to agree, 14% (2) were
agree and 36% (5) were strongly agree. The iPad results showed that 10% (1) was disagree
10% (1) was near to disagree; 10% (1) was neutral; 30% (3) were near to agree and 40%
(4) were agree. On the other hand, the comparative study of the user's opinions for the
both devices was not signi�cance (p = 0.169 > 0.05).
The navigation on the applications is similar. The iPhone results showed that 21%
(3) were disagree; 29% (4) were neutral; 14% (2) were near to agree, 21% (3) were agree
and 14% (2) were strongly agree. The iPad results presented that 20% (2) were near to
disagree; 20% (2) were neutral; 30% (3) were near to agree and 30% (3) were agree. The
comparative study was also showed no signi�cance (p = 0.489 > 0.05).
The text on the applications is easy to read. The iPhone results showed that 7% (1)
was near to disagree; 7% (1) was neutral; 36% (5) were near to agree, 14% (2) were agree,
36% (5) were strongly agree. On the other side, the iPad results presented that 10% (1)
was disagree; 50% (5) were near to agree, 30% (3) were agree and 10% (1) was strongly
agree. The comparison of the user opinions for the iPhone and iPad presented that there
was no signi�cance (p = 0.183 > 0.05).
I recover easily when I commit a typing mistake. The iPhone results presented that
31% (4) were neutral; 31% (4) were near to agree, 15% (2) were agree and 23% (3) were
strongly agree. Similarly, the iPad results showed that 10% (1) was disagree, 20% (2)
were near to disagree; 10% (1) was neutral; 40% (4) were near to agree and 20% (2) was
agree. The comparative results showed that there was no signi�cance(p = 0.069 > 0.05)
in users opinions for both devices.
It is easy to �nd the information (help, pop-up messages) I need. The iPhone results
47
showed that 8% (1) was strongly disagree, 8% (1) was disagree, 23% (3) were near to
disagree; 8% (1) was neutral; 31% (4) were near to agree, 8% (1) was agree and 15%
(2) were strongly agree. On the other side the iPad results presented that 20% (2) were
disagree, 10% (1) was near to disagree; 20% (2) were neutral; 30% (3) were near to agree
and 20% (2) were agree. The comparative study of the user views for the both devices
presented there was no signi�cance(p = 0.333 > 0.05).
The applications often support the zooming gestures. The iPhone results presented
that 7% (1) was disagree, 14% (2) were near to disagree; 21% (3) were neutral; 21% (3)
were near to agree, 21% (3) were agree and 14% (2) were strongly agree. On the iPad
side, 10% (1) was near to agree; 40% were neutral; 20% (2) were near to agree and 30%
(3) were agree. The comparative study of the users opinion about the iPhone and iPad
showed there was no signi�cance (p = 0.467 > 0.05).
I like swiping the page (up, down, left, right) of the newspaper. The iPhone results
showed that 14% (2) were strongly disagree, 14% (2) were near to disagree; 7% (1) was
near to agree, 29% (4) were agree and 36% (5) were strongly agree. On the other side,
the iPad results showed that 10% (1) was disagree; 20% (2) were neutral; 30% (3) were
agree and 40% (4) were strongly agree. The relationship of the users opinion presented
that there was no signi�cance (p = 0.424 > 0.05) for the iPhone and iPad.
It is easy to browse the web in the Safari browser. The iPhone results presented
that 14% (2) were strongly disagree; 7% (1) was neutral; 29% (4) were near to agree,
21% (3) were agree and 29% (4) were strongly agree. Similarly, the iPad results showed
that 20% (2) were near to agree, 40% (4) were agree and 40% (4) were strongly agree.
The comparative study of the users opinions presented that there was no signi�cant
di�erence(p = 0.155 > 0.05) for the iPhone and iPad. Figure 21 presented the graphs
results of the user opinions across each statement for both devices.
48
Figure 23: Percentage Graph Results of the Degree of Agreement Across Each Applica-tions Statement on the iPhone.
Figure 24: Percentage Graph Results of the Degree of Agreement Across the Each State-ment of the Applications on the iPad.
49
4.3.5 Participant Comments
Number of Comments for the iPhone Number of Comments for the iPad
Feature/Participant Comments Positive Negative Positive Negative
Frequency % Frequency % Frequency % Frequency %
As a Device (Overall satisfaction) 8 57 0 0 6 60 0 0
Graphics 5 36 0 0 6 60 0 0
Keypad 3 21 4 29 7 70 0 0
Application Crashes 0 0 2 14 0 0 1 10
Automatic Uninstallation of Applications 0 0 1 7 0 0 0 0
Battery Time 0 0 3 21 0 0 5 50
Blue-tooth Connectivity 0 0 5 36 0 0 4 40
Consistency in Applications 0 0 5 36 0 0 4 40
Help 0 0 4 29 0 0 3 30
Signal Strength 0 0 0 0 0 0 3 30
Stuck in Recovery Mode 0 0 2 14 0 0 0 0
Wireless Connectivity 0 0 0 0 0 0 4 40
Table 4: Results of Surveyed Participants Comments Against the Di�erent Features ofthe iPhone and iPad.
Di�erent types of comments were received from the participants. The authors divided
these comments into positive and negative for di�erent features as shown in Table 5.
As a device 57% on iPhone and 60% for iPad were given the positive comments by the
respondents, and no negative comment was received. It means that the participants are
satis�ed with these devices. Similarly for the graphics of these devices 36% and 60%
participants had positive comments for iPhone and iPad respectively. The ratio on the
keypad of both devices was di�erent. 21% and 70% participants liked the keypad as well as
29% negative comments were also received for iPhone. The reasons for negative comments
on the iPhone were small keys and small target area as compared to the iPad and missing
the swype feature (e.g. �NOKIA N8� Android system) . Due to this particular feature
(swype) two users gave up the iPhone and bought an Android system.
Battery time of the iPhone and iPad were also commented by the participants and
the ratio of negative comments of both devices were 21% and 50%. These comments
include both the battery timing and long charging time especially for iPad. 30 % of the
participants were also given the negative comments for the Blue-tooth connectivity on
the iPhone and for the iPad 40% were given the negative comments. The main problem
with the Blue-tooth of the iPhone and iPad is that it creates problem when connecting
to the other company devices ( e.g. Nokia, Samsung and Simon) . The consistency issue
is mostly discussed for the iPhone and iPad in every usability study of these devices.
36% negative comments were received for the iPhone and 40% for iPad. Three users i.e.
30% mentioned the signal strength problem for the iPad. The issue was that it shows a
weak signal even if someone is using the iPad very close to the router. Missing help was
50
also mentioned by the participants. 29% of the iPhone and 30% of the iPad participants
commented negatively as they mentioned that for majority of the problem we need to
Google for the issues. This problem may su�er those participants that are using 3G
limited connection.
Negative comments were also received for the applications crashes and automatically
uninstallation. The ratio for crash application was 20% and 10% for iPhone and iPad
respectively and for the auto uninstallation was 7% only for the iPhone.
4.4 Measuring Usability Attributes
The empirical usability measuring reported in this section focuses on the usability metrics
such as �task time�, �error� and �satisfaction�. It was conducted to �nd out the perfor-
mance di�erences between the device levels (iPhone versus iPad) and among the Partic-
ipant group levels (novice user, experienced user, elderly user) across the device levels.
So to measure the performance it is necessary to examine the task time and number of
mistakes or errors for each participant against each task.
4.4.1 Task Time
Task / Mean & p-value
Participant Group Level Device Level
Novice User Experienced User Elderly User p-value iPhone iPad p-value
mean(sec) mean(sec) mean(sec) mean(sec) mean(sec)
Facebook Login 113.4 43.8 142.7 1.625E−22 112.4 87.6 0.001
Send a Message 122.1 56.3 147.3 3.438E−19 119.2 97.8 0.002
Location Identi�cation 136.9 63.5 160.7 1.91E−18 128.7 112.1 0.030
Location Close View 178.2 103.5 212.3 2.11E−16 185.6 143.7 9.718E−06
New Note 170.1 95.9 181.4 2.41E−18 166.3 131.9 2.7E−06
Evernote Logout 26.6 19.8 31.2 3.191E−06 28.4 23.2 0.003
Table 5: Results from the ANOVA Tests for Estimating the Signi�cance Among theGroup Levels and Between the Device Levels Across Each Task. The p-value Shown theSigni�cance Among the Mean Values of Each Participant Group Against the Each Task.However, the Bold (p-values) Values Shown the Signi�cant Di�erences Among the Groupsand Between the Devices.
51
Participant GroupTask
iPhone iPad
Level Mean (Sec) p-value Mean (Sec) p-value
Facebook Login 129.6 47.8 4.174E−11 97.2 39.9 1.62E−09
Novice Users Send a Message 134.1 60.4 1.38E−07 110.1 52.1 2.14E−07
vs. Location Identi�cation 146.3 59.9 2.21E−10 127.5 67.1 4.86E−05
Experienced Users Location Close View 194.7 128.9 0.001 161.7 78.1 1.4E−07
New Note 204.9 103.2 5.13E−11 135.2 88.6 6.57E−06
Evernote Logout 32.8 19.1 0.001 20.3 20.6 0.414
Facebook Login 129.6 159.8 0.023 97.2 125.6 0.014
Novice Users Send a Message 134.1 163.2 0.031 110.1 131.3 0.059
vs. Location Identi�cation 146.3 179.8 0.012 127.5 141.6 0.206
Elderly Users Location Close View 194.7 233.3 0.014 161.7 191.3 0.041
New Note 204.9 190.8 0.185 135.2 171.8 0.005
Evernote Logout 32.8 33.5 0.438 20.3 28.8 0.001
Facebook Login 47.8 159.8 1.228E−08 39.9 125.6 7.02E−08
Experienced Users Send a Message 60.5 163.2 7.99E−09 52.1 131.2 1.93E−07
vs. Location Identi�cation 59.9 179.8 6.1E−10 67.1 141.5 2.1E−06
Elderly Users Location Close View 128.9 233.3 1.43E−06 78.1 191.3 5.44E−09
New Note 103.2 190.8 6.04E−07 88.6 171.8 1.22E−07
Evernote Logout 19.1 33.5 8.59E−05 20.6 28.8 0.001
Table 6: Participant Group Levels Comparison Results. The p-values Showed the Di�er-ences Between the Participant Group Levels on the iPhone and iPad. However, the Boldp-values Shown that there was a Signi�cant Di�erence Between the Group Levels.
Task/Device Novice User Experienced User Elderly User
Level & p-value iPhone iPad p-value iPhone iPad p-value iPhone iPad p-value
(mean) (mean) (mean) (mean) (mean) (mean)
Facebook Login 129.6 97.2 0.001 47.8 39.9 0.022 159.8 125.6 0.007
Send a Message 134.1 110.1 0.030 60.5 52.1 0.005 163.2 131.2 0.001
Location Identi�cation 146.3 127.5 0.108 59.9 67.1 0.085 179.8 141.6 0.001
Location Close View 194.7 161.7 0.021 128.9 78.1 0.001 233.3 191.3 0.021
New Note 204.9 135.2 3.81E−09 103.2 88.6 0.001 190.8 171.8 0.048
Evernote Logout 32.8 20.3 0.001 19.1 20.6 0.179 33.5 28.8 0.102
Table 7: Device Levels Means Comparisons Results. The p-value Showed the Di�erencesBetween the Means of the iPhone versus iPad Against each User Group. However, theBold p-values Shown the Signi�cant Di�erence.
1. Facebook Login Results In the experiment, the �rst task was �facebook login�
using the prede�ned user name and password. To analyze the statistical data, the authors
used three methods such as two way ANOVA, unpaired t-test and paired t-test. In the �rst
step, the two way ANOVA test was used to see if there was a signi�cant di�erence between
52
the device levels and among the participant group levels. After analyzing the data, the
authors found that mean di�erences of participant group levels (113.4, 43.8, 142.7) across
the device variable was statistically signi�cant(p = 1.625E−22 < 0.05). Further, the
unpaired t-test was used to see where the di�erences within the participant group levels
across the each device level were. The mean di�erences with the comparison of the novice
user versus experienced user using the iPhone [(129.6 vs. 47.8) & p = 4.174E−11 < 0.05]
and iPad [(97.2 vs. 39.9) &p = 1.62E−09 < 0.05] were signi�cant. The results showed
that the experienced user performed better using the iPhone and iPad than the novice
user. Similarly, the mean di�erences with the comparison of the novice user versus elderly
user using the iPhone [(129.6 vs. 159.8) & p = 0.023 < 0.05] and iPad [(97.2 vs. 125.6)
& p = 0.014 < 0.05] were signi�cant. In this comparison the results showed that the
novice user performed the task faster than the elderly user at both devices. Moreover,
the mean di�erences with the comparison of the experienced user versus elderly user
using the iPhone [(47.8 vs. 159.8) &p = 1.228E−08 < 0.05] and iPad [(39.9 vs. 125.6)
&p = 7.02E−08 < 0.05] were signi�cant. The results showed that the experienced user
performed the task at both devices faster than the elderly user.
Again, the ANOVA test was used to see if there was a signi�cant di�erence between
the device levels across the participant group variable. The di�erence between mean of
the device levels (112.4 vs. 87.6) across the participant group variable was signi�cant (p =
0.001 < 0.05). Paired t-test was used to �nd further where the di�erences in the device
levels across the participant's group variable were. After analyzing the results showed
that the mean di�erence with the comparison of device levels (129.6 vs. 97.2) across the
novice user was signi�cant (p = 0.001 < 0.05). The statistical results showed that the
performance of the iPad was better than the iPhone across the novice user. Likewise the
comparison of the device levels (47.8 vs. 39.9) against the experienced user was signi�cant
(p = 0.022 < 0.05). The result showed that here also the performance of iPad is better
than the iPhone against the experienced user. In the next mean comparison of the device
levels (159.8 vs. 125.6) across the elderly user was also signi�cant(p = 0.007 < 0.05). The
result showed that iPad performance was better than the iPhone for the elderly user.
The Figure 22, the graph shown the means di�erences [(129.6, 97.2), (47.8, 39.9),
(159.8, 125.6)] in performance of the each participant group level across the device levels.
53
Figure 25: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task Facebook Login.
2. Send a Message Results The second task named �send a message� was also related
to the Facebook application. The participant's task was to write and send a text message
using a speci�ed email address. Two way ANOVA was used to see the di�erences among
the participant group levels and between the device levels. The di�erences in the means
of participant group levels (122.1, 56.3, 147.3) across the device variable were signi�cant
(p = 3.438E−19 < 0.05). Further unpaired t-test was used to �nd where the signi�cant
di�erences with the comparison of each participant group level across each level of device
variable were. The di�erences with the comparison of the novice user versus experienced
user using the iPhone [(134.1 vs. 60.4) & p = 1.38E−07 < 0.05] and iPad [(110.1 vs.
52.1) & p = 2.14E−07 < 0.05] were statistically signi�cant. The results showed that the
performance of the experienced user was better than the novice user on both devices.
Likewise the mean di�erences for the comparison of the novice user versus elderly user
using the iPhone (134.1 vs. 163.2) was signi�cant(p = 0.031 < 0.05) but using the
iPad (110.1 vs. 131.3) was not statistically signi�cant (p = 0.059 > 0.05). The results
showed that novice user performed better than the elderly user using the iPhone but on
the iPad, the performance of both groups was almost equal. The mean di�erences with
the comparison of the experienced user versus elderly user using the iPhone [(60.5 vs.
163.2) &p = 7.99E−09 < 0.05] and iPad [(52.1 vs. 131.2) & p = 1.93E−07 < 0.05] were
signi�cant. The performance of the experienced user was better than the elderly user
using both devices.
The di�erence in the means of device levels (119.2 vs. 97.8) across the participant
group variable was statistically signi�cant (p = 0.002 < 0.05). Paired t-test was used
to see where the di�erences between the device levels across each Participant group level
were. The di�erence in means of the device levels (134.1 vs. 110.1) across the novice user
54
was signi�cant (p = 0.030 < 0.05). The result showed that the performance of the iPad
was better than the iPhone while using the novice user. Alike, the di�erence in means
of device levels (60.5 vs. 52.1) across the experienced user was statistically signi�cant
(p = 0.005 < 0.05). The means result showed that the iPad performance was better than
the iPhone while using the experienced user. The di�erence in the means of the device
levels (163.2 vs. 131.2) across the elderly user was also signi�cant (p = 0.001 < 0.05).
The statistical result showed that the performance of the iPad was better than the iPhone
while using the elderly user.
In Figure 23, the graph represented the means di�erences [(134.1, 110.1), (60.4, 52.1),
(163.2, 131.3)] in the performances of each group level across the each device level.
Figure 26: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task Send a Message.
3. Location Identi�cation Results The third task named �location identi�cation�
was related with the Google mobile map application. The participant's task was to locate
Blekinge Tekniska Högskola (BTH) using the Google mobile map application. First, the
two way ANOVA was used to see if there was a statistically signi�cance among the partic-
ipant group levels and between the device levels. After analyzing, the mean di�erences in
the participant group levels (136.9, 63.5, 160.7) across the device variable were statistically
signi�cant (p = 1.9E−18 < 0.05). Further, Unpaired t-test was used to see the di�erence
within the participant group levels against each level of device variable. The di�erences
between means of the novice user versus experienced user using the iPhone [(146.3 vs.
59.9) & p = 2.21E−10 < 0.05] and iPad [(127.5 vs. 67.1) & p = 4.86E−05 < 0.05] were
statistically signi�cant. The statistical results showed that the experience user performed
better than the novice user on both the devices. In the novice user versus elderly user
comparison, the means di�erences using the iPhone [(146.3 vs. 179.8) was statistically
55
signi�cant (p = 0.012 < 0.05) but using the iPad (127.5 vs. 141.6) was not statistically
signi�cant (p = 0.206 > 0.05). Here, the performance of the novice user was better than
elderly user, however using the iPad, there was no signi�cant di�erence in the performance
of both groups. The means di�erences for the comparison of the experienced user versus
elderly user using the iPhone [(59.9 vs. 179.8) & p = 6.1E−10 < 0.05] and iPad [(67.1 vs.
141.5) &p = 2.1E − 06 < 0.05] were statistically signi�cant. The results showed that the
performance of the experienced user was better than the elderly user using both devices.
The di�erences in the means of device levels (128.7 vs. 112.1) across the participant
group variable was signi�cant (p = 0.03 < 0.05). The authors used the paired t-test
to investigate where the signi�cant di�erences between the device levels across the each
participant group level were. The means di�erence of the device levels (146.3 vs. 127.5)
across the novice user was not statistically signi�cant (p = 0.108 > 0.05). The result
showed that the performance of the iPhone and iPad was almost same while using the
novice user. Against the experienced user the means comparison of the device levels (59.9
vs. 67.1) was not statistically signi�cant (p = 0.085 > 0.05). Here again the Performance
of the iPhone and iPad was identical across the experienced user. However, the mean
di�erences of device levels (179.8 vs. 141.6) across the elderly user were statistically
signi�cant (p = 0.001 < 0.05). The result showed that the performance of the iPad was
better than the iPhone while using the elderly people.
In Figure 24, the graph represented the means di�erences [(146.3, 127.5), (59.9, 67.1),
(179.85, 141.55)] of each participant group level across the each device level.
Figure 27: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task Location Identi�cation.
4. Location Close View Results The fourth task named �location close view� was
also related with the Google mobile map application. The participant's task was to select
56
the satellite view and made the close view of BTH using the �zoom in� gestures on both the
devices. After analysis the results showed that the di�erence in the average time taken by
the participant group levels (178.2, 103.5, 212.3) over the device variable was statistically
signi�cant (p = 2.11E−16 < 0.05). Unpaired t-test was used to �nd where the signi�cant
di�erences in the comparison of the group levels were. The comparison of the novice user
versus experienced user using the iPhone [(194.7 vs. 128.9) & p = 0.001 < 0.05] and
iPad [(161.7 vs. 78.1) &p = 1.4E−07 < 0.05] was statistically signi�cant. The results
showed that the experienced user performance was better than the novice user using
the iPhone and iPad. Similarly, the comparison of the novice user versus elderly user
using the iPhone [(194.7 vs. 233.3) & p = 0.014 < 0.05] and iPad [(161.7 vs. 191.3)
&p = 0.041 < 0.05] was signi�cant. The performance of novice user on both devices was
better than the elderly user. Another comparison of the experienced user versus elderly
user using the iPhone [(128.9 vs. 233.3) & p = 1.43E−06 < 0.05] and iPad [(78.1 vs.
191.3) &p = 5.44E−09 < 0.05] was also statistically signi�cant. The results showed that
the experienced user took less time than the elderly user to perform the task by using
both the devices.
In the device levels comparison, the means di�erences of the device levels (185.6 vs.
143.7) across the group variable was statistically signi�cant (p = 9.72E−06 < 0.05). Paired
t-test was used to see the di�erences between the device levels against the each level of the
participant group variable. The comparison of the device levels (194.7 vs. 161.7) against
the novice user was statistically signi�cant (p = 0.021 < 0.05). The results showed that
the performance of the iPad was better than the iPhone while using the novice user.
The comparison of the device levels (128.9 vs. 78.1) against the experienced user was
signi�cant (p = 0.001 < 0.05). In this case the performance of the iPad was also better
than the iPhone. In the next comparison of the device levels (233.3 vs. 191.3) against
the elderly user was also signi�cant (p = 0.021 < 0.05). The iPad performance was better
than the iPhone while using the elderly user.
In Figure 25, the graph represented the means [(194.7, 161.7), (128.9, 78.1), (233.3,
191.3)] di�erences of participant group levels across the device levels.
57
Figure 28: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task Location Close View.
5. New Note Results The �fth task named �new note� was related to the Evernote
application. The participant's tasks were login to the application using the prede�ned
username and password, write a prede�ned text and to saved it. After analyzing the users
data, the authors found that the di�erences in average taken time by the participant
group levels (170.1, 95.9, 181.4) across the device variable was statistically signi�cant
(p = 2.41E−18 < 0.05). Unpaired t-test was used to see where the di�erences between the
participant group levels were. The means di�erences of the novice user versus experienced
user using the iPhone [(204.9 vs. 103.2) & p = 5.13E−11 < 0.05] and iPad [(135.2
vs. 88.6) & p = 6.57E−06 < 0.05] were statistically signi�cant. The results showed
that the experienced user performed better than the novice user on both the devices.
Likewise, the means di�erences of the novice user versus elderly user using the iPhone
(204.9 vs. 190.8) was not statistically signi�cant (p = 0.185 > 0.05). However using the
iPad (135.2 vs. 171.8), there was a signi�cant di�erence (p = 0.005 < 0.05).The results
showed that there was no di�erence in the performance of the novice user and elderly user
while using the iPhone. However using the iPad, the novice user performance was better
than the elderly user. The means di�erences of the experienced user versus elderly user
using the iPhone [(103.2 vs. 190.8) & p = 6.04E−07 < 0.05] and iPad [(88.6 vs. 171.8)
&p = 1.22E−07 < 0.05] were also statistically signi�cant. The experienced user performed
better than the elderly user on both devices.
The means di�erences between the device levels (166.3, 131.9) across the participant
group variable were statistically signi�cant (p = 2.7E−06 < 0.05). Further, paired t-test
was used to see where the di�erences between the device levels across the each participant
group level were. The means di�erence of the device levels (204.9 vs. 135.2) across the
novice user was statistically signi�cant (p = 3.81E−09 < 0.05). The result showed that
58
the performance of the iPad was better than the iPhone while using the novice user. The
means di�erence of the device levels (103.2 vs. 88.6) across the experienced user was
statistically signi�cant (p = 0.001 < 0.05). In this case, the performance of the iPad was
better than the iPhone. The means di�erence of the device levels (190.8 vs. 171.8) across
the elderly user was statistically signi�cant (p = 0.048 < 0.05). The results showed that
the performance of the iPad was better than the iPhone while using the elderly user.
The Figure 26, the graph represented the means di�erences [(204.9, 135.2), (103.2,
88.6), (190.8, 171.8)] of the participant group levels across the device levels.
Figure 29: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task New Note.
6. Evernote Logout Results The last task named �Evernote logout� was also related
to the Evernote application. The participant's task was to logout by locating it from the
menu options. After analyzing the data, the means di�erences of the participant group
levels (26.5, 19.8, 31.2) were statistically signi�cant (p = 3.19E−06 < 0.05). Unpaired
t-test was used to see the di�erence between the group levels against each device level.
The means di�erence of the novice user versus experienced user using the iPhone (32.8,
19.1) was statistically signi�cant (p = 0.001 < 0.05). However, against the iPad (20.3,
20.6), there was no signi�cant di�erence(p = 0.414 > 0.05). The results showed that the
experienced user performed better than the novice user using the iPhone but on the iPad,
there was no statistical di�erence in the performance of the both groups. The means
comparison of the novice user versus elderly user using the iPhone (32.8, 33.5) was not
statistically signi�cant (p = 0.438 > 0.05) but on the iPad (20.3, 28.8), there was a
signi�cant di�erence(p = 0.001 < 0.05). In this comparison, the results showed that the
performance of the both groups on the iPhone was statistically identical, however on the
iPad, the novice user performed better than the elderly user. The means di�erences of the
59
experienced user versus elderly user using the iPhone [(19.1, 33.5) & p = 8.59E−05 < 0.05]
and iPad [(20.6, 28.8) &p = 0.001 < 0.05] were statistically signi�cant. The performance
of the experienced user was better than the elderly user on the iPhone and iPad.
The means di�erences in the comparison of the device levels (28.4 vs. 23.2) were sta-
tistically signi�cant(p = 0.003 < 0.05). Paired t-test was used to see where the signi�cant
di�erences between the means of the iPhone and iPad across the participant group levels
were. The means di�erence in the device levels (32.8 vs. 20.3) across the novice user was
statistically signi�cant (p = 0.001 < 0.05). The result showed that the performance of
the iPad was better than the iPhone while using the novice user. The means di�erence in
the device levels (19.1 vs.20.6) across the experienced user was not statistically signi�cant
(p = 0.179 > 0.05). The results showed that the iPhone and iPad performed similar while
using the experienced user. The means di�erence of the device levels (33.5 vs. 28.8) across
the elderly user was not statistically signi�cant(p = 0.102 > 0.05). The statistical result
showed that the iPhone and iPad performed similar while using the elderly user.
The Figure 27, the graph represented the means di�erences of the participant group
levels [(32.8, 20.3), (19.1, 20.6), (33.5, 28.8)] across the device levels.
Figure 30: Device Levels Comparison Graph of the Performance Measurement of theParticipant Group Levels for the Task Evernote Logout.
60
4.4.2 Accuracy Measurement
To measure the accuracy, the authors noted down the number of mistakes such as wrong
selection, typing mistake, Zooming and dictionary problems. Paired t-test was used to
analyze the mistakes data and to measure the signi�cant di�erence of each type of mistake
on the device levels (iPhone and iPad). Table 9 shows the mean results of the mistakes
made by each group of the participant and signi�cant di�erence on both the devices.
Group/ Mistake
Wrong Selection Typing mistake Zooming Dictionary
Mean & p- value Mean & p- value Mean & p- value Mean & p- value
iPhone iPad p iPhone iPad p iPhone iPad p iPhone iPad p
Novice User 8.7 6.2 0.108 39.3 28.5 0.007 6.5 5.5 0.182 24.8 23.8 0.580
Experienced User 3.0 3.3 0.360 4.7 4.5 0.416 1.0 0.5 0.182 0 0 0
Elderly user 31.7 26.2 0.051 52.7 38.2 0.005 11.5 9.0 0.182 34.8 29.3 0.029
Table 8: Means Results and Di�erences of Each Type of Mistake Made by Each UserGroup on the iPhone and iPad. However, the Bold p-values Shown the Signi�cant Di�er-ence.
1. Wrong Selection In the wrong selection mistake, the participants were often se-
lected the wrong application (e.g. most of the participants chose the Notes application
instead of the Evernote), chose the wrong option to performed further steps in a task.
The means di�erences of the wrong selection mistakes across the iPhone versus iPad
made by the novice user [(8.7 vs. 6.2) &p = 0.108 > 0.05], experienced user [(3.0 vs.3.3)
& p = 0.360 > 0.05] and elderly user [(31.7 vs. 26.2) &p = 0.051 > 0.05] were not statis-
tically signi�cant. The results showed that the novice user, experienced user and elderly
user made the wrong selection mistakes equally on the iPhone as well as on the iPad.
2. Typing Mistake The accidental mistakes, those a participant committed while
performing a task. The participant who had not the awareness with the touch screen
system; they did make the maximum number of typing mistake. The means di�erence
of the typing mistake across the device levels (39.3 Vs 28.5) made by the novice user
was statistically signi�cant (p = 0.007 < 0.05). This means, the novice user made less
typing mistakes using the iPad than using the iPhone. The means di�erence of the typing
mistakes across the device levels (4.7 vs. 4.5) made by the experienced user was not
statistically signi�cant(p = 0.416 > 0.05). The result showed that the experienced user
made typing mistakes equally on the iPhone as well as on the iPad. However, the means
di�erence of the typing mistake across the device levels (52.7 vs. 38.2) made by the elderly
user was statistically signi�cant (p = 0.005 < 0.05). The results showed that the elderly
user made less typing mistakes using the iPad than using the iPhone.
61
3. Zooming In the experiment, there was one task where the zooming gesture would
be tested. The means di�erences of the wrong use of the zooming gestures across the
device levels; used by the novice user [(6.5 vs. 5.5) & p = 0.182 > 0.05], experienced user
[(1.0 vs.0.5) & p = 0.182 > 0.05] and elderly user [(11.5 vs. 9.0) & p = 0.182 > 0.05] were
not statistically signi�cant. The results showed that the three groups used the wrong
zooming gestures equally on the iPhone and on the iPad.
4. Dictionary During the experiment, the spell auto correction option was turn on.
Number of the participants typed the words by looking at the keypad. In most of the
cases auto correction function changed the written word and at the end participant would
not obtain the exact results.
The means di�erences of the auto correction dictionary problems across the devices
levels (24.8 vs. 23.8) faced by the novice user (p = 0.580 > 0.05) was not statistically
signi�cant. The results showed that the novice user faced dictionary problems equally on
the iPhone and iPad. However, the means di�erence of the auto correction dictionary
problems across the device levels (34.8 vs. 29.3) faced by the elderly user was statistically
signi�cant(p = 0.029 < 0.05). The result showed that the elderly user had more controlled
the auto correction dictionary problems on the iPad as compared to the iPhone. In Figures
28, 29 and 30, the graphs showed the di�erences of mistakes between the devices across
the novice user, experienced user and elderly user respectively.
Figure 31: Novice User Mistakes Results Graph on the iPhone and iPad.
62
Figure 32: Experienced User Mistakes Results Graph on the iPhone and iPad.
Figure 33: Elderly User Mistakes Results Graph on the iPhone and iPad.
63
4.4.3 Satisfaction Results
The basic purpose of the post test questionnaire was to measure the user's satisfaction
level about the overall performances of both systems. Each questionnaire consisted of 12
statements and each statement was same for both the devices. The Wilcoxon Matched
Paired Signed Ranks test was used to analyze the user's responses. Analysis and result
(Table 10) of each statement against each level of participant group variable are listed in
below.
Q
Novice User Experienced User Elderly User
Median STDEVp- value
Median STDEVp- value
Median STDEVp- value
iP id iP id iP id iP id iP id iP id
Q 1 6 6.5 .979 1.414 0.278 6.5 6.5 0.605 0.605 1.00 5 6 1.021 0.716 6.1E−05
Q 2 5 5 1.461 1.105 0.946 6 6 0.825 0.813 0.685 5 6 1.165 1.099 0.001
Q 3 6 7 0.973 1.293 0.339 6 6.5 0.587 0.801 0.812 5 6 1.174 0.967 6.6E−05
Q 4 4 5.5 1.552 1.188 0.001 5.5 6 1.137 1.170 0.082 5 6 1.294 1.050 0.008
Q 5 3.5 6 1.603 1.843 0.001 5 6 1.503 0.945 0.027 3 5.5 1.501 1.507 1.5E−05
Q 6 6 6 1.454 0.968 0.391 6 6 0.778 0.968 1.00 4 5.5 1.424 1.316 0.001
Q 7 6 4 1.273 1.031 0.035 6 6 0.923 1.040 0.461 5 6 1.164 0.745 0.001
Q 8 5 6 1.281 1.234 0.010 5 6 1.226 0.852 0.027 4 6 1.239 1.031 0.001
Q 9 5 6 1.196 1.005 0.002 6 6 1.234 1.119 0.339 4.5 6 1.309 0.725 0.001
Q 10 5 6 1.164 1.089 0.021 7 7 0.754 0.598 0.312 6 7 0.834 0.587 0.001
Q 11 5 6 1.518 1.277 0.057 6 6 1.026 1.137 0.301 6 7 0.834 0.761 0.012
Q 12 6 6.5 1.302 0.864 0.001 7 7 0.759 0.688 0.844 5 6 0.967 1.146 0.003
Table 9: Post Test Questionnaires Results in the Form of the Median, the StandardDeviation and the Di�erences Between the Users Opinions Against Each Question. How-ever, the Bold p-values Shown the Signi�cant Di�erence. Q, iP and id Represented theQuestion, iPhone and iPad Respectively.
1. Ease to Use The system is easy to use for opening applications. The medians
di�erences of the novice user responses [(6.0 vs. 6.5) &p = 0.278 > 0.05] and experienced
user responses [(6.5 vs. 6.5) & p = 1.00 > 0.05] across the iPhone versus iPad were not
statistically signi�cant. This showed that the novice user and experienced user found the
opening applications same on the iPhone as well as on the iPad. However, the medians
di�erence of the elderly user responses across the iPhone versus iPad (5.0 vs. 6.0) was
statistically signi�cant(p = 6.104E−05 < 0.05). The result showed that the elderly user
found easy to open the application using the iPad than using the iPhone.
2. System Design The system is designed for all age groups. The medians di�erences
of the novice user responses [(5.0 vs. 5.0) &p = 0.946 > 0.05] and experienced user re-
sponses [(6.0 vs. 6.0) &p = 0.685] across the iPhone versus iPad were not statistically sig-
64
ni�cant. The analysis showed that the novice user and experienced user preferred equally
both devices for all age groups. The medians di�erence of the elderly user responses across
the iPhone versus iPad (5.0 vs. 6.0) was statistically signi�cant (p = 0.001 < 0.05). The
results showed that the elderly user preferred the iPad.
3. Task Enjoyment In the post test questionnaire, two statements were concerned to
the task enjoyment.
I like using the interface of the system. This statement was related to the touch screen
interface and the user responses showed the degree of agreement about such statement.
The medians di�erences of the novice user responses [(6.0 vs.7.0) &p = 0.339 > 0.05]
and experienced user responses [(6.0 vs. 6.5) & p = 0.812 > 0.05] showed that there
were no signi�cant di�erence across the iPhone versus iPad. The result showed that
both the groups had the same preferences for both the devices interfaces. The elderly
user responses medians across the iPhone versus iPad (5.0 vs. 6.0) had the signi�cant
di�erence (p = 6104E−05 < 0.05). The results showed that they were enjoyed using the
iPad as compared to the iPhone.
I enjoy using the keypad of the system. The statistical analysis showed that medians
di�erences of the novice user responses [(3.5 vs. 6.0) & p = 0.001 < 0.05], experienced
user responses [(5.0 vs. 6.0) & p = 0.027 < 0.05] and elderly user responses [(3.0 vs. 5.5)
&p = 1.526E−05 < 0.05] across the iPhone versus iPad were statistically signi�cant. The
results of the three user group's responses presented that they enjoyed using the iPad
keypad and ranked it higher than the iPhone.
4. Zooming Gestures I can easily use the zoom in and zoom out signs (gestures).In
the experiment; the zooming gestures were used in the Google mobile map application.
The statistical analysis showed that the medians of the novice user responses [(4.0 vs. 5.5)
&p = 0.001 < 0.05) and elderly user responses [(5.0 vs. 6.0) & p = 0.008 < 0.05) across
the iPhone versus iPad were signi�cant. The results showed that both the groups easily
used the zoom in/out gestures on the iPad than on the iPhone. However the experienced
users responses medians di�erence across the iPhone versus iPad (5.5 vs. 6.0) was not
statistically signi�cant (p = 0.082 > 0.05). The result presented that the experienced user
performed the zooming gestures equally on the iPhone as well as on the iPad.
5. Simplicity The section named �simplicity� contained two related statements.
The applications (Evernote, Facebook) are simple. The medians di�erences of the
novice user ranking [(6.0 vs. 6.0) &p = 0.391 > 0.05] and experienced user ranking [(6.0
vs. 6.0) &p = 1.00 > 0.05] across the iPhone versus iPad were not statistically signi�cant.
The results showed that they found the applications equally simple on using the iPhone
and iPad. Conversely, the elderly user ranking medians across the iPhone versus iPad
65
(4.0 vs. 5.5) was statistically signi�cant (p = 0.001 < 0.05). The elderly user found the
applications simpler using on the iPad than using on the iPhone.
I can easily perform the login task. The medians di�erences of the novice user ranking
[(6.0 vs. 6.5) & p = 0.001 < 0.05] and elderly user ranking [(5.0 vs. 6.0) & p = 0.003 <
0.05] across the iPhone versus iPad were statistically signi�cant. The statistical results
showed that both user groups performed the login task easily using the iPad than the
iPhone. However the experienced user ranking medians di�erence across the iPhone versus
iPad (7.0 vs. 7.0) was not statistically signi�cant (p = 0.844 > 0.05). The result showed
that the experienced user found the login tasks same using the applications on both the
devices.
6. Applications Navigation The three statements were related with the understand-
ability of the navigation signs on the applications.
I can easily understand the symbols in the applications. The medians di�erences of the
novice user responses [(6.0 vs. 4.0) & p = 0.035 < 0.05] and elderly user responses [(5.0 vs.
6.0) &p = 0.001 < 0.05] across the iPhone versus iPad were statistically signi�cant. The
results showed that the novice user better understood the symbols of applications using
the iPhone. However the elderly user preferred the understandability of the application's
symbol on iPad. The medians di�erence of the experienced users ranking across the iPhone
versus iPad (6.0 vs. 6.0) was not statistically signi�cant (p = 0.461 > 0.05). The statistical
results of experienced user ranking showed that they understand the applications symbols
equally on the iPhone as well as on the iPad.
The navigation on the application is simple. The ranking medians di�erences of the
novice user [(5.0 vs. 6.0) & p = 0.01 < 0.05], experienced user [(5.0 vs. 6.0) & p = 0.027 <
0.05] and the elderly user [(4.0 vs. 6.0) & p = 0.001 < 0.05] across the iPhone versus iPad
were statistically signi�cant. Three user groups preferred the navigation simplicity on the
iPad.
Navigation labels are clear and concise. The ranking medians di�erences of the novice
user [(5.0 vs. 6.0) & p = 0.002 < 0.05] and elderly user [(4.5 vs. 6.0) & p = 0.001 < 0.05]
across the iPhone versus iPad were statistically signi�cant. Both the groups preferred the
navigation clarity on the iPad. However the experienced user ranking medians di�erence
across the iPhone versus iPad (6.0 vs. 6.0) was not statistically signi�cant (p = 0.339 >
0.05). The experienced user found the navigation labels clarity equally on both the devices.
The text on the applications is easy to read. The ranking medians di�erences of the
novice user [(5.0 vs. 6.0) &p = 0.021 < 0.05] and elderly user [(6.0 vs. 7.0) &p =
0.001 < 0.05] across the iPhone versus iPad were statistically signi�cant. Both the groups
preferred the visibility of the iPad than the iPhone. The experienced user ranking medians
di�erence across the iPhone versus iPad (7.0 vs. 7.0) was not statistically signi�cant
66
(p = 0.312 > 0.05). They found the text on the applications same using both devices.
However their ranking median showed that they preferred both the devices equally.
I recover easily when I commit a typing mistake. The statistical analysis showed that
the ranking medians di�erences of the novice user [(5.0 vs. 6.0) & p = 0.057 > 0.05] and
experienced user [(6.0 vs. 6.0) & p = 0.301 > 0.05] across the iPhone versus iPad were not
statistically signi�cant. The results showed that both the user groups tackle the typing
mistake equally against both devices. The ranking medians di�erence of the elderly user
across the iPhone versus iPad (5.0 vs. 6.0) was statistically signi�cant (p = 0.012 < 0.05).
The statistical result revealed that the elderly user handled the typing mistake in a better
way using the iPad than the iPhone.
Figure 34: The Graph Presents the Median Comparison of the Degree of AgreementAgainst Each Question by Three User Groups on the iPhone.
67
Figure 35: The Graph Presents the Median Comparison of the Degree of AgreementAgainst Each Question by Three User Groups on the iPad.
4.4.4 Participant Comments
Participant Number of comments for the iPhone Number of comments for the iPad
Comments Frequency % Frequency % Frequency % Frequency %
As a Device (Overall satisfaction) 22 37 0 0 30 50 0 0
Graphics 12 20 0 0 13 22 0 0
Keypad 13 22 8 13 18 30 0 0
Consistency in Applications 0 0 9 15 0 0 4 7
help 0 0 5 8 0 0 5 8
Visibility 0 0 6 10 0 0 3 5
Table 10: Frequency and Percentage Results of the User Comments on the Di�erentAspects of Both the Devices.
Table 11 shows the comments received from the participants in a post test questionnaires.
As a device 37% for iPhone and 50% for the iPad were given the positive comments by
the participants and no negative comments were received. It means that the satisfaction
level of these devices is high. The ratio of positive comments for graphics of these de-
vices was 20% and 22 % respectively. 22 % also commented positively for the keypad
of iPhone as well as 13 % negative comments. These comments were from novice and
68
elderly participant. The reason is small keypad (small keys) that cause the accidental
errors while using this keypad. Similarly the 30% positive comments were received for
the iPad keypad and no negative comments at all. The percentage negative comments on
the consistency of the iPhone and iPad was 15% and 7% respectively. It shows that the
consistency issues are more considered in the small devices. Missing help option was also
commented by the participant. Eight percent of the participants for both the iPhone and
iPad were interested to have help option for these devices. At last the visibility were also
commented, 10% and 5% negative comments were received for the iPhone an iPad. These
comments were mostly by the elderly participants that need to have more visible option
especially when the device is too small.
69
Chapter 5
5 CONCLUSIONS AND FUTURE WORK
5.1 Conclusions
The aim of this study was to evaluate and measure the comparative usability performances
of the iPhone an iPad. The preliminary focus of the study was to �nd the usability issues
regarding both the devices. Several di�erent methods have been used in the study when
investigating the answers of prede�ned research questions. The study in this thesis was a
literature review, a survey study and an empirical usability testing experiment.
In the literature review, the authors studied the di�erent usability issues such as
inconsistency in applications, applications crashes and accidental errors.The detail list of
the issues also presented in Table 2. However, the survey responses validated such issues
and highlighted some additional issues such as low battery time, Blue-tooth connectivity
problems, wireless connectivity problem, week signal strength and missing help. Two
participants commented on the missing of a swyping feature. According to them if the
iPhone and iPad had such feature then their typing performance should be improved.
A survey study was conducted to evaluate and measure the signi�cance in the four
main parts of the devices i.e. system, touch screen, keypad and applications. Each part
contained the multiple related statements that explored the di�erent usability aspects.
The detail and results of each statement are given in Table 4. After the statistical analysis,
the authors did not �nd a signi�cant di�erence between the statements regarding the
iPhone and iPad apart from in four statements. In the two statements (i.e. �nding a
new application in the Apple store and replacing the contents location on the interface)
the users agreed that the iPad performance was better than the iPhone. However, in
the other two statements (i.e. use of system in the sun light and zooming gestures) they
preferred the iPhone rather than the iPad. Its means 90.47% users addressed the same
issues in the iPhone and iPad. There is no di�erence in their preferences regarding the
iPhone and iPad. 4.76% users preferred the iPhone and 4.46% users preferred the iPad
in above mentioned conditions.
The empirical usability testing experiment focused on studying the usability perfor-
mances of the iPhone and iPad considering three target groups. The performances com-
parisons were conducted by comparing the di�erent participants groups and both devices.
In total 60 users participated in the empirical usability performance testing study. The
selection of the participants was based on their earlier experience with mobile phones
usage. The detail of the participant are given in Figure 13.
The comparisons results of the participant groups are shown in Table 7, the devices
in Table 8 and the errors on both devices in Table 9 provided the signi�cance in the
70
performances. The novice users versus experienced users comparison results across the
iPhone (all six tasks) and the iPad (�rst �ve tasks) presented that the experienced users
performed faster than the novice users with lower error rate. The results of the novice
users versus elderly user comparison presented that the novice user using the iPhone (
�rst four tasks) and using the iPad (facebook login, location close view, new note and
Evernote logout) performed faster than the elderly users with lower error rate. Similarly
the results in the last comparison of the experienced users versus elderly users shown that
the experienced users performed all the six tasks faster than the elderly users with lower
error rate. Based on such �nding the authors can conclude that the experienced users
performances were better than the other two groups on both devices. The authors can
also conclude that if someone is using the iPhone he/she could easily use the iPad.
In the devices comparison results, the performances of the iPad across the novice users
(in tasks facebook login, send a message, location close view, new note Evernote logout),
the experienced users (in tasks facebook login, send a message, location close view and new
note) and the elderly users (�st �ve tasks) were better than the iPhone with lower error
rate. Base on such �ndings of the empirical usability testing, the authors conclude that
in the controlled environment the performance of the iPad was better than the iPhone.
Three participant groups performed each task faster on the iPad than the iPhone apart
from in one task. The task named �location identi�cation� the iPhone and iPad performed
the same across the novice user and experienced user but across the elderly user, the iPad
performed better than the iPhone.
Table 10 presents the results of the satisfaction level of the participants in a control
environment for both devices. The satisfaction level acquired through the post test ques-
tionnaire during the experiment. The results of the novice users and the experienced users
presented that their satisfaction levels were higher for the iPad than the iPhone. The ex-
perienced users results shown that their satisfaction levels were same for the iPhone as
well as for the iPad. However, with the keypads performances they looked more satis�ed
on the iPad than the iPhone.
5.2 Future Work
This section presents the considerations for further research based on the �ndings and
conclusions from the work carried out in this research study.
There are several aspects of this study that could be addressed in future research
studies. In this thesis, the survey study was conducted on a small scale and with a small
sample size. So in the future it could be possible to conduct the study by increasing the
sample size of the population for more generalization of the results. The authors also
found in this study that very little work has been done in the comparative usability of
the iPhone and iPad considering elderly people. The elderly people are becoming a larger
71
part of the population in the world especially in the developed countries. Some times
they could feel loneliness and social isolation in the society. They could be engaged in
the games activities with wide range of bene�ts including entertainment, relief from social
isolation and mental exercising. So it is important to investigate how such devices could
be helpful in the social interaction of the elderly people.
The increasing ratio of the elderly people to the young people in the world also makes it
essential that the elderly people remain �t, independent, scure well into high age. Medical
informatics undertakes to improve the standard of living for such growing population of
the elderly people, general health problems or chronic conditions including diabetes and
blood pressure. The elderly people also su�ering memory problems and sometimes they
tend to wander and get lost. In a mobile devices a cell-based positioning technique is used
to locate and �nd persons but with the limited accuracy. To maintain such types of the
elderly users requirements, the authors suggest to �nd out which potential applications
could provide a bene�t to the elderly people and how.
It has been seen that the elderly people also avoid to use the new mobiles technologies
because of the complexities of the functions. It could be possible to discover some of the
parameters of complexity reductions for mobile devices by applying the HCI and usability
engineering methodologies. These parameters could be helpful in ordered to enhance the
design and the development of the interfaces.
72
Chapter 6
6 APPENDICES
6.1 Appendix A Survey Questionnaire
Question (Strongly disagree - Strongly Agree) 1 2 3 4 5 6 7
The system is easy for opening applications O O O O O O O
comments:
The system is easy for closing applications O O O O O O O
comments:
The system is easy for Finding help O O O O O O O
comments:
I can easily use the system in the sun light O O O O O O O
comments:
It is easy to �nd a new application in the Apple store O O O O O O O
comments:
The layout of the applications on the system screen is clear O O O O O O O
comments
The graphics of the system is appealing for my age group O O O O O O O
comments:
I can easily change the background colour of the interface O O O O O O O
comments:
I think the system is designed for all age groups O O O O O O O
comments
I like the �nishing of the interface O O O O O O O
comments:
The interface of the system is pleasant O O O O O O O
comments:
It is easy to install the required application O O O O O O O
comments:
It is simple to uninstall the application O O O O O O O
comments:
The system has all the functions and capabilities I expect it to have O O O O O O O
comments:
Cont...
73
I like using the interface of the system O O O O O O O
comments:
I can easily use the text �zoom in and zoom out� signs (gestures) O O O O O O O
comments:
I like the gestures for selecting and zooming the text O O O O O O O
comments
I think the gestures are easy to learn O O O O O O O
comments:
I think the gestures are easy to remember O O O O O O O
comments:
I can easily change the location of the contents (icons) on the interface O O O O O O O
comments:
I can easily select the written text for deletion O O O O O O O
comments
I enjoy using the keypad while writing Emails O O O O O O O
comments:
I like the keypad in landscape style while writing Emails O O O O O O O
comments:
It is easy to use the special characters (e.g. , ` ' _ *) O O O O O O O
comments:
The arrangement of the alphabet on the keypad is in accordance to my choice O O O O O O O
comments:
Small keys can cause typing mistakes O O O O O O O
comments:
I can easily �nd the numbers (e.g. 1,2,3. . . ) on the keypad O O O O O O O
comments:
The single character keypad improves my writing performance O O O O O O O
comments:
I can easily move the cursor forward and backward in the editor while writing text O O O O O O O
comments:
The applications (e.g. Evernote, calendar) are simple O O O O O O O
comments:
The applications (e.g. Facebook, calendar) work very well in portrait style O O O O O O O
comments:
I think the applications can easily change from portrait style to landscape style O O O O O O O
comments:
I can easily understand the symbols in the applications O O O O O O O
comments:
74
Cont...
The navigation on the applications is simple O O O O O O O
comments:
The navigation labels are clear and concise O O O O O O O
comments:
The navigation on the applications is similar O O O O O O O
comments:
The text on the applications is easy to read O O O O O O O
comments:
I recover easily when I commit a typing mistake O O O O O O O
comments:
It is easy to �nd the information (help, pop up messages) O O O O O O O
comments:
I need The applications often support the zooming gesture O O O O O O O
comments:
I like swiping the page (up, down left, right) of the newspaper O O O O O O O
comments:
It is easy to browse the web in the Safari browser O O O O O O O
comments:
75
6.2 Appendix B Survey Questionnaire Results
DoA
Q1 Q2 Q3 Q4 Q5 Q6
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 - - 0 0 - - 0 0 - - 0 0 - - 0 0
2 1 - 7 0 1 1 8 11 2 1 14 10 - 2 0 20 - 1 0 10 1 - 7 0
3 - - 0 0 - 1 0 11 1 2 7 20 1 - 7 0 1 1 7 10 - - 0 0
4 - - 0 0 3 1 25 11 6 4 43 40 3 4 21 40 2 3 14 30 3 1 21 10
5 3 1 21 10 1 - 8 0 2 2 14 20 5 3 36 30 2 3 14 30 2 - 14 0
6 2 7 14 70 3 4 25 44 1 1 7 10 3 1 21 10 1 1 7 10 3 6 21 60
7 8 2 57 20 4 2 33 22 2 - 14 0 2 - 14 0 8 1 57 10 5 3 36 30
DoA
Q7 Q8 Q9 Q10 Q11 Q12
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 1 - 7 0 - - 0 0 - - 0 0 - - 0 0
2 1 - 7 0 2 - 14 0 1 - 7 0 1 - 7 0 1 - 7 0 1 - 7 0
3 - - 0 0 2 1 14 10 - 2 0 20 - - 0 0 - - 0 0 - 1 0 11
4 1 - 7 0 3 3 21 30 3 1 21 10 1 2 7 20 2 1 14 10 1 1 7 11
5 3 1 21 11 2 2 14 20 2 1 14 10 3 2 21 20 3 4 21 40 2 3 14 33
6 4 5 29 56 - 3 0 30 3 3 21 30 3 3 21 30 4 2 29 20 4 2 29 22
7 5 3 36 33 5 1 36 10 4 3 29 30 6 3 43 30 4 3 29 30 6 2 43 22
DoA
Q13 Q14 Q15 Q16 Q17 Q18
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 1 - 7 0 1 - 7 0 - - 0 0 - - 0 0
2 1 - 7 0 4 3 29 30 - - 0 0 - 1 0 10 - - 0 0 1 - 7 0
3 1 1 7 11 1 1 7 10 1 1 7 13 - - 0 0 2 2 14 20 1 2 7 20
4 1 1 7 11 1 2 7 20 2 1 14 13 1 1 7 10 - 2 0 20 1 1 7 10
5 1 1 7 11 4 2 29 20 1 4 7 50 1 4 7 40 1 1 7 10 6 2 43 20
6 5 4 36 44 3 2 21 20 6 - 43 0 2 2 14 20 7 4 50 40 1 5 7 50
7 5 2 36 22 1 - 7 0 3 2 21 25 9 2 64 20 4 1 29 10 4 - 29 0
Cont...
76
DoA
Q19 Q20 Q21 Q22 Q23 Q24
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 1 - 7 0 1 1 7 10 - 2 0 20 1 - 0 0
2 - - 0 0 1 - 7 0 2 3 14 30 - 1 0 10 2 - 14 0 - 2 0 22
3 1 - 7 0 1 1 7 10 2 2 14 20 3 - 21 0 - 1 0 10 3 - 21 0
4 2 4 14 40 1 2 7 20 2 1 14 10 2 1 14 10 3 3 21 30 1 2 7 22
5 4 2 29 20 1 5 7 50 3 3 21 30 2 2 14 20 2 2 14 20 2 1 14 11
6 3 3 21 30 4 2 29 20 2 - 14 0 1 4 7 40 4 - 29 0 4 4 29 44
7 4 1 29 10 6 - 43 0 2 1 14 10 5 1 36 10 3 2 21 20 3 - 21 0
DoA
Q25 Q26 Q27 Q28 Q29 Q30
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 2 1 14 10 1 - 7 0 - - 0 0 1 1 7 10 2 - 14 0 - - 0 0
2 - 1 0 10 2 1 14 10 1 - 8 0 1 - 7 0 1 2 7 22 - - 0 0
3 1 1 7 10 1 2 7 20 - 2 0 20 2 1 14 10 5 2 36 22 4 - 29 0
4 _ 1 - 10 2 5 14 50 2 - 15 0 1 4 7 40 1 2 7 22 1 1 7 11
5 3 3 21 30 3 - 21 0 2 3 15 30 4 3 29 30 2 3 14 33 4 4 29 44
6 2 3 14 30 3 1 21 10 1 3 8 30 2 1 14 10 2 - 14 0 2 3 14 33
7 6 - 43 0 2 1 14 10 7 2 54 20 3 0 21 0 1 - 7 0 3 1 21 11
DoA
Q31 Q32 Q33 Q34 Q35 Q36
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 - - 0 0 - - 0 0 - - 0 0 0 0 0 0
2 1 - 7 0 1 - 7 0 - - 0 0 - - 0 0 - 1 0 10 3 0 21 0
3 1 - 7 0 - 1 0 11 2 3 14 30 - 1 0 10 1 1 7 10 - 2 0 20
4 - - 0 0 2 1 14 11 2 2 14 20 2 3 15 30 2 1 14 10 4 2 29 20
5 3 4 21 44 3 4 21 44 4 2 29 20 3 2 23 20 4 3 29 30 2 3 14 30
6 5 2 36 22 4 2 29 22 4 3 29 30 3 4 23 40 2 4 14 40 3 3 21 30
7 4 3 29 33 4 1 29 11 2 - 12 0 5 - 38 0 5 - 36 0 2 - 14 0
DoA
Q37 Q38 Q39 Q40 Q41 Q42
F % F % F % F % F % F %
iP id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id ip id
1 - - 0 0 - - 0 0 1 - 8 0 - - 0 0 2 - 14 0 2 - 14 0
2 - 1 0 10 - 1 0 10 1 2 8 20 1 - 7 0 - 1 0 10 - - 0 0
3 1 - 7 0 - 2 0 20 3 1 23 10 2 1 14 10 2 - 14 0 - - 0 0
4 1 - 7 0 4 1 31 10 1 2 8 20 3 4 21 40 - 2 0 20 1 - 7 0
5 5 5 36 50 4 4 31 40 4 3 31 30 3 2 21 20 1 - 7 0 4 2 29 20
6 2 3 14 30 2 2 15 20 1 2 8 20 3 3 21 30 4 3 29 30 3 4 21 40
7 5 1 36 10 3 - 23 0 2 - 15 0 2 - 14 0 5 4 36 40 4 4 29 40
77
6.3 Appendix C Survey Questionnaire p-values
S.No: Question P Value
1 The system is easy for opening applications 0.183
2 The system is easy for closing applications 0.404
3 The system is easy for Finding help 0.083
4 I can easily use the system in the sun light 0.011
5 It is easy to �nd a new application in the Apple store 0.018
6 The layout of the applications on the system screen is clear 0.322
7 The graphics of the system is appealing for my age group 0.341
8 I can easily change the background colour of the interface 0.489
9 I think the iPhone is designed for all age groups 0.467
10 I like the �nishing of the interface 0.382
11 The interface of the system is pleasant 0.467
12 It is easy to install the required application 0.174
13 It is simple to uninstall the application 0.279
14 The system has all the functions and capabilities I expect it to have 0.265
15 I like using the interface of the system 0.233
16 I can easily use the text �zoom in and zoom out� signs (gestures) 0.027
17 I like the gestures for selecting and zooming the text 0.108
18 I think the gestures are easy to learn 0.424
19 I think the gestures are easy to remember 0.214
20 I can easily change the location of the contents (icons) on the interface 0.015
21 I can easily select the written text for deletion 0.198
22 I enjoy using the keypad while writing Emails 0.446
23 I like the keypad in landscape style while writing Emails 0.143
24 It is easy to use the special characters (e.g. , ` ' _ *) 0.206
25 The arrangement of the alphabet on the keypad is in accordance to my choice 0.080
26 Small keys can cause typing mistakes 0.183
27 I can easily �nd the numbers (e.g. 1,2,3. . . ) on the keypad 0.129
28 The single character keypad improves my writing performance 0.143
Cont...
78
29 I can easily move the cursor forward and backward in the editor while writing text 0.500
30 The applications (e.g. Evernote, calendar) are simple 0.385
31 The applications (e.g. Facebook, calendar) work very well in portrait style 0.477
32 I think the applications can easily change from portrait style to landscape style 0.201
33 I can easily understand the symbols in the applications 0.098
34 The navigation on the applications is simple 0.048
35 The navigation labels are clear and concise 0.169
36 The navigation on the applications is similar 0.489
37 The text on the applications is easy to read 0.183
38 I recover easily when I commit a typing mistake 0.069
39 It is easy to �nd the information (help, pop up messages) 0.333
40 I need The applications often support the zooming gesture 0.467
41 I like swiping the page (up, down left, right) of the newspaper 0.424
42 It is easy to browse the web in the Safari browser 0.155
79
6.4 Appendix D Post Test Questionnaire
Questionnaire for Experiment Evaluating the Usability of the system
(iPhone /iPad)
Participant Number: ______________ Sex: ___________________
Home Country: _____________________ Age: ________________
Education level: 2 BSc 2 MSc 2 PhD 2 Other __________________
This questionnaire gives you an opportunity to describe your experiences of the Experi-
ment. Your responses will help us to evaluate the usability of the system (iPhone/iPad).
To as great degree as possible, consider all the tasks that you have performed during
experiment while you answer these questions. Please read each statement and indicate
how strongly you disagree or agree with the statement by marking (X) in the circle. If a
statement does not apply to you, leave it empty or use the word (N/A) at the comments
line.
Please write comments to elaborate on your answers (if you have any).
As you complete the questionnaire, please do not hesitate to ask any questions.
Thank you.
Question (Strongly disagree - Strongly Agree) 1 2 3 4 5 6 7
The system is easy to use for opening applications O O O O O O O
comments:
I think the system is designed for all age groups O O O O O O O
Comments:
I like using the interface of the system O O O O O O O
Comments:
I can easily use the zoom in/out sings (gestures) O O O O O O O
Comments:
I enjoy using the keypad while writing the text O O O O O O O
Comments:
The applications (Evernote, Facebook) are simple O O O O O O O
Comments:
I can easily understand the symbols in the applications O O O O O O O
Comments:
Cont...
80
The navigation on The application is simple O O O O O O O
Comments:
The navigation labels are clear and concise O O O O O O O
Comments
The text on the applications is easy to read O O O O O O O
Comments:
I recover easily when I commit a typing mistake O O O O O O O
Comments:
I can easily perform the login task O O O O O O O
Comments:
81
6.5 Appendix E Task Time on the iPhone and iPad
P.ID Category
iPhone Task/Sec iPad Task/Sec
T1 T2 T3 T4 T5 T6 T1 T2 T3 T4 T5 T6
P01 210 120 127 175 255 65 95 198 232 231 138 26
P02 152 220 177 212 258 48 73 69 94 223 156 24
P03 120 133 108 205 195 42 134 156 265 177 179 19
P04 65 85 51 297 224 19 86 79 72 121 136 16
P05 108 157 139 167 197 34 116 106 158 291 136 18
P06 133 230 170 200 260 27 160 158 162 155 158 25
P07 126 61 142 138 163 16 74 54 48 179 84 33
P08 151 105 78 199 162 54 75 106 60 68 106 23
P09 Novice 81 105 198 131 143 57 29 79 62 99 125 20
P010 User 75 56 173 135 228 44 81 154 186 121 227 25
P011 Group 135 124 148 201 255 33 101 112 125 178 187 16
P012 159 139 119 198 235 18 99 129 113 166 165 21
P013 109 113 153 204 201 26 111 108 142 139 104 13
P014 145 155 166 115 234 27 88 100 109 153 145 19
P015 133 138 159 222 223 29 102 79 122 167 124 21
P016 134 177 187 203 159 32 85 102 123 171 99 19
P017 140 148 178 212 164 27 118 89 105 153 103 24
P018 132 134 155 219 188 19 94 96 137 121 126 12
P019 147 158 163 225 139 25 101 130 113 124 95 19
P020 138 123 135 236 216 14 121 97 122 197 112 13
Cont...
82
P11
Category T1 T2 T3 T4 T5 T6 T1 T2 T3 T4 T5 T6
49 35 55 61 118 18 36 42 61 47 62 19
P13 53 48 48 78 97 18 78 55 87 87 104 29
P14 47 63 51 245 119 9 58 73 82 102 104 27
P15 35 45 34 86 87 21 43 43 94 86 92 24
P16 89 84 75 223 106 27 47 70 117 63 103 24
P17 55 57 82 255 67 18 27 38 65 75 87 26
P18 64 90 57 137 99 20 58 52 64 54 65 17
P19 Experienced 54 55 33 185 89 13 43 52 32 65 75 28
P110 User 26 44 62 77 91 18 49 61 81 64 102 13
P111 Group 55 59 65 99 115 22 46 48 50 58 88 15
P112 49 53 46 108 99 19 35 48 39 76 79 17
P113 58 57 76 167 111 25 40 55 65 98 99 21
P114 44 62 55 106 98 15 34 42 44 84 90 22
P115 41 50 67 132 105 19 29 35 59 69 89 15
P116 53 65 54 109 113 25 21 44 57 91 91 18
P117 34 72 88 99 121 15 32 61 69 79 98 18
P118 42 64 62 117 99 26 28 49 61 88 76 19
P119 29 80 69 123 117 17 25 60 73 97 83 16
P120 38 69 74 112 94 19 32 56 64 86 88 17
Cont...
83
P21
Category T1 T2 T3 T4 T5 T6 T1 T2 T3 T4 T5 T6
255 119 141 210 158 20 51 159 131 158 176 33
P22 95 63 185 104 181 71 87 61 141 181 70 27
P23 195 141 100 187 51 20 62 117 61 51 108 16
P24 74 98 99 310 129 23 132 119 101 129 191 32
P25 113 156 121 160 260 35 113 163 137 260 240 45
P26 78 260 123 208 202 43 91 160 116 202 213 34
P27 124 117 150 187 155 23 76 54 48 155 214 35
P28 84 119 199 152 279 20 82 80 66 279 264 16
P29 Elderly 133 105 181 357 111 15 74 80 113 111 128 46
P210 User 138 133 100 177 148 15 137 69 193 148 133 24
P211 Group 188 192 209 284 211 34 165 111 144 215 183 31
P212 159 205 258 305 288 43 124 119 203 267 204 20
P213 144 178 199 255 198 29 99 124 109 200 115 15
P214 201 232 211 289 245 35 132 179 176 201 180 24
P215 135 141 188 254 193 32 123 100 94 199 123 19
P216 211 201 243 278 188 45 198 172 211 234 145 32
P217 238 188 212 251 213 44 179 191 179 233 200 27
P218 199 215 192 245 184 35 201 208 188 214 163 33
P219 244 197 253 199 201 49 195 161 213 201 198 36
P220 188 204 233 254 222 39 192 199 207 188 189 31
84
References
[1] R. M. Baecker, �Readings in Human-Computer Interaction:�, toward the year
2000. Morgan Kaufmann, 1995.
[2] J. X. Xu, �Implementing Multi-lingual Support and Online Catalogue System
for a Company Web Site�, 2010.
[3] P. Elmer-Dwitt, �Evernote for iPhone, iPod touch, and
iPad on the iTunes App Store�, 2011. Available at:
http://itunes.apple.com/us/app/evernote/id281796108?mt=8# [Accessed
March 6, 2011]
[4] L. Arhippainen, �Studying user experience: issues and problems of mobile
services�, 2009.
[5] BugHuntress QA Lab, �Mobile Usability Testing Problem and Solutions�, In:
Proceedings of the Conference Quality Assurance: Management & Technolo-
gies (QAMT Ukraine'07), 2007.
[6] M. Fausto and P. Alberto, �Context planning and user pro�ling in mobile ser-
vices�, in Information Technology Interfaces (ITI), 2010 32nd International
Conference on, 2010, pp. 301-306.
[7] A. Marcus, J. V. Ferrante, T. Kinnunen, K. Kuutti, and E. Sparre, �Baby
faces: user-interface design for small displays�, in CHI 98 conference sum-
mary on Human factors in computing systems, 1998, pp. 96-97.
[8] A. H. Betiol and W. de Abreu Cybis, �Usability testing of mobile devices: A
comparison of three approaches�, Human-Computer Interaction-INTERACT
2005, pp. 470-481, 2005.
[9] K. Drotner, �Di�erence and diversity: trends in young Danes' media uses�,
Media, culture & society, vol. 22, no. 2, p. 149, 2000.
[10] V. Oksman, �Young People and Seniors in Finnish'Mobile Information Soci-
ety�, Journal of Interactive Media in Education, vol. 2, no. 2, p. 2, 2006.
[11] N. T. T. DoCoMo, �Mobile phones increasingly popular among the elderly�,
Press Release, vol. 34, no. 11, 2000.
[12] U. Villaseca and P. Tomás, �Usability of Mobile Devices for Elderly People�,
Unpublished master's thesis, National University of Ireland Galway, Galway,
Ireland, 2010.
85
[13] J. Sunwoo, W. Yuen, C. Lutteroth, and B. Wünsche, �Mobile games for
elderly healthcare�, in Proceedings of the 11th International Conference of
the NZ Chapter of the ACM Special Interest Group on Human-Computer
Interaction on ZZZ, 2010, pp. 73-76.
[14] T. Hsien-Hui, P. Si-Ying, �Holistically Understanding the User Model of the
Elderly People While Using Mobile Phones�, DITL, Chang Gung University,
Taoyuan, Taiwan 2008.
[15] H. Neil, �AppleInsider | Apple's iPhone 3GS has
99 percent satisfaction rate.� [Online]. Available:
http://www.appleinsider.com/articles/09/08/14/apples_iphone_3gs_has_
99_percent_satisfaction_rate.html. [Accessed: 22-Aug-2011].
[16] M. Ebner, C. Stickel, and J. Kolbitsch, �iPhone/iPad human interface de-
sign�, HCI in Work and Learning, Life and Leisure, pp. 489-492, 2010.
[17] J. Keijzers, E. den Ouden, and Y. Lu, �Usability benchmark study of com-
mercially available smart phones: cell phone type platform, PDA type plat-
form and PC type platform�, in Proceedings of the 10th international confer-
ence on Human computer interaction with mobile devices and services, 2008,
pp. 265�272.
[18] K. Hornbæk, �Current practice in measuring usability: Challenges to usabil-
ity studies and research�, International journal of human-computer studies,
vol. 64, no. 2, pp. 79�102, 2006.
[19] B. Shackel, �Usability-context, framework, de�nition, design and evaluation�,
Human factors for informatics usability, pp. 21-37, 1991.
[20] B. W. Boehm, J. R. Brown, and M. Lipow, �Quantitative evaluation of soft-
ware quality�, in Proceedings of the 2nd international conference on Software
engineering, 1976, p. 605.
[21] E. Folmer and J. Bosch, �Architecting for usability: a survey�, Journal of
systems and software, vol. 70, no. 1-2, pp. 61�78, 2004.
[22] J. Nielsen, �Usability engineering.�, Morgan Kaufmann, 1993.
[23] E. DIN, �9241-11. Ergonomic requirements for o�ce work with visual display
terminals (VDTs)�Part 11: Guidance on usability�, International Organiza-
tion for Standardization, 1998.
86
[24] L. Chung and J. do Prado Leite, �On non-functional requirements in software
engineering,� Conceptual Modeling: Foundations and Applications, pp. 363-
379, 2009
[25] J. Lazar, J. H. Feng, and H. Hochheiser, �Research methods in human-
computer interaction.�, John Wiley & Sons Inc, 2009.
[26] ISO, �9241-11: Ergonomic requirements for o�ce work with visual display
terminals (VDTs)-Part 11: guidance on usability�, Geneve, CH: ISO. 1998.
[27] J. Nielsen and J. Levy, �Measuring usability: preference vs. performance�,
Communications of the ACM, vol. 37, no. 4, pp. 66-75, 1994.
[28] K. Kim, R. W. Proctor, and G. Salvendy, �The relation between usability
and product success in cell phones�, 2011.
[29] M. Barbacci, M. H. Klein, T. A. Longsta�, C. B. Weinstock, and C.-M. U.
P. P. S. E. INST, �Quality attributes�, vol. 6. Citeseer, 1995.
[30] S. Je�, �A Practical Guide to Measuring Usability�, 2010.
http//:www.MeasuringUsability.com
[31] J. Preece, H. Sharp, and Y. Rogers, �Interaction design.�, Apogeo Editore,
2004.
[32] F. Liu, �Usability evaluation on websites,� in Computer-Aided Industrial De-
sign and Conceptual Design, 2008. CAID/CD 2008. 9th International Con-
ference on, pp. 141�144.
[33] Z. Zhang, �Overview of usability evaluation methods�, 2001.,
http://www.cs.umd.edu/~zzj/UsabilityHome.html
[34] Y. Yang, Y. Zhang, Z. Hou, Z. Y. Chen, and B. Lemaire-Semail, �FingViewer:
A new multi-touch force feedback touch screen,� in Consumer Electronics
(ICCE), 2011 IEEE International Conference on, pp. 837�838.
[35] A. K. B. Mahmood and S. Sulaiman, �Investigation of �ngertip blobs on
optical multi-touch screen,� in Information Technology (ITSim), 2010 Inter-
national Symposium in, vol. 1, pp. 1�6.
[36] A. Bragdon, E. Nelson, Y. Li, and K. Hinckley, �Experimental Analysis of
Touch-Screen Gesture Designs in Mobile Environments,� 2011.
[37] O. Millberg and A. Skogberg, �Usable and Dynamic Interface for Mobile
Reporting,� 2010.
87
[38] Apple, http://www.Apple.com, 2011.
[39] apple, http://www. Apple.com, 2010.
[40] Sam Costello, "About.com iPod Guide.�, 2011. [Online]. Available:
http://ipod.about.com/bio/Sam-Costello-24480.htm. [Accessed: 22-Aug-
2011].
[41] L. Tsung, �An Age-based Study on iPhone Usability.�, 2008.
[42] B. Chaparro, B. Nguyen, M. Phan, A. Smith, and J. Teves, �Keyboard Per-
formance: iPad versus Netbook.�, 2010.
[43] J. Kelly and J. Schrape, �100 days with an iPad: Lessons learnt and apps
acquired.�, 2010.
[44] J. Nielsen and R. BUDIU, �Usability of iPad Apps and Websites�, First
Research, 2010.
[45] R. Kumar, �Research methodology: A step-by-step guide for beginners.�,
Sage Publications Ltd, 2010.
[46] J. W. Creswell, �Research design: Qualitative, quantitative, and mixed meth-
ods approaches.�, Sage Publications, Inc, 2009.
[47] I. Newman and C. R. Benz, �Qualitative-quantitative research methodology:
Exploring the interactive continuum.�, Southern Illinois Univ Pr, 1998.
[48] C. R. Kothari, �Research methodology: methods and techniques.�, New Age
International, 2008.
[49] ORISE, �Quantitative Research Methods�, 2007b. [Online]. Available:
http://www.orau.gov/cdcynergy/demo/Content/activeinformation/tools/
toolscontent/quantiativemethods.htm#resources. [Accessed: 22-Aug-2011]
[50] M. E. Whitman and A. B. Woszczynski, �The handbook of information sys-
tems research.�, Igi Global, 2004.
[51] P. D. Leedy and J. E. Ormrod, Practical research: Planning and design. NJ,
2009.
[52] ORISE, �Qualitative Research Methods�, 2007a. [Online]. Available:
http://www.orau.gov/cdcynergy/demo/Content/activeinformation/tools/
toolscontent/qualitativemethods.htm#resources. [Accessed: 22-Aug-2011].
88
[53] T. Covey and others, �Usage and usability assessment: Library practices and
concerns�, 2002.
[54] M. Kaps, W. R. Lamberson, and W. Lamberson, �Concepts of experimental
design.,� Biostatistics for animal science, pp. 263�271, 2004.
[55] D. C. Montgomery, Design and analysis of experiments. John Wiley & Sons
Inc, 2008.
[56] G. M. Breakwell, �Research methods in psychology.� Sage Publications Ltd,
2006.
[57] F. Sani, J. Todman, and J. B. Todman, �Experimental design and statistics
for psychology: a �rst course.�, Wiley-Blackwell, 2006.
[58] J. Lazar, J. H. Feng, and H. Hochheiser, �Research methods in human-
computer interaction.�, John Wiley & Sons Inc, 2009.
[59] U. Reja, K. L. Manfreda, V. Hlebec, and V. Vehovar, �Open-ended vs. close-
ended questions in web questionnaires,� Advances in Methodology and Statis-
tics (Metodolo²ki zvezki), vol. 19, pp. 159�177, 2003.
[60] R. Ho, �Handbook of univariate and multivariate data analysis and interpre-
tation with SPSS.�, CRC Press, 2006.
[61] D. G. Rees, �Essential statistics.�, CRC Press, 2001.
[62] Mann-Whitney test, �The Prism Guide to Interpreting Statisti-
cal Results.� [Online]. Available: http://www.graphpad.com/ arti-
cles/interpret/Analyzing_two_groups/ mann_whitney.htm. [Accessed:
22-Aug-2011].
[63] S. S. Shapiro and M. B. Wilk, �An analysis of variance test for normality
(complete samples)�, Biometrika, vol. 52, no. 3/4, pp. 591-611, 1965.
[64] J. W. Davis, �Handbook of Univariate and Multivariate Data Analysis and
Interpretation with SPSS�, The American Statistician, vol. 62, no. 3, pp.
268�268, 2008.
[65] J. H. McDonald, �Handbook of biological statistics.�, Sparky House Publish-
ing, 2009.
[66] J. R. Lanzante, �A cautionary note on the use of error bars�, Journal of
climate, vol. 18, no. 17, p. 3699, 2005.
89
[67] G. R. Marczyk, G. R. Marczyk, D. DeMatteo, and D. Festinger, �Essentials
of research design and methodology�, vol. 2. Wiley, 2010.
[68] C. Robson, �Real world research.�, Blackwell Oxford, 2003.
[69] Facebook, �Factsheet | Facebook.� Available at :
http://www.facebook.com/press/info.php? factsheet [Accessed March
6, 2011]
[70] P. Tripathi, M. Pandey, & D. Bharti, �Towards the identi�cation of usability
metrics for academic Web-sites.� In Computer and Automation Engineering
(ICCAE), 2010 The 2nd International Conference on. IEEE, pp. 393-397.
[71] Burcher, �Facebook usage statistics� Dec 31st 2010 vs Dec 31st 2009 vs Dec
31st 2008. Available at: http://www.nickburcher.com/2011/01/facebook-
usage-statisticsdec- 31st-2010.html [Accessed March 6, 2011].
90
