LONG PAPER
The cluster scanning system
Pradipta Biswas • Peter Robinson
Published online: 18 August 2012
� Springer-Verlag 2012
Abstract Users with severe physical impairment often
use computers with one or two switches using a scanning
system. Scanning is a technique of successively high-
lighting portions of screen. This paper presents a new
scanning system that works through clustering screen
objects. The system is initially calibrated through simula-
tion and later validated through a user trial. Results show
that it outperforms existing block scanning systems.
Keywords Assistive technology � Virtual user simulator �Scanning systems � Cluster scanning
1 Introduction
Many users with physical disabilities cannot interact with a
computer through a conventional keyboard and mouse. For
example, Spasticity, Amyotrophic Lateral Sclerosis, and
Cerebral Palsy confine movement to a very small part of
the body. People with these disorders may interact with a
computer through one or two switches with the help of a
scanning mechanism. Scanning is a technique based on
successively highlighting items on a computer screen and
pressing a switch when the desired item is highlighted.
The authors have developed a new scanning system that
works by clustering screen objects in a graphical user
interface (GUI). Currently, the system has been imple-
mented for the Microsoft Windows operating system;
however, it can be extended to any other GUI-based
operating systems. A previous paper compared the cluster
scanning system with other scanning systems using simu-
lation [1]. This paper validates the results obtained in
simulation by evaluating the cluster scanning system
through a controlled experiment with users with motor
impairment.
The following two sections describe the scanning sys-
tems and the results obtained using simulation. Section 4
presents the user trial, followed by concluding remarks.
2 The scanning systems
In this study, we compared two scanning systems: a con-
ventional block scanning system and the new cluster
scanning system.
2.1 The block scanning system
A block scanning system iteratively segments the screen
into equally sized sub-areas. The user has to select a sub-
area that contains the intended target. The segmentation
process iterates until the sub-area contains a single target
(Fig. 1).
2.2 The cluster scanning system
The cluster scanning system initially collects all possible
targets in a screen. Then, it iteratively divides a screen into
several clusters of targets based on their locations (Fig. 2).
The user has to select the cluster that contains the intended
target. The clustering process iterates until the cluster
P. Biswas (&)
Department of Engineering, University of Cambridge,
Cambridge CB2 1PZ, UK
e-mail: [email protected]
P. Robinson
Computer Laboratory, University of Cambridge,
Cambridge CB3 0FD, UK
e-mail: [email protected]
123
Univ Access Inf Soc (2013) 12:237–245
DOI 10.1007/s10209-012-0277-9
contains a single target (Fig. 2). The cluster scanning
system works by enumerating objects being shown on the
screen and storing positions of windows, buttons, icons and
other possible targets (c.f. Appendix 1). The algorithm
starts by considering all the processes running on the
computer.
If a process is controlling a window, then the algorithm
also considers all child and thread processes owned by it.
During the enumeration process, the algorithm identifies
the foreground window and separately stores the positions
of the foreground window and targets within it from the
background windows. The algorithm also calculates the
area occupied by the foreground window. Then, it sepa-
rately clusters the targets in the foreground window and
background windows. The ratio of the number of clusters
in foreground and background windows is proportional to
the ratio of the area occupied by the foreground window in
the whole screen. We used the Fuzzy c-means algorithm
[7] to cluster the targets into similarly sized groups. The
algorithm is similar to the k-means clustering algorithm.
The k-means algorithm partitions points into k clusters
where each point belongs to the cluster with nearest mean.
This algorithm aims at minimising the following objective
function
Fig. 1 The block scanning
system
238 Univ Access Inf Soc (2013) 12:237–245
123
J ¼Xk
j¼1
Xn
i¼1
xðjÞi � cj
������
2
where xðjÞi � cj
������
2
is a distance measure between a data
point xðjÞi and the cluster centre cj. The Fuzzy c-means
algorithm returns the membership values of data points into
different clusters instead of putting the data points into
separate clusters. As a result, when the data points are not
naturally separated, it returns overlapping clusters. The
c-means algorithm takes the number of clusters (c) as input.
It aims at minimising the following objective function
Jm ¼XN
i¼1
XC
j¼1
umij xi � cj
�� ��21�m\ /;
where m is any real number greater than 1, uij is the degree
of membership of xi in the cluster j, xi is the ith of
d-dimensional measured data, cj is the d-dimension centre
of the cluster, and ||*|| is any norm expressing the similarity
between any measured data and the centre.
By differentiating task completion time with respect to
the number of clusters, it was found that five is the opti-
mum number of clusters. Therefore, targets are clustered
into five regions.
3 Evaluation through simulation
We recorded sample interactions by two able-bodied users
to generate a list of tasks, which were fed to the simulator
[2, 3].
• The model for the cluster scanning system takes the
scan delay, the number of clusters, the intended target
and the total number and positions of targets in a screen
as input and gives the target acquisition time as output.
The model calculates the target acquisition time by
running the cluster scanning algorithm on the input.
• The model for the block scanning system takes the scan
delay (s), the number of blocks (k) and the number of
iterations (r) as input, and gives the target acquisition
time as output. The model calculates the target
acquisition time by running the block scanning algo-
rithm on the input. The minimum target acquisition
time is s 9 r (while the target can be reached by always
selecting the first block) and the maximum is equal to
s 9 k 9 r (while the target has to be reached by always
selecting the last block).
We investigated the block scanning system for different
numbers of blocks and different numbers of iterations, and
the cluster scanning system for different numbers of
Fig. 2 The cluster scanning
system
Univ Access Inf Soc (2013) 12:237–245 239
123
clusters. The cluster scanning system performed best when
the number of clusters was five. However, among the dif-
ferent versions of Cluster and Block scanning processes, it
was found that a type of block scanning that divided the
screen into four equal sized partitions for four iterations
performed best.
It was expected that the cluster scanning process
would perform better since it uses information about
target types (e.g. labels are not considered as possible
targets) and locations in the clustering process. So, the
analysis was extended to consider the actual tasks
undertaken by participants. Most of the time, participants
used instant messenger software and browsed the World
Wide Web. The present version of the clustering process
does not consider locations of hyperlinks in the target
acquisition process, and so as a consequence it might
miss possible targets during Web surfing. The further
study revealed that participants should take less time to
complete a task using the cluster scanning system than
other scanning systems if the clustering process could
include all targets in a screen. Details of this study can be
found in [1].
4 The experiment
We validated results obtained in the simulation study
through a controlled experiment with users with motor
impairment. Details of the experiment are discussed in the
following sub-sections.
4.1 Participants
We collected data from 8 motor impaired and 8 able-
bodied participants (Table 1). The motor impaired partic-
ipants were recruited from a local centre, which works on
treatment and rehabilitation of disabled people and they
volunteered for the study. All motor impaired participants
used computer at least once each week. Able-bodied par-
ticipants were university students and expert computer
users. None of them had used the scanning systems before.
4.2 Procedure
In this experiment, the participants were instructed to
press a set of buttons placed on a screen (Fig. 3) in a
particular sequence. All the buttons were coloured grey
except the next target, which was red. The same task was
repeated for all the scanning systems. In particular, we
evaluated the cluster and block scanning systems. We
recorded cursor traces, target height, width and task
completion times. For internal validity of the experiment,
no scan delay adaptation algorithm was used. The scan
delay was kept constant at 2 s for motor impaired par-
ticipants and at 1 s for the control group. These values
were selected after observing their reaction time and were
greater than the reaction time. All participants were
trained adequately with the scanning systems before
undertaking the experiment.
4.3 Material
We used a push button switch [10] and an Acer Aspire
1640 Laptop with 1,280 9 800 pixel screen resolution. We
also used the same seating arrangement (same table height
and distance from table) for all participants. We have
investigated the effect of physical strength of users on the
scanning system. As all users could use their hands to
operate computer peripherals, their hand strength was
measured. Additionally, the hand evaluation devices are
cheap, easy to operate and have good test–retest reliability
[5]. So these are reliable and useful tools for measuring
physical strength making these results useful in practice.
The six variables listed below were measured for hand
strength evaluation. Each was measured three times and the
average was calculated. We evaluated only the dominant
hand (the hand participants used to operate the mouse).
Photographs of the measurement technique can be found in
Appendix 2.
Table 1 List of participants
Age Gender Impairment
C1 27 F Able-bodied
C2 28 F
C3 30 M
C4 30 M
C5 31 M
C6 28 F
C7 30 F
C8 26 F
P1 30 M Cerebral Palsy affecting manual dexterity,
wheel chair user
P2 43 M Cerebral Palsy affecting manual dexterity, also
have tremor in hand, wheel chair user
P3 30 M Dystonia, cannot speak, cannot move fingers,
wheelchair user
P4 62 M Left side (non-dominant) paralysed after a
stroke in 1973, also have tremor
P5 44 M Cerebral attack, significant tremor in whole
upper body part, fingers always remain folded
P6 46 F Did not mention disease, hard to grip things, no
tremor
P7 [45 F Spina Bifida/Hydrocephalus, wheelchair user
P8 [45 M Cerebral Palsy from birth, restricted hand
movement, no tremor
240 Univ Access Inf Soc (2013) 12:237–245
123
• Grip Strength measures how much force a person can
generate by gripping by hand. It was measured using a
mechanical dynamometer.
• Tip Pinch Strength measures the maximum force
generated by a person squeezing something between
the tips of the thumb and index finger and was
measured using a mechanical dynamometer.
The following ranges of motion are defined with respect
to the standard anatomical position [[4], (c.f. Appendix 2)].
• Radial deviation is the motion that rotates the wrist
away from the midline of the body when the person is
standing in the standard anatomical position [4]. When
the hand is placed over a table with palm facing down,
this motion rotates the hand about the wrist towards the
thumb. Maximum radial deviation was measured using
a goniometer.
• Ulnar deviation is the motion that rotates the wrist
towards the midline of the body when the person is
standing in the standard anatomical position. When the
hand is placed over a table with palm facing down, this
motion rotates the hand about the wrist towards the
little finger. It was measured with the goniometer.
• Pronation is the rotation of the forearm so that the palm
moves from a facing up position to a facing down
position and was measured using a wrist-inclinometer.
• Supination is the opposite of pronation, the rotation of
the forearm so that the palm moves from a facing down
position to a facing up position and again was measured
with the wrist-inclinometer.
4.4 Results
Initially, the total task completion time was measured for
the scanning systems (Table 2; Fig. 4). It can be seen that
participants took less time to complete the task using the
cluster scanning system. The dotted bars in Fig. 4 mean
that two participants could not complete the task using the
block scanning system.
To further investigate the scanning systems, we mea-
sured the following three dependent variables:
• Number of missed clicks: We counted the number of
times the participants wrongly pressed the switch.
• Idle Count: The scanning systems periodically high-
light the buttons. This variable measures the number of
cycles when the participants did not provide any input,
though they were expected to do so.
• Efficiency: The scanning systems require a minimum
time to complete any task which depends on the
particular scanning system and not on the performance
of the user. We calculated the efficiency as the ratio
Optimal TimeActual Time
. An efficiency of 100 % indicates opti-
mal performance, 50 % indicates taking twice the
minimal time, and 0 % indicates failure to complete the
task.
Table 2 presents the efficiency of each participant.
No significant difference was found between the per-
formances of motor impaired and able-bodied users by an
equal variance equal variance t test at p \ 0.05 level.
However, efficiency, the average number of missed clicks
and idle count are significantly lower in the cluster
scanning system than in the block scanning system in an
equal variance paired t test (p \ 0.05) (Fig. 5). Addi-
tionally, two participants (P3 and P7) could not complete
the task using the block scanning system, while all par-
ticipants could complete the task using the cluster scan-
ning system.
Table 3 shows the correlation coefficients of these
variables with the hand evaluation metrics. The only sig-
nificant effect is a correlation between the number of
missed clicks in the cluster scanning system and grip
Fig. 3 Screenshot of the
experiment
Univ Access Inf Soc (2013) 12:237–245 241
123
strength. There was a similar but weaker effect in the block
scanning system. It seems that hand strength does not affect
performance of users with the scanning systems. Also, no
effect of target size (height and width) on the task com-
pletion time was found, which is not surprising as the
scanning systems did not depend on the target size.
4.5 Discussion
The simulator predicted that the task completion time
would be less in the cluster scanning system than the block
scanning system when the cluster scanning system can
consider all possible targets in its clustering process. The
experiment also shows similar results. The total task
completion time, sub-optimal task completion time, idle
time and number of missed clicks are less in the cluster
scanning system than the block scanning system. The
efficiency of the cluster scanning system can be attributed
to the following factors:
• The cluster scanning system does not introduce any
new interface element like a frame or form in the screen
as Autonomia [8] or FastScanner [6] systems do.
• The cluster scanning system does not blindly divide the
screen in a predefined number of segments as the
ScanBuddy [9] or block scanning systems do. It clusters
the target so that the targets are evenly divided into
blocks and a block is not drawn in a region that does
not contain any target.
No effect of hand strength on pointing performance was
found while participants used the scanning systems. There
are two possible explanations:
• The switch used in scanning only requires a gentle push
to operate and the hand strength of motor impaired
users is sufficient to operate the switch.
• The scanning software does the navigation itself and
the users need not move their hand to move the pointer.
Table 2 Comparing scanning systems
Participants Cluster scanning system Block scanning system
Efficiency Task
completion
time (in s)
Efficiency Task
completion
time (in s)
P1 0.39 166 0.13 618
P2 0.76 86 0.62 125
P3 0.89 73 0.75 104
P4 0.58 113 0.15 171
P5 0.54 120 0.48 107
P6 0.61 107 0.33 236
P7 0.68 97 0.57 137
P8 0.50 131 0.47 165
C1 0.56 58 0.51 77
C2 0.84 39 0.53 73
C3 0.49 67 0.37 105
C4 0.76 43 0.63 62
C5 0.71 46 0.67 58
C6 0.56 59 0.57 68
C7 0.62 53 0.66 59
C8 0.42 77 0.46 84
Average 0.62 83 0.49 141
Comparing Cluster and Block Scanning Systems
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
200000
P1 P2 P3 P4 P5 P6 P7 P8 C1 C2 C3 C4 C5 C6 C7 C8
Participants
Tas
k C
om
ple
tio
n T
ime
(in
mse
c)
Cluster Scanning System
Block Scanning System
Fig. 4 Task completion times
for the scanning systems
242 Univ Access Inf Soc (2013) 12:237–245
123
This result with the scanning system also shows that an
appropriate choice of an assistive technology can make
interaction independent of the physical strength of users.
This study also confirms the value of automatically
evaluating assistive interfaces using a simulator [1–3].
Before running a formal user trial, a system designer may
tune interface parameters or select the best design alter-
native using the simulator. As each alternative design does
not need to be evaluated by a user trial, the simulator will
reduce the development time significantly.
5 Conclusion
We have developed a new scanning system that works
by clustering screen objects. In a previous paper, we
evaluated the cluster scanning system and found it
superior to the block scanning system. This paper has
presented a study involving motor impaired users to
evaluate the cluster scanning system in practice and to
validate the results of the simulation. We found that the
total task completion time, efficiency, idle time and
number of missed clicks are less in the cluster scanning
system than for the block scanning system. The results in
turn also validate the simulator. The performance on the
scanning system does not also seem to depend on the
physical strength of users, or in other words physical
impairment does not seem to affect performance for the
scanning system. It can be inferred that motor impaired
users found the cluster scanning system faster, easier and
more accurate than the conventional block scanning
system.
Comparing Cluster and Block Scanning Systems
0.42
3.75
6.96
18.59
0.620.49
Avg. Missed Click
Avg. Idle Count
Avg. Efficiency
Block Scanning System
Cluster Scanning System
Fig. 5 Comparing the scanning
systems
Table 3 Correlation
coefficients for the scanning
systems
* Significant at p \ 0.05
Cluster scanning system Block scanning system
Missed click Idle count Efficiency Missed click Idle count Efficiency
Correlations
GS -0.580* -0.191 0.168 -0.429 -0.331 0.283
TPS -0.374 -0.105 0.110 -0.271 -0.153 0.093
ROM
Wrist
-0.414 -0.154 0.189 -0.127 -0.120 0.068
ROM
Forearm
0.000 0.106 -0.079 -0.268 -0.225 0.076
Significance
GS 0.018 0.478 0.534 0.097 0.210 0.289
TPS 0.153 0.699 0.686 0.310 0.572 0.731
ROM
Wrist
0.111 0.569 0.484 0.639 0.659 0.803
ROM
Forearm
1.000 0.695 0.770 0.315 0.401 0.778
Univ Access Inf Soc (2013) 12:237–245 243
123
Appendix 1: The cluster scanning algorithm
Subroutine Cluster Scan
Get Targets
Cluster Targets
End Subroutine
Subroutine Get Targets
Enumerate all currently running processes
If the process creates a window and the window is visible
Store its position
If the window is the foreground window
Store its handle
End if
Get all child processes
Get all Thread Processes
End if
End Subroutine
Subroutine Get Child Processes
Get Class name
//Store positions of buttons, textboxes and combo boxes
If class name contains “Button” or class name =”text” or class name =”combo”
Get Position
If Parent Process is Foreground Window
Store its position with a mark
Else
Store its position without any mark
End if
//Store positions of icons
Else If class name =”SysListView32”
Get number of Icons
Get Position of all Icons
If Parent Process is Foreground Window
Store positions of all icons with a mark
Else
Store positions of all icons without any mark
End if
End if
End Subroutine
Subroutine Get Thread Processes
If the process creates a Window and the window is visible
Get Window’s Position
If Parent Process is Foreground Window
Store its position with a mark
Else
Store its position without any mark
End if
End if
End Subroutine
Subroutine Cluster Targets
Get the percentage of area occupied by the foreground window
Run Fuzzy c-means algorithm on the targets of the foreground window with
values of c proportional to percentage of area occupied by the foreground
window
Run Fuzzy c-means algorithm on the rest of the targets with values of c
proportional to (1 - percentage of area occupied by the foreground window)
End Subroutine
Appendix 2: Measurement of hand strength
See Figs. 6, 7, 8.
Fig. 6 a Measuring grip strength, b Measuring tip pinch strength,
c Standard anatomical position
244 Univ Access Inf Soc (2013) 12:237–245
123
References
1. Biswas, P., Robinson, P.: A new screen scanning system based on
clustering screen objects. J. Assist. Technol. 2(3) Sept 2008,
ISSN: 1754-9450
2. Biswas, P., Robinson, P.: Evaluating interface layout for visually
and motor impaired users through simulation. Univers Access
Inform Soc 12(1), Springer, Jan 2012
3. Biswas, P., Robinson, P.: Evaluating the design of inclusive
interfaces by simulation. In: Proceedings of the ACM Interna-
tional Conference on Intelligent User Interfaces (IUI) 2010
4. Kaplan, R. J.: Physical medicine and rehabilitation review, 2nd
edn. MacGraw Hill, New York (2006)
5. Mathiowetz, V., Weber, K., Volland, G., Kashman, N.: Reli-
ability and validity of hand strength evaluation. J Hand Surg
(1984)
6. Ntoa, S., Savidis, A., Stephanidis, C.: FastScanner: an accessi-
bility tool for motor impaired users. In: Proceedings of the
ICCHP 2004, LNCS 3118, pp. 796–803 (2004)
7. Ross, T.J.: Fuzzy logic with engineering application, Interna-
tional Edition. McGraw-Hill, New York (1997)
8. Steriadis, C.E., Constantnou, P.: Using the scanning technique to
make an ordinary operating system accessible to motor impaired
users. The autonomia system. In: Proceedings of the International
ACM SIGACCESS Conference on Computers and Accessibility
(ASSET 2002)
9. The ScanBuddy system. Available from: www.ahf-net.com/
Scanbuddy.htm. Accessed on 21 May 2007
10. The Super-Switch. Available at: http://rjcooper.com/super-
switch/index.html. Accessed on 1 July 2007
Fig. 7 Measuring ranges of
motion of wrist. a Range of
motion of wrist (Palm facing
down), b Measuring radial
deviation, c Measuring ulnar
deviation
Fig. 8 Measuring ranges of motion of forearm. a Pronation, b Mea-
suring pronation, c Supination, d Measuring supination
Univ Access Inf Soc (2013) 12:237–245 245
123