LONG PAPER

The cluster scanning system

Pradipta Biswas • Peter Robinson

Published online: 18 August 2012

� Springer-Verlag 2012

Abstract Users with severe physical impairment often

use computers with one or two switches using a scanning

system. Scanning is a technique of successively high-

lighting portions of screen. This paper presents a new

scanning system that works through clustering screen

objects. The system is initially calibrated through simula-

tion and later validated through a user trial. Results show

that it outperforms existing block scanning systems.

Keywords Assistive technology � Virtual user simulator �Scanning systems � Cluster scanning

1 Introduction

Many users with physical disabilities cannot interact with a

computer through a conventional keyboard and mouse. For

example, Spasticity, Amyotrophic Lateral Sclerosis, and

Cerebral Palsy confine movement to a very small part of

the body. People with these disorders may interact with a

computer through one or two switches with the help of a

scanning mechanism. Scanning is a technique based on

successively highlighting items on a computer screen and

pressing a switch when the desired item is highlighted.

The authors have developed a new scanning system that

works by clustering screen objects in a graphical user

interface (GUI). Currently, the system has been imple-

mented for the Microsoft Windows operating system;

however, it can be extended to any other GUI-based

operating systems. A previous paper compared the cluster

scanning system with other scanning systems using simu-

lation [1]. This paper validates the results obtained in

simulation by evaluating the cluster scanning system

through a controlled experiment with users with motor

impairment.

The following two sections describe the scanning sys-

tems and the results obtained using simulation. Section 4

presents the user trial, followed by concluding remarks.

2 The scanning systems

In this study, we compared two scanning systems: a con-

ventional block scanning system and the new cluster

scanning system.

2.1 The block scanning system

A block scanning system iteratively segments the screen

into equally sized sub-areas. The user has to select a sub-

area that contains the intended target. The segmentation

process iterates until the sub-area contains a single target

(Fig. 1).

2.2 The cluster scanning system

The cluster scanning system initially collects all possible

targets in a screen. Then, it iteratively divides a screen into

several clusters of targets based on their locations (Fig. 2).

The user has to select the cluster that contains the intended

target. The clustering process iterates until the cluster

P. Biswas (&)

Department of Engineering, University of Cambridge,

Cambridge CB2 1PZ, UK

e-mail: pb400@cam.ac.uk

P. Robinson

Computer Laboratory, University of Cambridge,

Cambridge CB3 0FD, UK

e-mail: Peter.Robinson@cl.cam.ac.uk

123

Univ Access Inf Soc (2013) 12:237–245

DOI 10.1007/s10209-012-0277-9

contains a single target (Fig. 2). The cluster scanning

system works by enumerating objects being shown on the

screen and storing positions of windows, buttons, icons and

other possible targets (c.f. Appendix 1). The algorithm

starts by considering all the processes running on the

computer.

If a process is controlling a window, then the algorithm

also considers all child and thread processes owned by it.

During the enumeration process, the algorithm identifies

the foreground window and separately stores the positions

of the foreground window and targets within it from the

background windows. The algorithm also calculates the

area occupied by the foreground window. Then, it sepa-

rately clusters the targets in the foreground window and

background windows. The ratio of the number of clusters

in foreground and background windows is proportional to

the ratio of the area occupied by the foreground window in

the whole screen. We used the Fuzzy c-means algorithm

[7] to cluster the targets into similarly sized groups. The

algorithm is similar to the k-means clustering algorithm.

The k-means algorithm partitions points into k clusters

where each point belongs to the cluster with nearest mean.

This algorithm aims at minimising the following objective

function

Fig. 1 The block scanning

system

238 Univ Access Inf Soc (2013) 12:237–245

123

J ¼Xk

j¼1

Xn

i¼1

xðjÞi � cj

������

2

where xðjÞi � cj

������

2

is a distance measure between a data

point xðjÞi and the cluster centre cj. The Fuzzy c-means

algorithm returns the membership values of data points into

different clusters instead of putting the data points into

separate clusters. As a result, when the data points are not

naturally separated, it returns overlapping clusters. The

c-means algorithm takes the number of clusters (c) as input.

It aims at minimising the following objective function

Jm ¼XN

i¼1

XC

j¼1

umij xi � cj

�� ��21�m\ /;

where m is any real number greater than 1, uij is the degree

of membership of xi in the cluster j, xi is the ith of

d-dimensional measured data, cj is the d-dimension centre

of the cluster, and ||*|| is any norm expressing the similarity

between any measured data and the centre.

By differentiating task completion time with respect to

the number of clusters, it was found that five is the opti-

mum number of clusters. Therefore, targets are clustered

into five regions.

3 Evaluation through simulation

We recorded sample interactions by two able-bodied users

to generate a list of tasks, which were fed to the simulator

[2, 3].

• The model for the cluster scanning system takes the

scan delay, the number of clusters, the intended target

and the total number and positions of targets in a screen

as input and gives the target acquisition time as output.

The model calculates the target acquisition time by

running the cluster scanning algorithm on the input.

• The model for the block scanning system takes the scan

delay (s), the number of blocks (k) and the number of

iterations (r) as input, and gives the target acquisition

time as output. The model calculates the target

acquisition time by running the block scanning algo-

rithm on the input. The minimum target acquisition

time is s 9 r (while the target can be reached by always

selecting the first block) and the maximum is equal to

s 9 k 9 r (while the target has to be reached by always

selecting the last block).

We investigated the block scanning system for different

numbers of blocks and different numbers of iterations, and

the cluster scanning system for different numbers of

Fig. 2 The cluster scanning

system

Univ Access Inf Soc (2013) 12:237–245 239

123

clusters. The cluster scanning system performed best when

the number of clusters was five. However, among the dif-

ferent versions of Cluster and Block scanning processes, it

was found that a type of block scanning that divided the

screen into four equal sized partitions for four iterations

performed best.

It was expected that the cluster scanning process

would perform better since it uses information about

target types (e.g. labels are not considered as possible

targets) and locations in the clustering process. So, the

analysis was extended to consider the actual tasks

undertaken by participants. Most of the time, participants

used instant messenger software and browsed the World

Wide Web. The present version of the clustering process

does not consider locations of hyperlinks in the target

acquisition process, and so as a consequence it might

miss possible targets during Web surfing. The further

study revealed that participants should take less time to

complete a task using the cluster scanning system than

other scanning systems if the clustering process could

include all targets in a screen. Details of this study can be

found in [1].

4 The experiment

We validated results obtained in the simulation study

through a controlled experiment with users with motor

impairment. Details of the experiment are discussed in the

following sub-sections.

4.1 Participants

We collected data from 8 motor impaired and 8 able-

bodied participants (Table 1). The motor impaired partic-

ipants were recruited from a local centre, which works on

treatment and rehabilitation of disabled people and they

volunteered for the study. All motor impaired participants

used computer at least once each week. Able-bodied par-

ticipants were university students and expert computer

users. None of them had used the scanning systems before.

4.2 Procedure

In this experiment, the participants were instructed to

press a set of buttons placed on a screen (Fig. 3) in a

particular sequence. All the buttons were coloured grey

except the next target, which was red. The same task was

repeated for all the scanning systems. In particular, we

evaluated the cluster and block scanning systems. We

recorded cursor traces, target height, width and task

completion times. For internal validity of the experiment,

no scan delay adaptation algorithm was used. The scan

delay was kept constant at 2 s for motor impaired par-

ticipants and at 1 s for the control group. These values

were selected after observing their reaction time and were

greater than the reaction time. All participants were

trained adequately with the scanning systems before

undertaking the experiment.

4.3 Material

We used a push button switch [10] and an Acer Aspire

1640 Laptop with 1,280 9 800 pixel screen resolution. We

also used the same seating arrangement (same table height

and distance from table) for all participants. We have

investigated the effect of physical strength of users on the

scanning system. As all users could use their hands to

operate computer peripherals, their hand strength was

measured. Additionally, the hand evaluation devices are

cheap, easy to operate and have good test–retest reliability

[5]. So these are reliable and useful tools for measuring

physical strength making these results useful in practice.

The six variables listed below were measured for hand

strength evaluation. Each was measured three times and the

average was calculated. We evaluated only the dominant

hand (the hand participants used to operate the mouse).

Photographs of the measurement technique can be found in

Appendix 2.

Table 1 List of participants

Age Gender Impairment

C1 27 F Able-bodied

C2 28 F

C3 30 M

C4 30 M

C5 31 M

C6 28 F

C7 30 F

C8 26 F

P1 30 M Cerebral Palsy affecting manual dexterity,

wheel chair user

P2 43 M Cerebral Palsy affecting manual dexterity, also

have tremor in hand, wheel chair user

P3 30 M Dystonia, cannot speak, cannot move fingers,

wheelchair user

P4 62 M Left side (non-dominant) paralysed after a

stroke in 1973, also have tremor

P5 44 M Cerebral attack, significant tremor in whole

upper body part, fingers always remain folded

P6 46 F Did not mention disease, hard to grip things, no

tremor

P7 [45 F Spina Bifida/Hydrocephalus, wheelchair user

P8 [45 M Cerebral Palsy from birth, restricted hand

movement, no tremor

240 Univ Access Inf Soc (2013) 12:237–245

123

• Grip Strength measures how much force a person can

generate by gripping by hand. It was measured using a

mechanical dynamometer.

• Tip Pinch Strength measures the maximum force

generated by a person squeezing something between

the tips of the thumb and index finger and was

measured using a mechanical dynamometer.

The following ranges of motion are defined with respect

to the standard anatomical position [[4], (c.f. Appendix 2)].

• Radial deviation is the motion that rotates the wrist

away from the midline of the body when the person is

standing in the standard anatomical position [4]. When

the hand is placed over a table with palm facing down,

this motion rotates the hand about the wrist towards the

thumb. Maximum radial deviation was measured using

a goniometer.

• Ulnar deviation is the motion that rotates the wrist

towards the midline of the body when the person is

standing in the standard anatomical position. When the

hand is placed over a table with palm facing down, this

motion rotates the hand about the wrist towards the

little finger. It was measured with the goniometer.

• Pronation is the rotation of the forearm so that the palm

moves from a facing up position to a facing down

position and was measured using a wrist-inclinometer.

• Supination is the opposite of pronation, the rotation of

the forearm so that the palm moves from a facing down

position to a facing up position and again was measured

with the wrist-inclinometer.

4.4 Results

Initially, the total task completion time was measured for

the scanning systems (Table 2; Fig. 4). It can be seen that

participants took less time to complete the task using the

cluster scanning system. The dotted bars in Fig. 4 mean

that two participants could not complete the task using the

block scanning system.

To further investigate the scanning systems, we mea-

sured the following three dependent variables:

• Number of missed clicks: We counted the number of

times the participants wrongly pressed the switch.

• Idle Count: The scanning systems periodically high-

light the buttons. This variable measures the number of

cycles when the participants did not provide any input,

though they were expected to do so.

• Efficiency: The scanning systems require a minimum

time to complete any task which depends on the

particular scanning system and not on the performance

of the user. We calculated the efficiency as the ratio

Optimal TimeActual Time

. An efficiency of 100 % indicates opti-

mal performance, 50 % indicates taking twice the

minimal time, and 0 % indicates failure to complete the

task.

Table 2 presents the efficiency of each participant.

No significant difference was found between the per-

formances of motor impaired and able-bodied users by an

equal variance equal variance t test at p \ 0.05 level.

However, efficiency, the average number of missed clicks

and idle count are significantly lower in the cluster

scanning system than in the block scanning system in an

equal variance paired t test (p \ 0.05) (Fig. 5). Addi-

tionally, two participants (P3 and P7) could not complete

the task using the block scanning system, while all par-

ticipants could complete the task using the cluster scan-

ning system.

Table 3 shows the correlation coefficients of these

variables with the hand evaluation metrics. The only sig-

nificant effect is a correlation between the number of

missed clicks in the cluster scanning system and grip

Fig. 3 Screenshot of the

experiment

Univ Access Inf Soc (2013) 12:237–245 241

123

strength. There was a similar but weaker effect in the block

scanning system. It seems that hand strength does not affect

performance of users with the scanning systems. Also, no

effect of target size (height and width) on the task com-

pletion time was found, which is not surprising as the

scanning systems did not depend on the target size.

4.5 Discussion

The simulator predicted that the task completion time

would be less in the cluster scanning system than the block

scanning system when the cluster scanning system can

consider all possible targets in its clustering process. The

experiment also shows similar results. The total task

completion time, sub-optimal task completion time, idle

time and number of missed clicks are less in the cluster

scanning system than the block scanning system. The

efficiency of the cluster scanning system can be attributed

to the following factors:

• The cluster scanning system does not introduce any

new interface element like a frame or form in the screen

as Autonomia [8] or FastScanner [6] systems do.

• The cluster scanning system does not blindly divide the

screen in a predefined number of segments as the

ScanBuddy [9] or block scanning systems do. It clusters

the target so that the targets are evenly divided into

blocks and a block is not drawn in a region that does

not contain any target.

No effect of hand strength on pointing performance was

found while participants used the scanning systems. There

are two possible explanations:

• The switch used in scanning only requires a gentle push

to operate and the hand strength of motor impaired

users is sufficient to operate the switch.

• The scanning software does the navigation itself and

the users need not move their hand to move the pointer.

Table 2 Comparing scanning systems

Participants Cluster scanning system Block scanning system

Efficiency Task

completion

time (in s)

Efficiency Task

completion

time (in s)

P1 0.39 166 0.13 618

P2 0.76 86 0.62 125

P3 0.89 73 0.75 104

P4 0.58 113 0.15 171

P5 0.54 120 0.48 107

P6 0.61 107 0.33 236

P7 0.68 97 0.57 137

P8 0.50 131 0.47 165

C1 0.56 58 0.51 77

C2 0.84 39 0.53 73

C3 0.49 67 0.37 105

C4 0.76 43 0.63 62

C5 0.71 46 0.67 58

C6 0.56 59 0.57 68

C7 0.62 53 0.66 59

C8 0.42 77 0.46 84

Average 0.62 83 0.49 141

Comparing Cluster and Block Scanning Systems

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

P1 P2 P3 P4 P5 P6 P7 P8 C1 C2 C3 C4 C5 C6 C7 C8

Participants

Tas

k C

om

ple

tio

n T

ime

(in

mse

c)

Cluster Scanning System

Block Scanning System

Fig. 4 Task completion times

for the scanning systems

242 Univ Access Inf Soc (2013) 12:237–245

123

This result with the scanning system also shows that an

appropriate choice of an assistive technology can make

interaction independent of the physical strength of users.

This study also confirms the value of automatically

evaluating assistive interfaces using a simulator [1–3].

Before running a formal user trial, a system designer may

tune interface parameters or select the best design alter-

native using the simulator. As each alternative design does

not need to be evaluated by a user trial, the simulator will

reduce the development time significantly.

5 Conclusion

We have developed a new scanning system that works

by clustering screen objects. In a previous paper, we

evaluated the cluster scanning system and found it

superior to the block scanning system. This paper has

presented a study involving motor impaired users to

evaluate the cluster scanning system in practice and to

validate the results of the simulation. We found that the

total task completion time, efficiency, idle time and

number of missed clicks are less in the cluster scanning

system than for the block scanning system. The results in

turn also validate the simulator. The performance on the

scanning system does not also seem to depend on the

physical strength of users, or in other words physical

impairment does not seem to affect performance for the

scanning system. It can be inferred that motor impaired

users found the cluster scanning system faster, easier and

more accurate than the conventional block scanning

system.

Comparing Cluster and Block Scanning Systems

0.42

3.75

6.96

18.59

0.620.49

Avg. Missed Click

Avg. Idle Count

Avg. Efficiency

Block Scanning System

Cluster Scanning System

Fig. 5 Comparing the scanning

systems

Table 3 Correlation

coefficients for the scanning

systems

* Significant at p \ 0.05

Cluster scanning system Block scanning system

Missed click Idle count Efficiency Missed click Idle count Efficiency

Correlations

GS -0.580* -0.191 0.168 -0.429 -0.331 0.283

TPS -0.374 -0.105 0.110 -0.271 -0.153 0.093

ROM

Wrist

-0.414 -0.154 0.189 -0.127 -0.120 0.068

ROM

Forearm

0.000 0.106 -0.079 -0.268 -0.225 0.076

Significance

GS 0.018 0.478 0.534 0.097 0.210 0.289

TPS 0.153 0.699 0.686 0.310 0.572 0.731

ROM

Wrist

0.111 0.569 0.484 0.639 0.659 0.803

ROM

Forearm

1.000 0.695 0.770 0.315 0.401 0.778

Univ Access Inf Soc (2013) 12:237–245 243

123

Appendix 1: The cluster scanning algorithm

Subroutine Cluster Scan

Get Targets

Cluster Targets

End Subroutine

Subroutine Get Targets

Enumerate all currently running processes

If the process creates a window and the window is visible

Store its position

If the window is the foreground window

Store its handle

End if

Get all child processes

Get all Thread Processes

End if

End Subroutine

Subroutine Get Child Processes

Get Class name

//Store positions of buttons, textboxes and combo boxes

If class name contains “Button” or class name =”text” or class name =”combo”

Get Position

If Parent Process is Foreground Window

Store its position with a mark

Else

Store its position without any mark

End if

//Store positions of icons

Else If class name =”SysListView32”

Get number of Icons

Get Position of all Icons

If Parent Process is Foreground Window

Store positions of all icons with a mark

Else

Store positions of all icons without any mark

End if

End if

End Subroutine

Subroutine Get Thread Processes

If the process creates a Window and the window is visible

Get Window’s Position

If Parent Process is Foreground Window

Store its position with a mark

Else

Store its position without any mark

End if

End if

End Subroutine

Subroutine Cluster Targets

Get the percentage of area occupied by the foreground window

Run Fuzzy c-means algorithm on the targets of the foreground window with

values of c proportional to percentage of area occupied by the foreground

window

Run Fuzzy c-means algorithm on the rest of the targets with values of c

proportional to (1 - percentage of area occupied by the foreground window)

End Subroutine

Appendix 2: Measurement of hand strength

See Figs. 6, 7, 8.

Fig. 6 a Measuring grip strength, b Measuring tip pinch strength,

c Standard anatomical position

244 Univ Access Inf Soc (2013) 12:237–245

123

References

1. Biswas, P., Robinson, P.: A new screen scanning system based on

clustering screen objects. J. Assist. Technol. 2(3) Sept 2008,

ISSN: 1754-9450

2. Biswas, P., Robinson, P.: Evaluating interface layout for visually

and motor impaired users through simulation. Univers Access

Inform Soc 12(1), Springer, Jan 2012

3. Biswas, P., Robinson, P.: Evaluating the design of inclusive

interfaces by simulation. In: Proceedings of the ACM Interna-

tional Conference on Intelligent User Interfaces (IUI) 2010

4. Kaplan, R. J.: Physical medicine and rehabilitation review, 2nd

edn. MacGraw Hill, New York (2006)

5. Mathiowetz, V., Weber, K., Volland, G., Kashman, N.: Reli-

ability and validity of hand strength evaluation. J Hand Surg

(1984)

6. Ntoa, S., Savidis, A., Stephanidis, C.: FastScanner: an accessi-

bility tool for motor impaired users. In: Proceedings of the

ICCHP 2004, LNCS 3118, pp. 796–803 (2004)

7. Ross, T.J.: Fuzzy logic with engineering application, Interna-

tional Edition. McGraw-Hill, New York (1997)

8. Steriadis, C.E., Constantnou, P.: Using the scanning technique to

make an ordinary operating system accessible to motor impaired

users. The autonomia system. In: Proceedings of the International

ACM SIGACCESS Conference on Computers and Accessibility

(ASSET 2002)

9. The ScanBuddy system. Available from: www.ahf-net.com/

Scanbuddy.htm. Accessed on 21 May 2007

10. The Super-Switch. Available at: http://rjcooper.com/super-

switch/index.html. Accessed on 1 July 2007

Fig. 7 Measuring ranges of

motion of wrist. a Range of

motion of wrist (Palm facing

down), b Measuring radial

deviation, c Measuring ulnar

deviation

Fig. 8 Measuring ranges of motion of forearm. a Pronation, b Mea-

suring pronation, c Supination, d Measuring supination

Univ Access Inf Soc (2013) 12:237–245 245

123