Geoportal Usability Evaluation
Xin HE, Hans Persson, Anders Östman
FPX, IHT, HIG
Sweden
Geoportal Usability Evaluation
Xin HE, Hans Persson, Anders Östman
FPX, IHT, HIG
Sweden
GEOTEST
2
GeoTest
• A project jointly initiated by
– National Land Survey of Sweden
– University of Gävle
– Future Position X
• A GIS cluster organization in Gävle, Sweden
3
[GeoTest] Aims
• Measuring Swedish SDI
• Contributing the implementation of INSPIRE in Sweden
4
[GeoTest] Scopes
• Performance
• Usability
• Feasibility
• (Cost & Benefit)
• (Data quality)
5
GEOTEST USABILITY EVALUATION
6
GeoTest Usability Evaluation
• ISO standards
• Institute for Humane Technology, Sweden
• FPX GeoUsability Lab
• Established network of test panels
7
[GeoTest] Applying ISO Standards
• Evaluate usability of GI services by applying standard
frameworks
– ISO 9241-11: Guidance on Usability
– ISO 9241-210: Human-centered design for interactive
system
– ISO 20282-3: Test method for measuring usability
8
GEOPORTAL
9
Geoportal
• An access point to different sources of GI on the web
• Many Member States provide access to their
infrastructures through geoportals
10
Geoportal Examples
11
USABILITY
12
Usability
• “Extent to which a product can be used by specified
users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of
use.”
• (ISO 9241-11:1998, definition 3.1)
13
[Usability] Effectiveness
• “the accuracy and completeness with which users
achieve specified goals”
• (ISO 9241-11:1998, definition 3.2)
14
[Usability] Efficiency
• “the resources expended in relation to the accuracy
and completeness with which users achieve goals”
• (ISO 9241-11:1998, definition 3.3)
15
[Usability] Satisfaction
• “freedom from discomfort, and positive attitudes
towards the use of the product”
• (ISO 9241-11:1998, definition 3.4)
16
USABILITY EVALUATION
17
Usability Framework (ISO 9241-11)
18
Evaluating Usability
• Understanding the goal for which the tool is designed
• Context of use in test
– Test panel
– Test tasks
• Usability measures
19
Usability Measures
• Effectiveness
– Level of completion
– Errors
• Efficiency
– Time
– (Cognitive workload)
• Satisfaction
– Psychometric questionnaire, e.g. SUMI and SUS
20
USABILITY EVALUATION OF A SWEDISH
GEOPORTAL
21
Geodata.se
22
Test Panel
23
Test Tasks
24
Usability Measures
25
SUMI
• Software usability measurement inventory (Kirakowski
and Corbett, 1993)
26
GeoUsability Lab
• 13.3 inches laptops
• Windows-based
• IE 8 installed
27
GeoUsability Lab
• Recording
– Task completion
– Time spent
– Keyboard strokes
– Mouse movements and
clicks
– Facial and oral expressions
28
[Results] Effectiveness and Efficiency
29
[Results] User Satisfaction
30
[Results] Design weakness
31
[Results] Differences in age groups
32
CONCLUSIONS
33
Conclusions from the pilot study
• Usability consideration should be a part of design
• Usability test detects design weakness
• Furthermore, it helps in understanding
– The activity
– Users
• It works for GI applications such as geoportals
34
Future works
• More tests
• Smarter method
– Test panel (Public, expert users, user experts?)
– Psychometric measurement
– Statistical inference
• Context of using geodata
35
Conclusions
• Feedbacks
• Design weakness detection
• Benchmarking
• Best practice and specification
36
Any questions?Any questions?