17
Insert the title of your presentation here Presented by Name Here Job Title - Date Exploring measures of usability for in-vehicle technology Presented by Alistair Weare HF Researcher at TRL
Insert the title of your presentation herePresented by Name HereJob Title - Date
Exploring measures of usability for in-vehicle technology
Presented by Alistair WeareHF Researcher at TRL
Page 2
Agenda
Introduction: What is usability?
Design Guidelines
Methods related to usability measurement
Conclusions
Questions / Feedback
1
2
3
4
5
Introduction: What is usability?
Page 3
Usability is an accepted concept, but not rigidly refined- Nielsen (1993) – 5 key usability attributes: Learnability, Efficiency,
Memorability, Errors, Satisfaction- ISO 9241: ‘The extent to which a product can be used by specified
users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’
- Jordan (1998) – 5 higher-order components of usability: Guessability, Learnability, Experienced User Potential (EUP), System potential, Re-usability (or memorability)
- ISO 17287 defined a new and related concept - ‘suitability’ - as: ‘The degree to which a [system] is appropriate in the context of the driving environment based on compatibility with the primary driving task’
Introduction: What is usability?
Page 4
In determining the usability of an IVIS (whether designed specifically for use in a vehicle or not), there must be an understanding of how the system fits into the larger vehicle-driver-road system
It must be useful to the driver within the higher driving task, efficient such that it presents a minimal distraction, and its ease of use must be compatible with any competing demands on the driver at the time of use (which may or may not be when driving)
Page 5
Design Guidance Sources
European regulations
There is currently little in the way of European legislation specifically related to the HMI of IVIS
European regulations may be a consideration in the future
Regulations and Standards
United States regulations
laws about in-vehicle distraction generally fall under the jurisdiction of individual states but with some at the national (federal) level
As an example of national provision, in October 2009 President Obama issued an Executive Order prohibiting Federal employees from texting while driving
International standards
Not legally binding, but form a framework for a common design philosophy
Standards attempt to define best practice and so may form basis for mandatory regulations
Play an important role, but only if kept up to date
Page 6
Design Guidance Sources
Japan: JAMA
25 specific requirements that apply to driver interfaces to ensure safe operation whilst driving
Requirements are grouped into: Installation of Display Systems, Functions of Display Systems, Display System Operation While Vehicle in Motion, and Presentation of Information to Users
Design Guidelines
United States: Alliance and
NHTSA
The US motor vehicle manufacturers have developed ‘Alliance Guidelines’ that cover similar, high-level, design principles to the ESoP
NHTSA guidelines are based on the ESoP/Alliance guidelines and introduce some specific assessment procedures
Provides a more definitive assessment but does rely on the testing of participants to determine levels of distraction (sampling issues)
Europe: European
statement of principles
High-level HMI design guidance with status as recommended code of practice for use in Europe
34 design principles, grouped into: Overall Design Principles, Installation Principles, Information Principles, Interactions with Controls and Displays Principles, System Behaviour Principles and Information about the System Principles
Design Guidance Sources
Page 7
Some international work to standardise warning signals One technical specification on prioritization of warning signals:
- ISO/TS 16951: Road Vehicles – Ergonomic aspects of transport information and control systems – Procedures for determining priority of on-board messages presented to drivers
Two technical reports containing guidance information:- ISO/PDTR 16352: Road Vehicles – Ergonomic aspects of transport
information and control systems – MMI of warning systems in vehicles- ISO/PDTR 12204: Road Vehicles – Ergonomic aspects of transport
information and control systems – Introduction to integrating safety critical and time critical warning signals
Warning Guidelines
Design Guidelines
Page 8
Key issue relating to ADAS is driver controllability, determined by:- by the possibility and driver’s capability to perceive the criticality of a
situation- the driver’s capability to decide on appropriate countermeasures
(such as overriding or switching off the system)- the driver’s ability to perform any chosen countermeasures (taking
account of the driver’s reaction time, sensory-motor speed and accuracy)
Controllability must exist at all levels of interaction: - during normal use within system limits- at and beyond system limits - during and after system failures
Driver Assistance Systems Guidelines
Design Guidelines
Page 9
Two key European projects relevant to controllability:- RESPONSE:
- Developed a Code of Practice for defining, designing and validating ADAS- The Code describes current procedures used by the vehicle industry to
develop safe ADAS with particular emphasis on the human factors requirements for ‘controllability’
- ADVISORS:- Attempted to integrate the RESPONSE Code within a wider framework of user-
centred design taking account of the usability of information, warning and assistance systems
(IHRA-ITS) - There is also activity by the International Harmonized Research
Activities – Intelligent Transport Systems Working Group to develop a set of high-level principles for the design of driver assistance systems
Driver Assistance Systems Guidelines
Page 10
Methods related to usability measurement
Evaluate actual driver performance when using the product/system in a
realistic context of use
Depends on:1. Which environment the method is used in (road, test
track, simulator, laboratory etc.)2. Which task manipulations occur (multiple task, single
task loading, no tasks given etc.) 3. Which dependent variables (operationalised as metrics)
are of interest.
Evaluate how well a product/system meets the design principles in the
relevant guidelines
Methods for assessing IVIS usability can take two different approaches:
Page 11
Methods related to usability measurement
Participants drive a real vehicle
Wide range of metrics Usually a complex set of
obderved behaviours
Road trials Carefully controlled
simulated environment Specific objective
measures Key benefit is
experimental repeatability
Simulator trials Focus on visual demand
of IVS Metrics include: time
taken to complete task, number of glances and number of errors
Occlusion
Available methods for evaluating actual driver performance:
Often used as part of sim study
Participants respond to changes in periphery
Speed and accuracy measured
Peripheral detection Standardised method
using sim environment Participant manoeuvres
compared to a normative model
Lane-change task Desk-based Component operations of
a task are timed Enables predictions of
overall task times
Keystroke Level Model
Page 12
Methods related to usability measurement
Evaluate actual driver performance when using the product/system in a
realistic context of use
Evaluate how well a product/system meets the design principles in the
relevant guidelines
Possible to develop checklists based on the design principles laid down in the guidelines (e.g. ESoP, NHTSA and JAMA guidelines).
To this end a functional IVIS usability checklist has been developed, based on an existing checklist produced for the UK Government in the late 1990’s, and incorporates requirements taken from the ESoP.
Methods for assessing IVIS usability can take two different approaches:
Methods related to usability measurement
Page 13
IVIS usability checklist
Excel Spreadsheet for PC, tablet etc.
Supportive Information for each question is available via a ‘help’ icon
Assessment Summary Sheet is automatically populated based on the data entered
Methods related to usability measurement
Page 14
Questions need to be sufficiently ‘elemental’ Three Checklist questions require further work on measurements
and evaluation criteria:- Is the IVIS securely fitted?- Is the IVIS visual display positioned close to the driver’s normal line of
sight?- Are presented messages visually simple?
Potential to include a rating system, whereby differing systems can be compared in terms of their overall usability
However, five key difficult issues hindering this:
IVIS usability checklist: Issues / considerations
Methods related to usability measurement
Page 15
Elements - Which elements are included within the rating? (E.g. all Checklist questions)
Scoring* - How are the individual elements scored? (E.g. +3/0/-3 or 1-10)- *Also, is it “better” to have a feature, even if poorly designed, than for
that feature to be absent; and how should the scoring reflect this? Weighting - How are the individual elements weighted? (E.g. all
even, high and low weights, individual weights) Combining - How are the scores and weights combined? Rating - How is the final number converted into the consumer
rating?
IVIS usability checklist: Issues / considerations
Conclusions
Page 16
Guidelines and checklists are helpful design aids
Usability is really important and this means usability within the context of use
HMI technology is always developing
Specific quantitative requirements are more likely to stifle innovation than promote safety
Safety is only partly about design – it’s mostly about driver behaviour
![[insert generic newspaper title here] #2](https://static.documents.pub/doc/80x56/568c493e1a28ab4916936830/insert-generic-newspaper-title-here-2.jpg)
