ITAG 2012, Nottingham, 24th Oct 2012 1

Spatial Mental Models and Navigation Support Apps for People who are Blind

Lindsay Evett, Malcolm Harrison, Allan Ridley, David Brown and Nick Shopland

Interactive Systems Research Group (ISRG)Computing and Technology TeamNottingham Trent University, UK

ITAG 2012, Nottingham, 24th Oct 2012 2

Independent Travel • People who are blind generally use sequential, route-

based strategies• Dependent on sighted guides to learn new routes in a

sequential way (using cane, guide dogs)• Quite a few systems support navigation outside (e.g.,

Trekker, smart phone apps; some indoors – e.g., see Battersby, this session)

• Some systems focus on obstacle avoidance, replacing sight, often have intrusive audio feed back

• Better to complement existing methods (dog and cane), focus on goal (getting safely from A to B) and skills, in an effective and usable manner

ITAG 2012, Nottingham, 24th Oct 2012 3

@rgh!

ITAG 2012, Nottingham, 24th Oct 2012 4

That’s more like it!

Egocentric vs. Exocentric/Allocentric

• Egocentric, sequential routes hard to learn (Martinsen et al, 2007)

• Body (egocentric), rather than external (exocentric/allocentric), frame of reference

• Those with more map based, exocentric, frames of reference perform better in navigational tasks (Simonnet et al, 2006)

• Map based navigation better than GPS route-based (Ishikawa, et al, 2008)

• “Cognitive maps” – enhance confidence, flexibility

ITAG 2012, Nottingham, 24th Oct 2012 5

ITAG 2012, Nottingham, 24th Oct 2012 6

Cognitive Maps

• External frames of reference and map based strategies more efficient and flexible – easier to remember, can take alternative routes, shortcuts, change destinations – they offer a more complete spatial representation (Martinsen et al, 2007)

• Active exploration of accessible maps to support development of map-based strategies and spatial mental models (Tversky, 1993) to facilitate independent navigational skills (Cognitive Collages)

• Can use at leisure (unlike real world support)

ITAG 2012, Nottingham, 24th Oct 2012 7

Spatial Mental Models

• Tversky (1993)– Mental representations of knowledge of environment not

simply representations of maps (include memory of journeys, maps, directions, landmarks etc)

– Disparate pieces of knowledge about environments – cognitive collages rather than single, coherent map; evidenced by errors of memory and descriptions

– For well-known environments, representations of spatial layout quite accurate, allowing perspective taking, re-orientation, spatial inference – Spatial Mental Models; not metric but preserve coarse spatial relations and includes reference points

ITAG 2012, Nottingham, 24th Oct 2012 8

GPS vs Maps and Direct experience

Ishikawa et al (2008) compared mobile GPS to paper maps and direct experience for walking routes:

– GPS users travelled longer distances, made more stops than the others

– GPS users travelled more slowly, made larger direction errors, drew poorer maps, found wayfinding more difficult than direct experience

– Reasons? GPS maps small (how much surrounding area needed?); GPS has local focus; less experience with GPS

ITAG 2012, Nottingham, 24th Oct 2012 9

Spatial vs Route Information

Others have pointed out the importance of spatial, rather than just route, information:

– Showing users their exact location relative to where they need to get to can help people who struggle with spatial skills

– Oliver and Burnett – route guidance systems suppress cognitive map development

– Lindstrom – necessary for people with disabilities to assimilate a mental map, and should be built in a mobile environment

– Lindstrom – users with disabilities stress the importance of being located should they no longer be able to orientate themselves

Questions

• Many navigational apps; how useful are they?• Find out about environment, benefits?• Find out about environment where known routes,

benefits?• Find out about environment, extend route, new

route, benefits?• Apps offer similar apparent (?) support but

differences in scale, detail, availability of global information, accessibility

ITAG 2012, Nottingham, 24th Oct 2012 10

11

AEGIS, HaptiMap Prototypes

• Mobile apps use GPS and map-based info; some are accessible

• We conducted usability evaluations of TouchOver Map and NavPoint (HaptiMap project, 2011; Brown et al, 2011)

• AEGIS and HaptiMap prototypes, and other apps, support navigation; do they support creation of Cognitive Maps, Spatial Mental Models?

• RECALL and VC projects based on proposal that Cognitive Maps (Global Reference Frames, see Frankenstein et al, 2011) and spatial strategies more effective for independent travel that route guidance (Hill et al, 1993)

ITAG 2012, Nottingham, 24th Oct 2012

ITAG 2012, Nottingham, 24th Oct 2012 12

Touch Over maps• HaptiMap project (2011) prototype haptic

map (Poppinga et al, 2011)• Touch-based with speech and haptic

feedback; user actions have a direct, reliable, relationship with maps

• Physical edges provide simple reference points

• Two blind users found it easy to use, could reproduce the map

• Blind user with residual vision found areas with patchy feedback, frustrating and poor info. (can use Google maps)

• Android app, tablet and mobile

ITAG 2012, Nottingham, 24th Oct 2012 13

Point Nav

• The PointNav HaptiMap prototype allows scanning an area for points of interest, selecting one of them, finding out more about this point and being guided to it

• Same two users found it easy to use• PointNav is implemented on Android OS,

provides speech and vibratory feedback. Allows loading of points of interest lists

• Set reference points from study of map?

ITAG 2012, Nottingham, 24th Oct 2012 14

Range of AppsApp Platform Accessible? Usable? Global/local

iPhone Maps Apple OS Yes (route) Yes Both – global not accessible – BUT…

Navigation Android Yes (route) Yes Both – global not accessible

Where am I? Android (Code Factory)

Yes No! Local – BUT inaccurate!

Walky Talky Android Yes…. Not really, too unstable

Local

Trekker Windows CE Yes (not for all – no visual)

Yes Local, but with some look ahead

NavPoint Android Yes (not set up)

Yes Can be Global

TouchOver Map Android Yes (not set up)

Yes Global

ITAG 2012, Nottingham, 24th Oct 2012 15

Evaluate Spatial Mental Models

Does the solution make spatial relationships available, allow multiple perspectives, sufficient for actual route finding? Possible tasks:1. Recreate map – verbal description or model2. Describe routes from A to B and from C to B (taking different

perspectives)3. Real tasks in areas of a known route:

• demonstrate ability to deal with obstacles• extend known route• demonstrate ability to orient with respect to reference points• create new route

ITAG 2012, Nottingham, 24th Oct 2012 16

Can Apps Support Independent Travel for People who are Blind?

• New address, no routes, no chance of trainers• (Learnt route to get there through Google

maps, Google street view, Trekker)• Use range of apps, with global and local

information to support and supplement• Extend route(s), new routes?• Development of cognitive map? Is having a

cognitive map better than routes alone?

ITAG 2012, Nottingham, 24th Oct 2012 17

Three New Routes!!

• Explored street map of new area with TouchOver Map, for as long as wanted, and to find 3 targets – about 40 minutes

• Active, independent exploration• From start point, point to targets and describe how to get

there• Find those targets in the real world• Yes!!• NavPoint useful but tricky• Other apps not much help – WalkyTalky aims to do what

Trekker does but didn’t work; new iPhone does

ITAG 2012, Nottingham, 24th Oct 2012 18

Further Evaluation

• Spatial mental models:– Point to other locations?– Landmarks?

• Is SMM memorable, can it be communicated?• Larger group – can others do it?

– Compare with and without traditional training

NB needs accessible set up, including search, size (s, m, l) and store

19

Location Based Services for People with Disabilities (this session)

• Location-based services, route learning and navigational support for wide range of disabilities

• Travel skills and confidence to learn new routes are core skills for leading independent lives

• Showing users their exact location relative to where they need to get to, and significant points, can help people who struggle with spatial skills

• Games-based learning• Incorporate haptic map?

ITAG 2012, Nottingham, 24th Oct 2012

ITAG 2012, Nottingham, 24th Oct 2012 20

Additional issues

• Safety – can’t just wave a phone about• Moving it around could change responses

(NavPoint)• Shoulder speaker/wizzy earphones

• Phone calls!

• Map accessibility in iOS 6; Walkit.com

21

Conclusions• User Centred Design, designing AT using technology

that doesn’t mark out people as ‘other’, cheap, easily available, mainstream – address abandonment of AT

• Haptic map supports/facilitates creation of Spatial Mental Models to support independent travel, independence for people who are blind

• Haptic maps – outdoors (and indoors)

• Location Based Services, navigation support – incorporate haptic maps (outdoors) Lewis et al, 2012

• Virtual Cane – indoors (outdoors?) Battersby, 2012

ITAG 2012, Nottingham, 24th Oct 2012

22

Contact Details

Dr Lindsay Evett,Prof. David J. Brown and the

Interactive Systems Research GroupComputing and Technology Team

NTU, UKTel: +44 115 848 8359

lindsay.evett@ntu.ac.uk

Tel: +44 115 848 8350david.brown@ntu.ac.uk

ITAG 2012, Nottingham, 24th Oct 2012

23

Acknowledgements

This research supports EU Leonardo Projects GOAL NET (UK/07/LLP-LdV/TOI-009); GOET (UK/08/LLP-LdV/TOI/163_181) and RECALL (504970-LLP-1-2009-1-UK-KA3-KA3MP); AEGIS – Integrated Project within the ICT programme of FP7

ITAG 2012, Nottingham, 24th Oct 2012

24

SourcesBattersby S J (2012), The mobile virtual cane, ITAG12, Nottingham UK Brown DJ, Standen PJ, Evett L, Battersby S and Shopland N. (2010). Designing Serious Games for People with Dual Diagnosis: Learning Disabilities and Sensory Impairments. In Educational Gaming. Chapter in Zemliansky, P and Wilcox, D. M., (eds), Educational Gaming, IGI GlobalBrown, D. J., Evett, L. J., Ridley, A., Shopland, N., Merritt, P., Harrison, M. and Van Isacker, K. (2011) Evaluation of Haptic RIA Maps and other navigational support systems for people who are blind, paper presented to 2nd International AEGIS Conference, Brussels, Belgium, November 2011, pp 334-341, http://www.epr.eu/aegis/wp-content/uploads/2011/12/AEGIS_Conference_proceedings-final.pdf accessed 9/1/12Frankenstein, J, Mohler, B, Bulthof, H H, and Meilinger, T (2011) Is the map in our head oriented North? Psychological Science, 23, 2: pp. 120-125HaptiMaps (2011). HaptiMap, Haptic, Audio and Visual Interfaces for Maps and Location Based Services, http://www.haptimap.org/ accessed 23/11/11Hill E W, J. J. Rieser, M. Hill, J. Halpin, R. Halpin (1993). How persons with visual impairments explore novel spaces: strategies of good and poor performers, J. Vis. Imp. and Blindness, 87, 8, pp. 295-301.Humanware (2012) Trekker Breeze, http://www.humanware.com/en-united_kingdom/products/blindness/talking_gps/trekker_breeze accessed 22/10/12 Ishikawa, T., Fujiwara, H., Imai, O., and Okabe, A. (2008) Wayfinding with a GPS-based mobile navigation system: A comparison with maps and direct experience, Journal of Environmental Psychology, 28, 74-82

ITAG 2012, Nottingham, 24th Oct 2012

ITAG 2012, Nottingham, 24th Oct 2012 25

Sources (contd.)Lewis, J., Popescu, C., Standen, P., and Saridaki, M. (2012) Evaluation of route learning software on Android for people with disabilities, ITAG12, Nottingham, UKLindstrom, J-I. (2006) New technologies to help people with disabilities and elderly people, in Roe, P. R. W. (ed) Towards and Inclusive Future: Impact and Wider potential of Information and Communication Technologies, COST219ter, funded by the EU RTD Framework Programme, http://www.snapi.org.uk/cost219ter/inclusive_future/index.htm accessed 22/10/12Martinsen, H., J. M. Tellevik, B. Elmerskog, M. Storlilokken (2007). Mental effort in mobility route learning, Journal of Visual Impairment and Blindness, 101, pp1-18.Oliver, K. J., G. E. Burnett, (2008). Learning-oriented vehicle navigation systems: a preliminary investigation in a driving simulator. In Proceedings of the 10th International Conference on Human–Computer Interaction with Mobile Devices and Services. pp. 119–126.Poppinga, B., Pielot, M., Magnusson, C., and Rassmus-Grohn, K. (2011) TouchOver Map: Audio-tectile exploration of interactive maps, MobileHCI 2011, Stockholm, Sweden, ACMSimonnet M, J-Y. Guinard J. Tisseau (2006) Preliminary work for vocal and haptic navigation software for blind sailors, Proc. 6th Intl Conf. Disability, Virtual Reality & Assoc. Tech. (ICDVRAT), Esbjerg, Denmark, 2006, pp. 255-262.Tversky, B. (1993). Cognitive maps, cognitive collages and spatial mental models, in Frank, A U and Campari, I (Eds.) Spatial Information Theory: A Theoretical Basis for GIS, Proceedings COSIT ‘93, Lecture Notes in Computer Science, 716, pp. 14-24, Springer, Berlin.