Date post: | 09-Dec-2014 |
Category: |
Technology |
Upload: | marc-pous |
View: | 1,632 times |
Download: | 4 times |
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Multimodal Interaction in Distributed and Ubiquitous Computing
Marc Pous and Luigi Ceccaronibdigital
ICIW 2010 - The Fifth International Conference on Internet and Web Applications and Services
May 9 - 15, 2010 - Barcelona, Spain
• Why multimodality?
• Multimodal distributed services deployment
• User tests and conclusions
SUMMARY
In our cities, urban services practically have not changed in a century
1: Location-based services
MOTIVATIONS
http://www.flickr.com/photos/nnova/4512346900/
http://picasaweb.google.com/marc.pous/Xina2007#5102695479944027890
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415374502175450370
http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415371534810675298
http://www.flickr.com/photos/loquat73/3385335980/
2: Interactions and multimodality
MOTIVATIONS
http://www.flickr.com/photos/nnova/4423756547/in/set-72157623598200316/
http://www.flickr.com/photos/nnova/4424521014/in/set-72157623598200316/
3: Accessibility
MOTIVATIONS
http://www.dynamiclanguageblog.com/2009/12/technology-grows-for-hard-of-hearing.html
MOTIVATIONS
What is an ICD?(Interactive Community Display)
INREDIS
3G = ACCESSIBILITY + UBIQUITY + INTEROPERABILITY
http://www.inredis.es
INREDIS ICD
Management of distributed services that offer the capability of processing and synthesizing multiple modalities of interaction
DISTRIBUTED INTERACTION SERVICES
• Voice services– Voice recognition
• Via microphone, real-time streaming converting the voice signal into text
– Voice synthesis
• Sign language service– Spanish sign language recognition
• Via webcam, real-time streaming converting the image into text
• Emotion service– Emotion recognition (voice + image)
• Via webcam and microphone, real-time streaming
• Avatar service– Emotional avatar– Sign-language speaker avatar
DISTRIBUTED SERVICES
SIGN LANGUAGE SERVICES
DISTRIBUTED SERVICES
VOICE SERVICES
How to consume multimodal distributed services offered by an ICD?
W3C MMI Architecture
• W3C MMI Architecture
http://www.w3.org/TR/mmi-arch/
W3C MMI Framework
http://www.w3.org/TR/mmi-framework/
Proposed multimodal interaction framework
• ICD Architecture
ICD ARCHITECTURE
(1) Interaction identification
(2) Main page- Map or avatar- Command buttons- Content- Scenario options
(3) Contextualized map with services
(4) Avatar with map and Street View service
ICD DESIGN and INTERFACES
USER–CENTERED DESIGN
• User profiles (Personas)– Blind users– Deaf users
• Unable to read– People living or visiting a city
• Scenarios– Context-aware informative scenario– Emergency scenario
IMPLEMENTATION
USABILITY TESTING
• PLI (People Led Innovation) methodology
• Preliminar usability tests with a limited number of users
• But the ICD implementation received good feedback!
CONCLUSIONS and FUTURE WORK
• Implementation based on the W3C MMI Architecture idea• Platform able to integrate multimodal interactive distributed
services and offer interaction in real-time• Web-based applications and device-independent
implementation• Improvement of the user experience and accessibility of
people with special needs
What are we working on now?– Real-time interaction: delay reduction between interactions– Browser-based apps not being able to easily access device hardware– Synchronous interaction vs. Asynchronous technologies– Improvement of the distributed services orchestration– Enhancement of the mobile usability
MOBILE PHONES?
http://www.slideshare.net/nokiaconversations/younghee-jung-and-jan-chipchase-presentation-for-nokia-design-london-studio-event-london-29th-april-2008
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc
Thank you!
Moltes gràcies!
¡Muchas gracias!
Marc Pous and Luigi [email protected] and [email protected]