Mobile Monday in SwitzerlandSmart Camera Applications
Mobile Monday – April ’09 1
Juha K. Laurila ([email protected])
Nokia Research Center (Lausanne)
April 6th - 2009
Nokia, NRC and Smart Camera Applications • Nokia – Devices and Services
• www.ovi.com
• Biggest camera manufacturer• Camera in 60% of manufactured devices - in many of them two
• higher volume than overall (stand-alone) digital camera market
Markets
Group Executive Board
Corporate Development Office
Corporate Funct ions
Services
Devices
Nokia Siemens Networks
• Rich set of sensors in mobile devices –
Mobile Monday – April ’09 2
• Rich set of sensors in mobile devices –endless opportunities for context-aware services and applications
• Camera = sensor
• NRC – commercial impact from deep science
• Outline:• Mobile augmented reality (MARA)
• Mirror world (Image Space)
Shopping
Weather
HotelInformation
Restaurant reviews& menus Calla taxi
Imagine a world where anythingcan be considered interactive …
Mobile Augmented Reality • Sensors• GPS
• compass
• accelerometers
• Camera• used to give context over which icons are overlaid
Mobile Monday – April ’09 3
Augmented Reality Based on Image RecognitionSystem Overview
GPS
Mobile Monday – April ’09 4
Server
Nokia Point & Find: Press Release, April 1st - 2009
Nokia Introduces Nokia Point & Find, a new way to Connect With Information and Services on the go Apr 01, 2009
New platform enables businesses to engage with consumers in real time with relevant content
San Francisco, CA, USA - Nokia introduces an innovative service concept that enables people on the move to access relevant information and services on the internet, simply by pointing their mobile phone camera at real-life objects. A beta version of Nokia Point & Find, focusing on movies, is now available in the UK and US. Capabilities will later expand into other services and countries.
"Simply by pointing their camera phone at a poster for a new movie, people can watch the trailer, read reviews, and find the closest cinema where it is playing," said Philipp Schloter, General Manager, Nokia Point & Find.
Nokia Point & Find is an open service platform on which other companies can build innovative customized experiences to drive better engagement with potential
Mobile Monday – April ’09 5
innovative customized experiences to drive better engagement with potential customers. Nokia is inviting businesses, content providers and agencies to discuss how the Nokia Point & Find service could help them with specific applications, campaigns or promotional activities. Unique experiences can be created through the self-service Nokia Point & Find Management Portal, or by working with the Nokia Point & Find professional service team.
Nokia Point & Find uses advanced real time image processing and recognition technologies to link the user to digital content and services. It also recognizes bar codes, integrates GPS positioning technology, and supports category-specific text-entry search. When the phone is pointed at an object, Nokia Point & Find uses a variety of the phone's capabilities including the camera, internet connectivity, and GPS positioning to evaluate the object. Then, by rapidly searching through a database of virtually tagged items, the system identifies the object and returns a set of links to associated content and services.
Companies interested in using Nokia Point & Find for their own campaigns can visit http://pointandfind.nokia.com to request further information. Nokia Point & Find's beta release is available for download into select mobile devices in the UK and US at http://pointandfind.nokia.com as well.
Other Possible Use CasesLandmark-based navigation
Mobile Monday – April ’09 6
Nokia Image Space Experience places and merge digital and real world via mobile device
Browser•Intuitive navigation and participation via the Flash based browser•Enjoy, create and share Scenes –photo enabled narratives, supported by the community content in the service
Browser•Intuitive navigation and participation via the Flash based browser•Enjoy, create and share Scenes –photo enabled narratives, supported by the community content in the service
Overview
•User takes photos, uploads them to service and the system automatically locates them and sets the attitude (heading, yaw and roll)
Overview
•User takes photos, uploads them to service and the system automatically locates them and sets the attitude (heading, yaw and roll)
Summary: A service which offers an easy to generate and intuitive to navigate image-based mirror world. Any type of media content can be placed to this world for exploring and experiencing via a web browser and mobile client. Strong community aspect.
Mobile Client•Records the camera pose when taking the photo•Enables viewing digital information with sensors in situ and a far
Mobile Client•Records the camera pose when taking the photo•Enables viewing digital information with sensors in situ and a far
community content in the servicecommunity content in the serviceand roll)
•Unbroken, immediate chain from taking photos to enjoying them in enriched and contextualized way
•Brings together people, places and content into an easy to grasp reference model
•From dense network of photos the system creates a point-cloud model and provides enhanced navigation interface
and roll)
•Unbroken, immediate chain from taking photos to enjoying them in enriched and contextualized way
•Brings together people, places and content into an easy to grasp reference model
•From dense network of photos the system creates a point-cloud model and provides enhanced navigation interface
SummarySome augmented reality and mirror world opportunities:• Content centric communities
• With contextualization• Dynamic content – from users and commercial providers
• Improve and simplify photowork and other media management• Navigation• Commercial use cases
• Real estate agencies etc.
Mobile Monday – April ’09 8
• Blended realities – Why?• Main metaphor: magic lens - annotate view of real world• Access information visually, on location
• Devices• hand-held camera phone NOW• data glasses FUTURE
Thank you !
Acknowledgements to Mixed Reality research teams in NRC-Tampere, NRC-Palo Alto and NRC-Helsinki
Mobile Monday – April ’09 9
NRC-Tampere, NRC-Palo Alto and NRC-Helsinki
Background Material
Mobile Monday – April ’09 10
Background Material
Nokia Research Center (NRC)
• Global corporate research unit
• Unique partner network
• Open innovation mode
• Industrial value from deep International organizations Corporations
Open source developers
Venture capitalcompanies
Operators &service providers
Universit ies & research institutes
Mobile Monday – April ’09 11
• Industrial value from deep science
• Contribute to renewal and transition of the company
• Research in four focus areas
• Presence in Switzerland since 2008
Rich Context ModelingRich Context Modeling New User InterfaceNew User Interface
High Performance Mobile PlatformsHigh Performance Mobile Platforms
Cognit ive RadioCognit ive RadioCognit ive Radio
organizations Corporations
Virtuality Continuum*) – Our FocusSingle-playergames
Face-to-facediscussion
Mobile Monday – April ’09 12
REALITY VIRTUAL REALITYAUGMENTEDREALITY
MIXEDREALITY
SOCIALLY-DEFINED, DIGITALLY-ENRICHED ENVIRONMENTS
INFORMATION OF THE PHYSICAL IS ENHANCING THE VIRTUAL WORLDS
AUGMENTEDVIRTUALITY
*) P. Milgram, et al., “Augmented Reality: A class of displays on the reality-virtuality continuum”
Mixed Reality Components – Our Focus
AUGMENTATION
LIFELOGGINGAUGMENTEDREALITY
… of people, objectsand places
Context-orchestrated rich content experiences and management, Experience Explorer
Interface & interact with your contextualized environment in situ, MARA Platform
INTERFACE & NETWORKS
INTERACTION
PERSONAL
INTERACTION
INTERACTION BETWEEN IN-PERSON
ESENTATIONS
Mixed reality services with sophisticated contextual interfaces for experiencing and managingyour digital life and your contextualized surroundings with the richest dynamic content
SIMULATION
INTIMATEEXTERNAL
VIRTUALWORLDS
MIRRORWORLDS
User-created mirror worlds as reflections of the physical world, Image Space
Extension of self-creativity and avatar interaction outside games
MODELS & IMMERSIONIDENTITY &
INTERACTION
SENSORS &
EVERYWARE
LARGE-SYSTEM
MONITORING
INTERACTION BETWEEN IN
AND VIRTUAL REPRESEN