Date post: | 20-Mar-2017 |
Category: |
Art & Photos |
Upload: | luruiyang |
View: | 38 times |
Download: | 0 times |
Mobile Augmented Reality
Augmented Reality
Mobile Computing
§ computing devices, linked by wireless networks, allow us to use computing facilities while roaming the real world
§ Today we also in WAR
Combines real and virtual world
Interactive in real time
Registered in 3D
Data Visualization
With Augmented reality, we can: Display data directly on data source (thermometer, gauges, …) without display
device. Connect data with physical context. Do not require display simplification (no context interpretation).
http://emcl.iwr.uniheidelberg.de/research sv.html)
http://www.maptek.com/
User Interaction
By augmented reality, we are able to interact with data: Without sucking yourself away from real world. Without physical dimension restriction. In a collaborative way.
http://vimeo.com/46304267 http://www.avatarmovie.com/index.html
Retail
In retail, augmented reality is able to: supply vertical retail to individual consumers (price, date, …). Display accurate product recommendation and personalized
advertisement. Enrich shopping experience.
http://www.asleyetracking.com/ https://www.youtube.com/watch?v=XM9ZOWPeiAk
Tourism
With big data, augmented reality enables to: Aggregate and compile the redundant fragmented data to build a detailed and
complete environmental model. (e.g. Google Earth) Provide personalized travel guide information. (intelligent recommendation is
regarded as the most attractive expectation of tourism. [Business Insider, 2012]) Gamification of travel (e.g. Ingress)
http://www.ingress.com/http://www.theguardian.com
Healthcare
AR enables to: Quickly access valuable information of patients. (with face recognition) Provide In-suit visualization of historical illnesses or tissue damage. Give health suggestion based on health stats and diet. Enable remote diagnosis in a collaborative way.
http://www.iqiyi.com/v_19rrh3pj3k.html https://www.youtube.com/watch?v=jZkHpNnXLB0&feature=kp
Mobile Augmented Reality
Develop the underlying technology
Develop prototype projects
Produce high standard AR applications
A bottom-up approach
�
Computation offloading�
Object tracking� Localization� Privacy� Hand gesture
recognition�Face recognition�
Mobile Augmented
Reality�
Driving assistant
Natural user interface Privacy
protection
Reading assistant
“ 27 published and filed patents in the field of
Mobile Augmented Reality (including Mobile Computing,
Networking, Privacy, Localization)
Total number of document(s): 15
Experts hand it to an augmented future
South China Morning Post | 2016-09-21Newspaper | CITY3 | CITY | technology | By Kinling Lo
Word Count: 248words | Image No: 1/1 | Image Size: 131cm-sq(14.4cm x 9.1cm)
科大打造AR開發平台 慳時間省成本有望「一統江湖」
Ta Kung Pao | 2016-09-21Newspaper | A18 | 教育
Word Count: 695words | Image No: 1/1 | Image Size: 508cm-sq(25.8cm x 19.7cm)
科大研AR新平台 App一周寫好
Wen Wei Po | 2016-09-21Newspaper | A21 | 新聞透視眼
Word Count: 794words | Image No: 1/1 | Image Size: 319cm-sq(22cm x 14.5cm)
科大研發平台 拒AR記錄臉孔
Hong Kong Economic Times | 2016-09-21Newspaper | A18 | 互聯網+ | By 黃蘊華
Word Count: 1,053words | Image No: 1/1 | Image Size: 408cm-sq(32.4cm x 12.6cm)
科大實驗室申AR 領域18項專利
Hong Kong Commercial Daily | 2016-09-21Newspaper | A15 | 香港新聞
Word Count: 579words | Image No: 1/1 | Image Size: 174cm-sq(8.2cm x 21.2cm)
雲機實鏡技術加快程式運作
Ta Kung Pao | 2016-09-21Newspaper | A18 | 教育
Word Count: 522words | Image No: 1/1 | Image Size: 124cm-sq(12.7cm x 9.8cm)
科大AR研究申18項專利
Sing Tao Daily | 2016-09-21Newspaper | F02 | 教育
Word Count: 592words | Image No: 1/1 | Image Size: 230cm-sq(25.3cm x 9.1cm)
Ubiquitous interface and interaction Towards Seamless Interaction between Digital and Physical Worlds
BACKGROUND
Traditional GUI
´ Display digital information
Augmented Reality
´ Unobtrusive
´ Sensing in environment
´ Blend physical and digital worlds
Ubii - Overview An Intergrated interface which
allows users to interact with objects in the environment with hand gestures
Take advantage of hand-
gesture recognition and object tracking
Connect and communicate
with smart devices in the environment
RELATED WORK
Interaction at a distance
´ Manipulate content that are unreachable
´ Manipulate devices that are incapable of touch interaction
Freehand interaction
´ Natural, intuitive, effective
´ Mid-air interaction
´ Recognize hand gestures/ body movements
S. Boring, D. Baur, A. Butz, S. Gustafson, and P. Baudisch.Touch projector: mobile interaction through video.
Microsoft Kinect
SYSTEM DESIGN — physical affordance
computer – file transfer printer - printing projector screen - projecting physical surface – file manipulation
all made possible with simple hand gestures
System flow of Ubii
SYSTEM DESIGN — Menu Design
Placement • Object-referenced placement • Attach menu to physical object • Tag visual marker on physical
object for alignment
SYSTEM DESIGN — Menu Design
Placement Orientation
• Align the menu with physical object surface
• Improve readability • Better 3D spatial presence
SYSTEM DESIGN — Menu Design
Placement Orientation Trigger Mechanism
• Menu are activated and deactivated based on the hand gestures
pinch normal
SYSTEM DESIGN — Menu Design
Placement Orientation Trigger Mechanism Ring Menu
• Menu items are distributed on an ring around the object
• A few menu items are active and the unused ones are folded
• Ring menu can be rotated to change the active items
The rotational Ubii menu
SYSTEM DESIGN — Interaction Design
a) pick b) drop c) drag d) rotate ring menu e) zoom
The 5 supported hand gestures of Ubii
IMPLEMENTATION — Hand Gesture Detection
´ Sample skin color in HSV space
´ Extract hand contours
´ Wrap hand contours with polygonal hulls
´ Calculate convexity defects to identify pinch gestures
Identifying pinch, drop and no-pinch gestures
Sampling skin colors from sample points
IMPLEMENTATION — Hand Gesture Detection
´ Algorithm can distinguish hand gestures from the background
´ Can distinguish between a pinch gesture and non-pinch holes
The pinch gesture detection. a) a typical pinch gesture; b) outer contour of hand is extracted; c) and d)red
closed regions are recognized as pinch holes; e) and f) green closed regions are eliminated as non-pinch holes.
EVALUATION — How effective Ubii is
1. Copying documents between computers
2. Printing documents 3. Displaying documents
on projector screens 4. Sharing documents
Comparison of task competition time of four experiments by using Ubii and traditional
methods.
On average, Ubii can reduce the operation time by at least half
Most participants are satisfied with their experience
EVALUATION — How effective Ubii is
Background
´ Technologies benefit lives, but also raise privacy concerns!
´ Easier for people to take photos without obvious signals
´ Popularity of online photo sharing platforms
´ Advanced recognition techniques
´ Google Glass is spotted as an example of rising visual privacy concerns from the public
Motivation
´ User studies
´ Consent mechanism is welcomed when being recorded
´ Life loggers care about the privacy of bystanders
´ Privacy concerns depend on the context: who, what, when, where, why, and how
´ Limitations of previous solutions
´ Static policies
´ Aesthetically awkward
´ Extra sensors (e.g., infrared imager)
Design Principles
´ Cardea design
´ Problem setting: mobile/wearable devices with built-in camera
´ Technical solution: computer vision techniques
´ Protection enforcement time/level: in situ/application level
´ Protection object: bystanders’ visual privacy
´ Opt-in vs. opt-out: opt-in
´ Objectives
´ Context dependent
´ Individualized
´ Dynamic
Design Overview
´ Allowing individuals to proactively convey their
context-dependent privacy preferences bound with
face features.
´ Location
´ Scene
´ People in the image
´ Hand gestures can be used to interact with cameras
to temporarily update current preferences
´ Bystanders can also use static tags as privacy
indicators.
System Architecture
§ Bystander app: registration; setting privacy preference profile § Recorder app: taking images § Cloud: storing users’ profiles and training face recognition model;
responding to clients; requests
Scene Classification
´ Places2 dataset: 401 scene categories, 10 million training images.
´ Training a classifier on 9 general scene categories which are common or sensitive.
Evaluation – scene classification
´ 8 volunteers take 759 images “in the wild”, with 638 images selected, covering 9 general scene groups
(a) recall (b) Confusion matrix
Evaluation – face recognition and matching
´ Selecting 50 subjects from LFW dataset, 5042 features for training and validation, 511 features as user test set, 166 features from 100 other subjects as non-user test set.
Evaluation - face recognition
´ Selecting 50 subjects from LFW dataset, 5042 features for training and validation, 511 features as user test set, 166 features from 100 other subjects as non-user test set.
(a) Training accuracy (b) Testing accuracy with probability threshold
Evaluation - face matching
´ Selecting 23 subjects from user test set who has more than 10 features. Using these 230 features as database features, and the other 281 features as query features.
(a) Cosine distance (b) Euclidean distance
Evaluation – gesture recognition
(a) Recall for different scenes
(b) Precision for different scenes
´ 338 hand gesture images with 208 “Yes” gesture, 211 “No” gestures, and 363 natural hands.
Evaluation – scene classification
´ 8 volunteers take 759 images “in the wild”, with 638 images selected, covering 9 general scene groups
(a) recall (b) Confusion matrix
Evaluation – Runtime
´ Client: Samsung Galaxy Note 4
´ Server: Intel i7-5820K CPU, 16GB RAM, GeForce 980Ti Graphic Card
´ Network: eduroam
(a) runtime
Evaluation – overall performance
´ 5 volunteers register as Cardea users, set their privacy profiles. In total, we take 224 images for evaluation.
Cardea summary
´ Design
´ Context awareness
´ Interactive control
´ Implementation
´ Deep neural networks
´ Deployment on android
´ Feasibility
´ Micro vision benchmarks
´ Overall performance
´ Runtime evaluation
The future
Using AR technology, users can view information of the devices around them. They can also visualize data and communicate with the Internet of Things
The future
With augmented reality and a natural user interface, users can interact with any smart objects in the environment In a factory, users can control and issue commands to robots with hand gestures