Date post: | 17-Dec-2015 |
Category: |
Documents |
Upload: | brendan-skinner |
View: | 215 times |
Download: | 0 times |
Can you see me now? Disability assistance in mobile devicesMark StumpGraduate StudentDepartment of Computer EngineeringRochester Institute of Technology
Agenda
•Introduction•Computer Vision •Android•Android/CV Fusion and Challenges•Problem•Preliminary Implementation and Results•Questions/Answers
Computer Vision
•How computers see and perceive the world
•Very diverse field with much up and coming development
Image Source:http://startupblog.files.wordpress.com/2009/10/webcam.jpg
Image Source: http://www.elexp.com/test/ping.jpg
Computer Vision•Subject Areas
▫Feature Detection Edge Detection Blob Detection Corner Detection
▫Recognition Optical Character
Recognition (OCR) Pose Estimation Gesture Recognition
•And many, many moreImage Source: http://stevehanov.ca/blog/index.php?id=62
Face Detection• Recognizes a face in an image
or video feed• Has many applications
▫Digital Cameras Autofocus
▫Security Face as a security Code
▫Advertising▫Energy Saving
Televisions detecting when someone is not looking at the screen and decreasing the brightness Image Source:
http://blog.cordiner.net/wp-content/uploads/2010/02/lenna_output.jpg
Face Detection• Various algorithms developed• Find faces through various
methods▫Comparing against a large
training set▫Looking for components of a
face in a scene (eyes, mouth, etc)
• With further development, can extract other information▫Gender▫Mood
Image Source: http://www.dcviews.com/press/images/Face-detection.jpg
Face Tracking•After a face is detected
▫Follow it as it moves through a scene
•Completed many ways▫Fast face detection▫Color Analysis▫Body analysis▫Template Matching
Image Source: http://www.globalspec.com/NpaPics/38/121978_020520089775_ExhibitPic.JPG
Android Introduction• Created by Google• “Software stack for mobile
devices that includes an operating system, middleware and key applications”
• Generally used in Cell Phones▫HTC Hero▫HTC Magic▫Motorola Droid
• Also beginning to branch other devices▫Dell Mini 5 Tablet PC▫Acer Aspire One D250 Netbook▫Barnes and Noble Nook eReader
Image Source: http://en.wikipedia.org/wiki/File:Android_and_cupcake.jpg
Android SDK•Freely available •Currently in release 2.2
(FroYo)•Provides for interfacing with
various portions of an Android device▫Screen▫Wireless (Bluetooth, Wifi)▫Motion Sensing▫GPS▫Camera▫Telephony
Image Source: http://developer.android.com/assets/images/home/froyo-android.png
Platform
•Applications primarily written in Java•Plugin available for Eclipse IDE•Runs of native “Dalvik Virtual Machine”
▫Register-based Architecture▫Used for platforms with limited resources
(like a phone)•Applications can also be written in C/C++
▫Android Native Development Kit (NDK)
Device Development (HTC Magic)• Android Phone, marketed by T-Mobile as
‘myTouch 3G’• Specifications
▫Operating System: Android 1.6▫ Processor: Qualcomm® MSM7200A™, 528 MHz▫ Memory
512 MB ROM 288 MB RAM
▫ Display: 3.2-inch TFT-LCD flat touch-sensitive screen with 320x480 HVGA resolution
▫ Connectivity Bluetooth 2.0 Wi-Fi 802.11 b/g Proprietary USB Interface (Mini-USB 2.0/Audio Jack in
one)▫ 3.2 Megapixel Camera with auto focus
Device Development (Motorola Droid)• Android Phone, marketed by Motorola on Verizon
Network• Specifications
▫Operating System: Android 2.1▫ Processor: Arm® Cortex™, 550 MHz▫ Memory
512 MB ROM 256 MB RAM
▫ Display: 3.7-inch 16:9 Widescreen touch-sensitive screen with 480x854 WVGA resolution
▫ Connectivity Bluetooth 2.1 + EDR Wi-Fi 802.11 b/g Micro USB Interface
▫ 5 Megapixel Camera with auto focus and dual-LED Flash
Image Source: http://www.softsailor.com/wp-content/uploads/2009/10/Motorola-Droid-Multimedia-Station.jpg
Image Source:http://smartphonesfans.com/wp-content/uploads/2010/03/motorola-droid.jpg
Device Development (HTC Evo)• Android Phone, marketed by HTC on Sprint
Network• Specifications
▫Operating System: Android 2.1▫ Processor: Qualcomm® Snapdragon™ 8650, 1 GHz▫ Memory
1 GB ROM 512 MB RAM
▫ Display: 4.3-inch touch-sensitive screen with 800x480 WVGA resolution
▫ Connectivity Bluetooth 2.1 + EDR Wi-Fi 802.11 b/g Micro USB Interface HDMI Output
▫ 8 Megapixel Camera with auto focus and dual-LED Flash With 1.3 MP front-facing camera
Image Source: http://nexus404.com/Blog/wp-content/uploads2/2010/03/Sprint-HTC-EVO-4G-1.jpg
Image Source:http://www.hypebeast.com/image/2010/03/htc-evo-phone-4.jpg
Android/Computer Vision
•Potential for almost any application to be developed
•Numerous Augmented Reality applications already available for platform
Image Source:http://www.uwplatt.edu/web/presentations/PennState/ar/pix/augmented-reality-hud.jpg
Google Goggles• Application created by Google Labs• “Visual Search Application”• Can take pictures of various objects and
complete searches based on them▫Landmarks▫Books▫Artwork▫Places▫Wine▫Logos▫Business Cards
Can pull phone numbers from a card to import into phone to call, store, etc.
Google Goggles - Landmark
Image Source: http://www.google.com/mobile/images/labs/goggles/goggles_landmark.jpg
Google Goggles – Business Card
Image Source: http://www.google.com/mobile/images/labs/goggles/goggles_contact.png
Google Goggles – Wine
Image Source: http://www.google.com/mobile/images/labs/goggles/goggles_wine.jpg
Scotoma (Blind Spots)•“A spot in the visual
field in which vision is absent or deficient”
•Causes▫Macular Degeneration
Degeneration of the eye▫Optic neuritis
Inflammation, Degeneration of the Optic Nerve
▫And others..Image Source: http://en.wikipedia.org/wiki/File:CentralScotoma.jpg
Scotoma Treatments•Treatments center
around stopping the spot from growing larger
•For most cases, no cure•Effects (Very Situational)
▫Impairs functional acuity▫Can affect color
perception
Image Source: http://www.incm.cnrs-mrs.fr/pperso/portraits/scotoma.png
Scotoma Effects (Continued)• ‘Eccentric Viewing’
▫Also called Preferred Retinal Loci (PRL)
▫“This is similar to looking slightly away from an object at night in order to see it better” Dan Roberts, via
Macular Generation Support, www.mdsupport.org
Image Source: http://www.acbvi.org/albums/Vision/images/112p_donut.jpg
My Goal!
•To develop a system that will aid a person in compensating with various types of scotoma▫Through the use of
mobile devices
Finally, my development•Program to help those with various scotoma
interact•Limit (and possibly eliminate) ‘eccentric
viewing’ •Goals
▫Run efficiently and quickly▫Provide easy-to-use and configurable interface▫Be implementable on a mobile device▫Implement cutting-edge face detection methods
developed in-house ‘Random Projection’-Based Methods
Current Status•Prototype Implemented on a lab machine•Relatively Inefficient Implementation•Relies heavily on OpenCV graphical libraries for
interface▫OpenCV is a freely available computer vision
library that provides many standard algorithms and aids for implementation
▫Originally launched by Intel in 1999•Uses ‘off-the-shelf’ algorithms for face
detection▫Viola-Jones Detection
Initial Android Development•Program written to perform edge detection
on various images•Uses standard edge detection method
▫Sobel Kernel •Further extended to implement the on-
board camera on the phone
Initial Android Development
HTC Magic (Java-Based Implementation)
HTC Magic (OpenCV-Based Implementation)
Initial Android Development
Motorola Droid(Java-Based Implementation)
Motorola Droid(OpenCV-Based Implementation)
Future•Research into optimizing hand-offs between
native libraries and interface•Optimize/Port previous lab-machine code to
Android•Experiment with different interface types
for configuration and use
References/Further Information• Merriam-Webster’s Online Dictionary
▫ http://www.merriam-webster.com/dictionary• Causes of Scotoma
▫ http://www.wrongdiagnosis.com/s/scotoma/causes.htm• Scotoma – Texas School for the Blind and Visually
Impaired▫ http://www.tsbvi.edu/Education/anomalies/scotoma.htm
• Eccentric Viewing▫ http://www.mdsupport.org/library/eccentric.html
• Energy saving of TV by face detection▫ http://portal.acm.org.offcampus.lib.washington.edu/citati
on.cfm?doid=1389586.1389693• Google Goggles Information
▫ http://www.google.com/mobile/goggles/