Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | vivian-dixon |
View: | 219 times |
Download: | 0 times |
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
CUbiC
ARIZONA STATE UNIVERSITY
Sreekar Krishna, Vineeth Balasubramanian, Sethuraman (Panch) Panchanathan
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
CUbiC
Enriching Social Situational Awareness in Remote Interactions -
Insights and Inspirations from Disability Focused Research
ARIZONA STATE UNIVERSITY
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
2
CUbiC
Recognition &
Learning
Sensing &
Processing Inter
actio
n &
Deliv
ery
Assistive Tech.Technologies for Daily
Living
Rehabilitation Assessment & Training
Medical Decision Support
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
3
CUbiC Success Stories - iCARE ReaderCamera
Phase 1• 2005• 3 prototypes developed• Deployed in ASU and AzSDB
Phase 2• 2006• Personal size• Customization capabilities
Phase 3• In development• Incorporating high resolution digital cameras on the glasses
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
4
CUbiC Success Stories - NoteTaker
The assistive technology for low vision and legally blind students for taking notes in the classroom
Zoomed video of the lecturer’s presentation in real time
Student notes with digital ink
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Multi Touch Camera ControlWinner of 2010 MS Imagine Cup
Award
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
5
Shopping Assistance
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
6
Social Assistance - Origin of the Problem
Dr. Terri Hedgpeth – Director, Disability Resource Center, ASU
Focus Group Study of Individuals who are Blind:
• “It would be nice to walk into a room and immediately get to know who are all in front of me before they start a conversation”.
• “It would be great to walk into a bar and identify a friend”.
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Enactor (Encoding) Recipient (Decoding)
Interpersonal Social Interactions
7
N. Ambady and R. Rosenthal, “Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.,” Psychological Bulletin. Vol. 111(2), vol. 111, Mar. 1992, pp. 256-274.
Body
Voice
Speech
Face
27%19%
Visual
18%35%
Audio
Ver
bal
Non
-ver
bal
65%
35%
Social Touch
What role does touch
play?
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
8
Socio-Behavior Example – Hand Shake
Social Interactions
Sensory
Perceptual
Cognitive
Motor
Handshake
Step 1: Eye Contact Step 3: Move (Proxemics)Step 2: Intent to interact
Step 4: Shake hands Step 5: Conversation distance
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
9
Social Interactions
Social Touch
Social SightSocial Hearing
Social
Stimulation
Social
Reciprocation
Face
Body
VoiceSocial
Cognition
Social Stimulation
Social Cognition
Social Reciprocation
Social Situational Awareness
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
• How many people?• Where are they located?• What are their facial
expressions?• Eye Gaze• Eye Contact• Body Mannerisms
SSA in Various Settings
10
Social Assistance Decision Making Remote Collaborations
TeamSTEPPS
• Leadership• Mutual Support• Communication• Attitude• Situation Monitoring• Patient Safety
• Expressing Opinion• Managing Conflict• Making Decision• Speed of Decision• Interaction with
Colleagues• Difficulty Establishing
Rapport
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
CUbiC Multidisciplinary Team
11
Technology, Psychology & Human Comm. Tech. Dissemination & Validation
Assistive TechnologyTeam
Dr. Sethuraman PanchanathanComp. Sci. ASU
Human Centered Multimedia
Dr. John Black Jr.Comp. Sci. ASUAssistive Tech.
Specialist
Dr. Vineeth Balasubramanian
Comp. Sci. ASUDecision Sys. & Risk
Analysis
Dr. Prasad BoradkarSchool of Design, ASUAssistive Tech. Design,
Interdisciplinary Design Initiatives
Dr. Jameson WetmoreConsortium of Science, Policy
& Tech., ASUAsst. Tech. Ethics, Practices &
Effective Dissemination
Dr. Terri HedgpethDir. Disability Resource
Center, ASUAsst. Tech. Usability
Expert, Early Adoption Specialist
Dr. Michele ShiotaPsychology, ASU
Interpersonal Interactions, Facial Expression, Dyadic Communications
Dr. Don HomaPsychology, ASU
Visual Perception, Working Memory, Haptic Concepts
Dr. Artemio RamirezHuman Comm., ASURemote Interactions
and Communications, Modeling Professional
Meetings, Conflict
Sreekar KrishnaElec. Engg.
ASUIntegration Engg.
Behavioral Psychology and Communication Team
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
12
Social Interaction Assistant
Stereotypy
Face ReadingSocial Scene Analysis
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Stereotypy
Any non-functional repetitive behaviorTwo main causes for stereotypy
Lack of sensory feedbackLack of cognitive feedback
Methods of control Stereotypy
• Curtail Behavior immediately
• Reward / PunishmentIntervention
• Do not intervene directly• Develop cognitive
replacement
Self Monitoring
13
Body Rocking is the most prevalent
stereotypy for people who are
blind and visually impaired
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Proposed solution
XY
Z
Rocking
Non - Rocking
14
Rocking action can be recognized with an
accuracy of 94% within 2 seconds
Behavioral Psychology literature shows that one rock action is approximately 2.2 seconds long. Effectively, recognizing a rocking behavior well within one rock cycle.
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
15
Dyadic Interaction – Face Reading
Camera
Social Interaction Asst.
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
16
Dynamic Delivery of Facial Mannerisms
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
17
Social Gaze & Interaction Space
IntimatePersonal
SocialPublic
1.5’ 4’ 12’ 25’0’
Interpersonal Space
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
18
Modeling Distance & Direction through Face Detection
Module 1: Color Analysis
Module 2: Markov Random Field LPCD
n
j
HzHzh
dopt
dk
kj
Tkj
opte
nh
zP1
2
1
2
12
)2(
1)(
Module 3: Evidence Aggregation
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Structured Mode Searching Particle Filter (SMSPF)
Initial Estimat
e
Corrected Estimate
Example Search
Windows
Motivation: Weak Temporal Redundancy
Approach: Stochastic Search over a large search space (Color Histogram Comparison)
Result: Approximate Estimate
Step 1Step 2
Motivation:ComplexObject Structure & Abrupt Motion
Approach: Deterministic Search over a small probable search space (Histogram of Gradients with Chamfer Match)
Result: Accurate Estimate
19
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
20
Face/Person Detection/Tracking
Face Detection Person Detection
Tracking
Model
Deliver
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
21
Social Scene Delivery System
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
22
Conveying Body Mannerisms
Body Posture Body Gestures
Enactor
Recipient
Social Mirror
Social Interaction Assistant
Social Interaction Assistant
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
Disability & Deficit Inspired Computing
Disabled Population
Activities of Daily Living (ADL)
Observe
Identify Barriers to ADL
Design and Develop Assistive
Tech.
Refine
Disseminate
Extrapolate to the general population
Did you know the typewriter was invented for the blind?
Blindness is only a concept
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
24
Mayo Multi-disciplinary Simulation Center
Doctors, Nurses, Professionals, etc.
On-body Affect Sensors
Environment Affect SensorsVision Audio
• Automated monitoring of group dynamics to determine communication breakdowns• Automatic evaluation of the social affinity between team members• Leadership evaluation and nomination through long term monitoring of teams and
individuals
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
25
Assisting Remote Interactions
Se-ries
158% 60% 62% 64% 66% 68% 70% 72% 74%
73%
69%
64%
Managing Conflict
Making Decision
Expressing Opinion
Challenges in virtual teams compared to face-to-face teamsTop five challenges faced during virtual team meeting
Series1
50 55 60 65 70 75 80 85 90 95
90
80
77
76
75
Different leadership styles
Insufficient time to build relations
Colleague not participate
Method of decision making
Speed of decision making
Personal challenges during virtual team meetings
Series1
45 55 65 75 85 95 105
94
85
81
77
68
66
Difficulty seeing the whole picture
Absence of collegiality
Sense of isolation
Reliance on email and telephone
Difficulty establishing rapport and trust
Inability to read non-verbal cues
The Challenges of Working in Virtual Teams: Virtual Teams Survey Report 2010. RW3 CultureWizard, 2010.
CENTER FOR COGNITIVE UBIQUITOUS COMPUTING
26
Socio-Behavioral Computing
Sensor & Actuator
Technologies
Sensor & Actuator
Technologies
Human Computer Interaction
Human Computer Interaction
Machine Learning and
Pattern Recognition
Machine Learning and
Pattern Recognition
SBC
Socio Behavioral Computing
• Affective Computing
• Social Robotics
• Human Communication Dynamics
• Human Centered Computing
• User Behavior Modeling
Questions?