Date post: | 21-Jan-2018 |
Category: |
Technology |
Upload: | dataversity |
View: | 238 times |
Download: | 4 times |
See Me, Feel Me, Touch Me, Heal Me The Rise of the Cognitive Interface
Adrian Bowles, PhDFounder, STORM Insights, Inc.
Lead Analyst, AI, Aragon Research
NOVEMBER 9, 2017
Context: Advanced Human Computer Interfaces Perception & Perceptive Input Management Emotive Output When to Use Advanced IO
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
AGENDA
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
NATURAL HCI HAS BEEN AN AI GOAL SINCE THE BEGINNING
PERCEPTIONUNDERSTANDING
LEARNING
PROBLEM-SOLVING
NATURAL LANGUAGE
PROCESSING
Big Data
Classic AI
Deep Learning
Context
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
HOW DO WE COMMUNICATE?
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
NLP
Natural Language Understanding NLU
Natural Language Generation NLG
?
COMMUNICATIONS GOES BEYOND NATURAL LANGUAGE PROCESSING (NLP)
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
People are good - but not perfect - when communicating in natural languages. We…
understand in context (environment & our own frame of reference) attempt to resolve ambiguity have to deal with competing signals, noise fill in words and meaning and may not hear/understand use our knowledge about the speaker and the situation to filter the input use visual and auditory cues to distinguish between what was said vs what was meant…
X
X’
Y
HOW DO PEOPLE COPE WITH IMPRECISION?
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
WHAT DO WE COMMUNICATE?
Facts
I send you a text at 2AM something bad happened
Concepts
I am angry
I am inconsiderate
I exaggerate
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
“You’re Not Just Responsible For What You Say, You Are Responsible For What People Hear”
August 9, 2016
General Michael Hayden (Retired), former Director, Central Intelligence Agency and National Security Agency
on advice to his senior staff.
NATURAL LANGUAGE PROCESSING IS HARD - FOR PEOPLE
Watch out for slang, context…
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
PICTURES VS WORDS
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
GESTURES ADD RICHNESS TO LANGUAGE
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
“THAT’S A BIG HOUSE” - MEANING MAY BE DIFFERENT FOR DIFFERENT SPEAKERS
Bob Mary
Al (Capone)
Wikipedia contributors. "Alcatraz Federal Penitentiary." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 30 Oct. 2017. Web. 8 Nov. 2017.
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
DRAW A QUARTER, TO SCALE - RESULTS DIFFER ACCORDING TO HIDDEN CONTEXT
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
Hearing (audioception) ~12,000 outer hair cells/ear
~3,500 inner hair cells
Vision (ophthalmoception) Photoreceptors - Per Eye~120,000,000 rod cells
(triggered by single photon)~6,000,000 cone cells
(require more photons to trigger)~ 60,000 photosensitive ganglion cells
Touch (tactioception) Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration
Smell (olfacoception) Chemoreception
Taste (gustaoception) Chemoreception
Human Cognition~100,000,000,000 (100B) Neurons
~100-500,000,000,000,000 (100-500T) Synapses
THE HUMAN ADVANTAGE FOR PERCEPTION & COGNITION
LearnReason
Understand
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
Listen Look
Touch
Smell Taste
Your System
LearnModel
Reason
Understand
PERCEPTIVE INPUT: IT’S A NEW SENSATION Perception
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
HOW DO WE COMMUNICATE?
A Mathematical Theory of Communication, C. E. Shannon,
The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, July, October, 1948.
Systems
Controls
LearnReason
Understand
Model
Data Mgmt
Human
Machine
Input Output
Gestures
Emotions
Language
Narrative Generation
Visualization
Reports
Haptics
Sensors (IOT)
SystemsControls
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
COGNITIVE SYSTEMS: COMMUNICATIONS & CONTROL
Perc
eptio
n
Human Input
Human-Oriented Output
GesturesEmotions
Language
Narrative GenerationVisualization
Reports
Haptics
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
COGNITIVE SYSTEMS: COMMUNICATIONS & CONTROL
Perc
eptio
n
MeaningConcepts
Intent
Context
Animation
LearnReason
Understand
Model
Data Mgmt
Human Input
Gestures
Language
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
PERCEPTIVE INPUT CLASSIFICATION
Context
LearnReason
Understand
Model
Data Mgmt
Detected by Human Senses
Derived
ImagesSee
Hear
Touch
SmellTaste
Sounds
ObjectsEmotions
Meaning
Concepts
Intent
MULTI-DIMENSIONAL INPUT ANALYSIS
Source: IBM Bluemix
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
TONE ANALYSIS OF TEXT: IBM WATSON
Emotion tones: Anger, Disgust, Fear, Joy, and Sadness.
Writing tones: Analytical, Confident, and Tentative.
Social tones: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.
Source: IBM
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
UNDERLYING ALGORITHMS ARE COMPLEX
Source: Affectiva
Depend on detection and classification of word use for text and physiological state changes for audio and video.
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
DEEP LEARNING FOR FEATURE, ENTITY, CONCEPT EXTRACTION/RESOLUTION (SIMPLIFIED!)
Visible Layer
Hidden Layers
Output Layer
Input: Observable Variables
H
IGH
AB
STR
ACTI
ON
LO
W
Output
Pixels
Depthof the Model
Edges
Object
Shapes/Parts
Object Class
Brightness/Contrast
GeometryRules
Featuresto
Extract
Methods
1. Feed Forward
2. Back-Propagate(adjust weights)
Model
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
CONCEPTS IN SPACE: IMPORTANT ASSUMPTIONS
Principles that control the development and representation of natural intelligence in the neocortex provide a guide to the implementation of machine intelligence.(Numenta Hierarchical Temporal Memory) A function applied to a string representing data or a concept results in a value or vector meaningful for comparison.
e.g. Using Kolmogorov complexity to measure the strength of relationships in Memory-Based Reasoning.
DistanceA metric for the similarity of two items based on their relative locations in n-space according to an algorithm.
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
Proximity Mapping
BoyBay
Map
Mop
Man
Nay May
MopeBuy
Hop
Mapped with vectors, proximity algorithm based on purpose.
Mapping for autocorrect/complete vs Mapping for meaning
BoyBay
Map
Mop
Man
Nay
May
Mope
BuyHop
Hope
HopeSimilar structure -> similar meaning in vision, not always in language.
CONCEPTS IN SPACE
Model
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
COGNITIVE COMPUTING FUNDAMENTALS: SAMPLE ASSUMPTIONS
ModelThe Corpus, Assumptions, Algorithms
Used to Generate & Score Hypotheses
or Calculate The Strength of a Relationship
Relationships can be discovered without understanding the meaning of any individual string, fact, or (NL-free concept modeling)
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
RECOGNIZING CONCEPTS VS UNDERSTANDING
Courtesy of LoopAI Labs.
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
EMOTION DETECTION IN VIDEO
7 EmotionsAnger Sadness Disgust Joy Surprise Fear Contempt
Source: Affectiva
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
KEY TREND: EMOTION DETECTION AS A FEATURE OR SERVICE
Source: Affectiva
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
HCI ADVANCES AUGMENT UNDERSTANDINGG
estu
re R
ecog
nitio
n/An
alyt
ics
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
NEXT TREND? USING SMARTPHONE SENSORS FOR RICHER CONTEXT
Type Typical Current Use
Accelerometer/motion Rotate screen, Switch screen to landscape/portrait
Ambient Light Adjust screen brightness
Barometer Measure altitude
Geo-Location (wifi/cellular) Location/Alerts
3-Axis Gyroscope Rotation rate for games, VR…
Proximity Turn off screen when phone is by your head
Touch ID fingerprint Security
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
EXPRESSIVE OUTPUT: EMOTIONS, URGENCY
Model
Emotions Meaning
Concepts IntentContext
Narrative GenerationImages
HapticsAnimation
Using Language, Avatars, Emotive Text to Speech
Emotive Output
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
EMOTIVE TEXT TO SPEECH IS NOT NEW
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
DATA Model/Article
Structure Template
Generated Narrative
NLG TECHNOLOGY CAN USE EMOTIONS IF THE MODEL SUPPORTS IT
Needs Context-Relevant Tone Assignment Data
Model
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
Copyright (c) Narrative Science.
COMMERCIAL NLG TECHNOLOGY WITH TONE ATTRIBUTES
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
NEUROBEHAVIORAL ANIMATION
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
Sentiment/Emotion/Theme/Concept Analysis
Don’t let the search for perfection interfere with the path to progress.
WHEN SHOULD YOU USE PERCEPTIVE INPUT/EMOTIVE OUTPUT? Getting Started
Copyright (c) 2017 by STORM Insights Inc. All Rights Reserved.
WHEN SHOULD YOU USE PERCEPTIVE INPUT/EMOTIVE OUTPUT?
When It Improves the User Experience
The System Tone Should Complement,Not Mirror, the User Tone
When You Can Select or Generate Appropriate Responses/Actions
Twitter @ajbowles Skype ajbowles
If you would like to connect on LinkedIn, please let me know that you that you registered for the Smart Data webinar series.
KEEP IN TOUCH
Upcoming SmartData Webinar Dates & Topics
Dec. 14 The Road to Autonomous ApplicationsJan. 11 AI At The Edge: Pushing Intelligence to Fog Computing NodesFeb. 8 A Pragmatic AI Maturity Model: Choosing the Right AI Technologies Based on Application Requirements and User Attributes