Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 217 times |
Download: | 1 times |
EE141
How to Motivate How to Motivate Machines to Learn and Machines to Learn and Help Humans in Making Help Humans in Making Water Decisions?Water Decisions? Janusz StarzykSchool of Electrical Engineering and Computer Science, Ohio University, USA
www.ent.ohiou.edu/~starzyk
EE141
Embodied Intelligence (EI) Embodiment of Mind EI Interaction with Environment How to Motivate a Machine Goal Creation Hierarchy Goal Creation Experiment Promises of EI
To economy To society
OutlineOutline
EE141
“…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological
basis of memory storage in neurons. “…The question of intelligence is the last great
terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted
to brain research
IntelligenceIntelligence
AI’s holy grailFrom Pattie Maes MIT Media Lab
EE141
Traditional AITraditional AI Embodied Intelligence Embodied Intelligence Abstract intelligence
attempt to simulate “highest” human faculties:
– language, discursive reason, mathematics, abstract problem solving
Environment model Condition for problem
solving in abstract way “brain in a vat”
Embodiment knowledge is implicit in the
fact that we have a body– embodiment is a
foundation for brain development
Intelligence develops through interaction with environment Situated in a specific
environment Environment is its best
model
EE141
Design principles of intelligent systemsDesign principles of intelligent systemsfrom Rolf Pfeifer “Understanding of Intelligence”, 1999
Interaction with complex environment
cheap design ecological balance redundancy principle parallel, loosely
coupled processes asynchronous sensory-motor
coordination value principle
Agent
Drawing by Ciarán O’Leary- Dublin Institute of Technology
EE141
Embodied Intelligence Embodied Intelligence
Definition Embodied Intelligence (EI) is a mechanism that learns
how to survive in a hostile environment
– Mechanism: biological, mechanical or virtual agent
with embodied sensors and actuators– EI acts on environment and perceives its actions– Environment hostility is persistent and stimulates EI to act– Hostility: direct aggression, pain, scarce resources, etc– EI learns so it must have associative self-organizing memory– Knowledge is acquired by EI
EE141
Embodiment
Actuators
Sensors
Intelligence core
channel
channel
Embodiment
Sensors
Intelligence core
Environment
channel
channelActuators
Embodiment
Actuators
Sensors
Intelligence core
channel
channel
Embodiment
Sensors
Intelligence core
Environment
channel
channelActuators
Embodiment of a MindEmbodiment of a Mind
Embodiment contains intelligence core and sensory motor interfaces under its control to interact with environment
Necessary for development of intelligence
Not necessarily constant or in the form of a physical body
Boundary transforms modifying brain’s self-determination
EE141
Brain learns own body’s dynamic Self-awareness is a result of
identification with own embodiment Embodiment can be extended by
using tools and machines Successful operation is a function
of correct perception of environment and own embodiment
Embodiment of a MindEmbodiment of a Mind
EE141
INPUT OUTPUT
Simulation or
Real-World System
TaskEnvironment
Agent Architecture
Long-term Memory
Short-term Memory
Reason
ActPerceive
RETRIEVAL LEARNING
EI Interaction with EnvironmentEI Interaction with Environment
From Randolph M. Jones, P : www.soartech.com
EE141
How to Motivate a MachineHow to Motivate a Machine ? ?
The fundamental question is how to motivate a machine to do anything, in particular to increase its “brain” complexity?
How to motivate it to explore the environment and learn how to effectively work in this environment?
Can a machine that only implements externally given goals be intelligent?If not how these goals can be created?
EE141
I suggest that hostility of environment motivates us. It is the pain that moves us. Our intelligence that tries to minimize this pain motivates our actions,
learning and development
We need both the environment hostility and the mechanism that learns how to reduce inflicted by the environment pain
How to Motivate a MachineHow to Motivate a Machine ? ?
I propose based on the pain mechanism that motivates the machine to act, learn and develop.
So the pain is good.Without the pain there will be no intelligence. Without the pain there will be no motivation to develop.
EE141
Pain-center and Goal CreationPain-center and Goal Creation
Simple Mechanism Creates hierarchy of
values Leads to formulation of
complex goals Reinforcement :
• Pain increase• Pain decrease
Forces exploration
+
-
Environment
Sensor
MotorPain level
Dual pain levelPain increase
Pain decrease
(-)
(+)
Excitation
(-)
(-)
(+)
(+)
Wall-E’s goal is to keep his plants from dying
EE141
Primitive Goal CreationPrimitive Goal Creation
- +
Pain
Dry soilPrimitive
level
opentank
sit on garbage
refillfaucet
w. can water
Dual pain
EE141
Abstract Goal CreationAbstract Goal Creation The goal is to reduce the primitive pain level Abstract goals are created to reduce abstract pains in order to satisfy the primitive goals Abstract pain center
- +
PainDual pain
+
Dry soil
Abstract pain
“water can” – sensory input
to abstract pain center
Sensory pathway(perception, sense)
Motor pathway(action, reaction)
Primitive Level
Level I
Level IIfaucet
-
w. can
open
water
ActivationStimulationInhibitionReinforcementEchoNeedExpectation
EE141
Abstract Goal HierarchyAbstract Goal Hierarchy
A hierarchy of abstract goals is created - they satisfy the lower level goals
ActivationStimulationInhibitionReinforcementEchoNeedExpectation
- +
+
Dry soilPrimitive Level
Level I
Level IIfaucet
-
w. can
open
water
+
Sensory pathway(perception, sense)
Motor pathway(action, reaction)
Level IIItank
-
refill
EE141
GCS vs. Reinforcement LearningGCS vs. Reinforcement Learning
Environment
CriticStates
Value Function
Policy
reward
action
Environment
CriticStates
Value Function
Policy
reward
action
Actor-critic design Goal creation system
Case study: “How can Wall-E water his plants if the water resources are limited and hard to find?”
Sensorypathway
Motorpathway
GCS
Environment
Pain
States
Gate control
Desired action &state
Action decision
Action
EE141
Goal Creation ExperimentGoal Creation Experiment
Sensory-motor pairs and their effect on the environment
PAIR #SENSORY MOTOR INCREASES DECREASES
1 water can water the plant moisture water in can
8 faucet open water in can water in tank
15 tank refill water in tank reservoir water
22 pipe open reservoir water lake water
29 rain fall lake water -
EE141
Results from GCS schemeResults from GCS scheme
0 100 200 300 400 500 6000
2
4pa
in
Dry soil
0 100 200 300 400 500 6000
1
2
pain
No water in can
0 100 200 300 400 500 6000
1
2
pain
No water in tank
0 100 200 300 400 500 6000
0.5
1
pain
No water in reservoir
0 100 200 300 400 500 6000
2
4
pain
No water in lake
EE141
Averaged performance over 10 trials:
GCS:
RL:0 100 200 300 400 500 600
0
0.5
1
pain
Primitive pain
0 100 200 300 400 500 6000
0.5
1
pain
Lack of food
0 100 200 300 400 500 6000
0.2
0.4
pain
Lack of money
0 100 200 300 400 500 6000
0.2
0.4
pain
Lack of bank savings
0 100 200 300 400 500 6000
0.2
0.4
pain
Lack of job opportunity
0 100 200 300 400 500 600-1
0
1
pain
Lack of school opportunity
Machine using GCS learns to control all abstract pains and maintains the primitive pain signal on a low level in
demanding environment conditions.
0 100 200 300 400 500 6000
10
20
30
GCS vs. Reinforcement LearningGCS vs. Reinforcement Learning
EE141
Goal Creation ExperimentGoal Creation Experiment
Action scatters in 5 CGS simulations
0 100 200 300 400 500 6000
5
10
15
20
25
30
35
40Goal Scatter Plot
Go
al I
D
Discrete time
EE141
Goal Creation ExperimentGoal Creation Experiment
The average pain signals in 100 CGS simulations
0 100 200 300 400 500 6000
0.5
Primitive pain – dry soil
Pa
in
0 100 200 300 400 500 6000
0.10.2
Lack of water in can
Pa
in
0 100 200 300 400 500 6000
0.10.2
Lack of water in tank
Pa
in
0 100 200 300 400 500 6000
0.10.2
Lack of water in reservoir
Pa
in
0 100 200 300 400 500 6000
0.050.1
Lack of water in lake
Pa
in
Discrete time
EE141
Promises of embodied intelligencePromises of embodied intelligence To society
Advanced use of technology– Robots– Tutors– Intelligent gadgets
Intelligence age follows– Industrial age– Technological age– Information age
Society of minds– Superhuman intelligence– Progress in science– Solution to societies’ ills
To industry Technological development New markets Economical growth
ISAC, a Two-Armed Humanoid RobotISAC, a Two-Armed Humanoid RobotVanderbilt UniversityVanderbilt University
EE141
2002 2010 2020 2030
Biomimetics and Bio-inspired SystemsBiomimetics and Bio-inspired SystemsImpact on Space Transportation, Space Science and Earth ScienceImpact on Space Transportation, Space Science and Earth Science
Mis
sio
n C
om
ple
xity
Biological Mimicking
Embryonics
Extremophiles
DNA Computing
Brain-like computing
Self Assembled Array
Artificial nanoporehigh resolution
Mars in situlife detector
Sensor Web
Biological nanoporelow resolution
Skin and Bone
Self healing structureand thermal protection
systems
Biologically inspired aero-space systems
Space Transportation
Memristors
EE141
Sounds like science fictionSounds like science fiction
If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.
But if it doesn’t seem like science fiction, it’s definitely wrong.
From presentation by Feresight Institute
EE141
Questions?Questions?