Three Kinds of Learning Chrisantha Fernando Marie Curie Fellow, Collegium Budapest Chrisantha...

Three Kinds of LearningThree Kinds of Learning

Chrisantha Fernando

Marie Curie Fellow, Collegium Budapest

Chrisantha Fernando

Marie Curie Fellow, Collegium Budapest

AimAim

I present some examples of learning Associative learning Causal inference Insight based problem solving

My aim is to understand the mechanisms underlying these learning behaviours.

I present some examples of learning Associative learning Causal inference Insight based problem solving

My aim is to understand the mechanisms underlying these learning behaviours.

Classical ConditioningClassical Conditioning

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture. Salivation(Response)

Smell of Food (US)

Sound of Metronome (CS)

QuickTime™ and a decompressor

are needed to see this picture.



Ivan P. Pavlov (1927)



Electric Shock (US)

Light touch (CS)

Withdrawal (Response)



Hawkins et al, 1989

Pre-Synaptic (Eccles) Post-Synaptic (Hebb)







QuickTime™ and aTIFF (LZW) decompressor




Coincidence detectors

ParameciaParamecia

What could they learn?What could they learn?

Temperature change precedes O2 change in marine ecosystems by 20 minutes.

Photon flux may precede temperature changes.

Aerobic to anaerobic respiration from mouth to gut (signaled by increasing temperature).

Temperature change precedes O2 change in marine ecosystems by 20 minutes.

Photon flux may precede temperature changes.

Aerobic to anaerobic respiration from mouth to gut (signaled by increasing temperature).



David S Goodsell, 1998 “The Machinery of Life”

Previous WorkPrevious Work



na = [A] + [AB]

nb = [B] + [AB]Gandhi et al, 2007

A Simple Learning CircuitA Simple Learning Circuit

In collaboration with molecular biologists, I have designed Hebbian learning circuits for plasmids carried by E. coli.

In collaboration with molecular biologists, I have designed Hebbian learning circuits for plasmids carried by E. coli.

v = w.u

dwi/dt = uiv



A Gene Regulatory Network

No pairing Pairing

Output P

Artificial Evolution in Silico

SBMLEvolver Synthetic Biology Toolbox





Evolution in vivo

What will evolve?

Fitness functionFitness function

€

fitness =cconst + T7(P)

min. P if nosignal is active6 7 8

(T8(P)− T7(P))maximise learning intest caseSet to 0, if < 0

1 2 4 4 3 4 4 (T8(P)−C8(P))

minimise learning incontrol caseSet to 0, if < 0

1 2 4 4 3 4 4 (T2(P)− T4(P))min. output as responseto U2 compared to U1 Set to 0, if < 0

1 2 4 4 3 4 4 + T2(P)− T8(P)

equal output as responseto U1 before and U2after learning

1 2 4 4 3 4 4 + a+ b+ c+ d

punishments1 2 4 3 4

€

dwijdt

= η o(vi − vθ )uj

€

dwijdt

= η o(wmax − wij )viuj

€

dwijdt

= η oviuj − αvi2wij

€

dwijdt

= η oviuj − αwij(1− wij )(wθ − wij )

€

dwijdt

= η oφ(vi − vi )uj

€

d vidt

= nv(vi2 − vi )

Presynaptic Subtractive Norm Oja (L2 Norm)

Weight consolidationBCM

MAPK ImplementationMAPK Implementation

Why?

Later….

C

A

B

R

C

A

B

R

Learns patient specific (contingent) associations

Susan

Jeffery

Similar Recent Work…Similar Recent Work…

Prediction, but no associative learning. Prediction, but no associative learning.







Tagkopoulos et al, 2008 (in press)

Limitations of Associative Learning

Limitations of Associative Learning

Can only learn associations between pre-specified stimuli

Learns only associations, not cause and effect relationships.

Can only learn associations between pre-specified stimuli

Learns only associations, not cause and effect relationships.

“A Concept of Cause and Effect”

“A Concept of Cause and Effect”

“My argument is that causal understanding gave rise to tool-making; that was the evolutionary advantage. It's tool-making that's really driven human evolution. This is not widely accepted, I'm afraid, but there's no question about it. It's tools that really made us human. They may even have given rise to language.”

“My argument is that causal understanding gave rise to tool-making; that was the evolutionary advantage. It's tool-making that's really driven human evolution. This is not widely accepted, I'm afraid, but there's no question about it. It's tools that really made us human. They may even have given rise to language.”

Lewis Wolpert, 2007

What is causal Inference?What is causal Inference?

Does dropping a coin into a tin of coins cause the number of coins in the tin go up? Can moving a piece on a chess-board cause the opponent's queen to be pinned? Can ignorance cause poverty? Can poverty cause crime? Can ignorance cause a TV set to be moved through a broken window? Can inserting a certain sort of twig in a certain way into a particular partly built nest cause the

nest to become more rigid?

Analysing the concept of causation is probably the hardest unsolved philosophical problem. It's at the root of problems about relations between mind and body (or relations between virtual and physical machines).

Does dropping a coin into a tin of coins cause the number of coins in the tin go up? Can moving a piece on a chess-board cause the opponent's queen to be pinned? Can ignorance cause poverty? Can poverty cause crime? Can ignorance cause a TV set to be moved through a broken window? Can inserting a certain sort of twig in a certain way into a particular partly built nest cause the

nest to become more rigid?

Analysing the concept of causation is probably the hardest unsolved philosophical problem. It's at the root of problems about relations between mind and body (or relations between virtual and physical machines).

Sloman, 2008 (pc)

Causal InferenceCausal Inference

What is the difference between causal inference and associative learning?

Weak: To utilize more than pair-wise correlations (perhaps unconsciously).

Strong: Combining observation of conditional probability P(X|Y) with novel appropriate interventions i.e. why don’t Pavlov’s dogs spontaneously ring the bell

when they are hungry? (without reinforcement). Humans do generate hypotheses based on CP and

produce interventions to test causal models. Parties -> Wine -> Insomnia Wine <- Parties -> Insomnia

What is the difference between causal inference and associative learning?

Weak: To utilize more than pair-wise correlations (perhaps unconsciously).

Strong: Combining observation of conditional probability P(X|Y) with novel appropriate interventions i.e. why don’t Pavlov’s dogs spontaneously ring the bell

when they are hungry? (without reinforcement). Humans do generate hypotheses based on CP and

produce interventions to test causal models. Parties -> Wine -> Insomnia Wine <- Parties -> Insomnia

Structuring InterventionsStructuring Interventions

A --> B --> C --> D Intervene at C: A-> B C-->D

A <-- B <-- C <-- D Intervene at C: A <-- B <-- C D

A --> B --> C --> D Intervene at C: A-> B C-->D

A <-- B <-- C <-- D Intervene at C: A <-- B <-- C D

Algorithms exist to discover causal networks.

Algorithms exist to discover causal networks.

Bayesian learning Know prior probability of causal graphs Know probability of observations given each graph Use Bayes theorum to calculate probability of graph given observations and priors Fined the best graph

Constraint-based learning For each pair of variables a and b in V, search for a set Sab such that (a ||_ b |Sab) holds in

P, i.e. a and b should be independent in P, conditioned on Sab. Construct an undirected graph G such that vertices a and b are connected with an edge iff no set Sab can be found. Connect dependent nodes

For each pair of non-adjacent variables a and b with a common neighbor c check if c is an element of Sab. If it is continue, if it is not then add arrow heads pointing at c i.e. a--> c <-- b.

In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions:

i. the orientation should not create a new v-structure, ii.the orientation should not create a directed cycle.

Bayesian learning Know prior probability of causal graphs Know probability of observations given each graph Use Bayes theorum to calculate probability of graph given observations and priors Fined the best graph

Constraint-based learning For each pair of variables a and b in V, search for a set Sab such that (a ||_ b |Sab) holds in

P, i.e. a and b should be independent in P, conditioned on Sab. Construct an undirected graph G such that vertices a and b are connected with an edge iff no set Sab can be found. Connect dependent nodes

For each pair of non-adjacent variables a and b with a common neighbor c check if c is an element of Sab. If it is continue, if it is not then add arrow heads pointing at c i.e. a--> c <-- b.

In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions:

i. the orientation should not create a new v-structure, ii.the orientation should not create a directed cycle.



CrowsCrows

What is the evidence for causal understanding in crows?

What is the evidence for causal understanding in crows?

QuickTime™ and aH.263 decompressor


I’m not convinced…I’m not convinced…

Tool use is innate in New Calodonian crows; no social learning is required.

Crows can make the right length and thickness of tool for the right hole, on the first trial. This could still be associative learning.

Must exclude simple strategies, e.g. random search, win-stay loose shift, reinforcement learning, operant conditioning, etc… “Rather than giving subjects a defined set of choices, they are placed in a situation

where they have a low probability of solving a task by chance alone (for example, in a hook making task an animal may be given a piece of pliable material that can be changed into an infinite number of shapes, but only a small subset of these shapes would be functional)”

How does one define the null-hypothesis, e.g. what is the probability of manufacturing a hook-shaped object by chance alone?

Tool use is innate in New Calodonian crows; no social learning is required.

Crows can make the right length and thickness of tool for the right hole, on the first trial. This could still be associative learning.

Must exclude simple strategies, e.g. random search, win-stay loose shift, reinforcement learning, operant conditioning, etc… “Rather than giving subjects a defined set of choices, they are placed in a situation

where they have a low probability of solving a task by chance alone (for example, in a hook making task an animal may be given a piece of pliable material that can be changed into an infinite number of shapes, but only a small subset of these shapes would be functional)”

How does one define the null-hypothesis, e.g. what is the probability of manufacturing a hook-shaped object by chance alone?





Both Betty and Bob use trial and error search.

Causal Inference in RatsCausal Inference in Rats

What is the evidence for causal inference in rats?

What is the evidence for causal inference in rats?







Causal Inference in ChildrenCausal Inference in Children

What is the evidence for causal inference in humans?

Understanding interventions (monkey sneezing + blickets, etc….) A+ --> B- --> AB+ --> AB+ (Children choose A) A+ --> A+ --> A+ --> B- --> B+ --> B+ (Choose randomly)

Retrospective disambiguation (by children) e.g. AB+ --> A-, AB+ --> A+

What is the evidence for causal inference in humans?

Understanding interventions (monkey sneezing + blickets, etc….) A+ --> B- --> AB+ --> AB+ (Children choose A) A+ --> A+ --> A+ --> B- --> B+ --> B+ (Choose randomly)

Retrospective disambiguation (by children) e.g. AB+ --> A-, AB+ --> A+



Gopnik & Schultz, 2004







Our approachOur approach

To study intra-brain causal inference. To study intra-brain causal inference.

Insight in HumansInsight in Humans



"Draw four continuous straight lines, connecting all the dots without lifting your

pencil from the paper."

How to Solve it?How to Solve it?

What is an ‘insight problem’? A problem that requires restructuring of the initial problem representation, e.g. goal states, operators, constraints.

What kinds of algorithm are used to solve these and related problems? What determines the set of goal, operators, constraints?

What is an ‘insight problem’? A problem that requires restructuring of the initial problem representation, e.g. goal states, operators, constraints.

What kinds of algorithm are used to solve these and related problems? What determines the set of goal, operators, constraints?

Missionaries and CannibalsMissionaries and Cannibals

3 missionaries and 3 cannibals must cross a river using a boat which can carry at most two people.

For both banks, if there are missionaries present on the bank, they cannot be outnumbered by cannibals.

The boat cannot cross the river by itself with no people on board.

3 missionaries and 3 cannibals must cross a river using a boat which can carry at most two people.

For both banks, if there are missionaries present on the bank, they cannot be outnumbered by cannibals.

The boat cannot cross the river by itself with no people on board.

Intermediate goals may be usedIntermediate goals may be used

Early moves balance number of M & C on each side of river.

Intermediate moves maximize progress from one side to other.

Later moves avoid revisiting previous states.

Early moves balance number of M & C on each side of river.

Intermediate moves maximize progress from one side to other.

Later moves avoid revisiting previous states.

There is some evidence…There is some evidence…

Non-maximal moves that allow a subsequent move to make more progress are retained as promising states for future trials.

Goal criteria are relaxed and changed based on the quality (immediate benefit) of generated solutions.

Sometimes hill-climbing to a ‘wrong’ goal criteria can get stuck in local minima.

Non-maximal moves that allow a subsequent move to make more progress are retained as promising states for future trials.

Goal criteria are relaxed and changed based on the quality (immediate benefit) of generated solutions.

Sometimes hill-climbing to a ‘wrong’ goal criteria can get stuck in local minima.





3 moves

CHRONICLE, MACGREGOR, AND ORMEROD,2004

-7426 legal 3-move sequences-2 reach ring solution-176 reach 2 group solution-7426 sequences are not eqiprobable under random selection assumption. -< 1/3 participants solve problem within 10 minutes.

-Choice of the correct first move based on the improved goal scores available from the second move was crucial.

-Few subjects even conceived of a two group solution when asked to “produce a shape where each coin only touches two others”.

Brain damage helps some problem solving!

Brain damage helps some problem solving!

II = III + I Type A

IV = III - I Type B

VI = VI + VI Type C



Solutions here



Reverberi et al 2005

Our approachOur approach

To study mechanisms for restructuring of problem representations.

To study mechanisms for restructuring of problem representations.



Poelwijk et al 2007

ConclusionsConclusions

What neural mechanisms underlie causal inference, and solving insight problems?

What changes allow humans to have these capacities but precludes other apes from having them?

What algorithms can best predict human performance in such problems?

What neural mechanisms underlie causal inference, and solving insight problems?

What changes allow humans to have these capacities but precludes other apes from having them?

What algorithms can best predict human performance in such problems?

Thanks toThanks to

Eors Szathmary Lewis Bingle Anthony Liekens Aaron Sloman Jon Rowe Dov Stekel Christian Beck & Thorsten Lenser

Eors Szathmary Lewis Bingle Anthony Liekens Aaron Sloman Jon Rowe Dov Stekel Christian Beck & Thorsten Lenser







QuickTime™ and a decompressorare needed to see this picture.

QuickTime™ and a decompressorare needed to see this picture.

Date post:	28-Mar-2015
Category:	Documents
Upload:	anna-greer
View:	214 times
Download:	1 times

Three Kinds of Learning Chrisantha Fernando Marie Curie Fellow, Collegium Budapest Chrisantha...

Documents