Transfer Learning

• Motivation and Types

• Functional Transfer Learning• Representational Transfer Learning• References

• The goal is to transfer knowledge gathered from previous experience.

• Also called Inductive Transfer or Learning to Learn.

• Example: Invariant transformations across tasks.

Transfer Learning

Motivation for transfer learning

Similar to self-adaptation: once a predictive model is built, there are reasons to believe the model will cease to be valid at some point in time.

The difference is that now source and target domains can be completely different.

Motivation Transfer Learning

Traditional Approach to Classification

DB1 DB2 DBn

Learning System

Learning System

Learning System

Transfer Learning

DB1 DB2

DB new

Learning System

Learning System

Learning SystemKnowledge

Source domain

Target domain

Transfer Learning

Scenarios:

1.Labeling in a new domain is costly.

DB1 (labeled)

Classification of Cepheids

DB2 (unlabeled)

Classification of LPV

Transfer Learning

Scenarios:

2. Data is outdated. Model created with one survey buta new survey is now available.

Survey 1

Learning System

Survey 2

?

Types of Transfer Learning

Figure obtained from Brazdil, et. Al. Metalearning: Applications to Data Mining, Chapter 7, Springer, 2009.

Transfer Learning

• Motivation and Types

• Functional Transfer Learning• Representational Transfer Learning• References

Input nodesInput nodes

Internal nodesInternal nodes

Output nodesOutput nodes

Left Left StraightStraight RightRight

Functional Transfer: Multitask Learning

Given example X, compute the output of every node until Given example X, compute the output of every node until we reach the output nodes:we reach the output nodes:

Input nodesInput nodes

Internal nodesInternal nodes

Output nodesOutput nodes

Example XExample X

Compute sigmoid Compute sigmoid functionfunction

Functional Transfer in Neural Networks

Train in Parallel with Combined Architecture

Figure obtained from Brazdil, et. Al. Metalearning: Applications to Data Mining, Chapter 7, Springer, 2009.

Transfer Learning

• Motivation and Types

• Functional Transfer Learning• Representational Transfer Learning• References

Knowledge of Parameters

Assume prior distribution of parameters

Source domain

Learn parameters and adjust prior distribution

Target domain

Learn parameters using the source priordistribution.

P(y|x) = P(x|y) P(y) / P(x)

Parameter Similarity

Task A Parameter A

Task B Parameter B ~ A

Assume hyper-distribution with low variance.

Assume Parameter Similarity

Knowledge of Parameters

Find coefficients ws using SVMs

Find coefficients wT using SVMsinitializing the search with ws

Feature Transfer

Feature Transfer:

Target domain

Source domain

Shared representation across tasks

Minimize Loss-Function( y, f(x))

The minimization is done over multiple tasks (multiple regions on Mars).

Feature Transfer

Identify commonFeatures to all tasks

Coded divided into pieces

New Solution

Add pieces of code from previous tasks

Start a new solution from scratch

Meta-Searching for Problem Solvers

Exploitation: Maximize reward

vs

Exploration: Maximize long-term success.

Learn to keep the ball away from the opponent.

First Task

Learn to score the opponent.

Second Task

Transfer Learning in Robotics

Instance Transfer Learning

Instance Transfer:

Learning System

Target domainSource

domainFilter samples

Larger target dataset

New program calledTrAdaboost

Instance Transfer Learning

New program calledTrAdaboost

•The main idea is to have a methodology to deal with a changing distribution.

•Examples in the source domain that look as belonging to a diff. distribution are discarded.

•Examples in the source domain that look similar to the target domain are added to the training set.

Boosting

DB

Incorrectly classifiedinstances increase weight

11

1

1 11

1

111

111

11

2

2 2

22

Boosting

DB

11

11

1

2

2

22

22

11

Combine all hypotheses to produce final weighted function:

w1 f1 + w2 f2 + … + wn fn

Automatic Instance Transfer

Boosting

Source domain

Target domain

Learning System

(Boosting)

Incorrectly classifiedinstances decrease weight

Incorrectly classifiedinstances increase weight

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Automatic Instance Transfer

Boosting for Transfer Learning, Wenyuan Dai, et. al. ICML 2007

Transfer Learning

• Motivation and Types

• Functional Transfer Learning• Representational Transfer Learning• References

Sinno Jialin Pan and Qiang Yang. A Survey on Transfer Learning.IEEE Transactions on Knowledge and Data Engineering, 22(10):1345-1359, Oct. 2010

Brazdil, P. et. al. Metalearning: Applications to Data Mining. Springer, 2009.

Dai, W., Boosting for Transfer Learning, Proceedings of ICML 2007.

Video on transfer learninghttp://www.youtube.com/watch?v=9ChVn3xVNDI&noredirect=1

References

Sinno Jialin Pan and Qiang Yang. A Survey on Transfer Learning.IEEE Transactions on Knowledge and Data Engineering, 22(10):1345-1359, Oct. 2010

Brazdil, P. et. al. Metalearning: Applications to Data Mining. Springer, 2009.

Dai, W., Boosting for Transfer Learning, Proceedings of ICML 2007.

Video on transfer learninghttp://www.youtube.com/watch?v=9ChVn3xVNDI&noredirect=1

References

Robot learns to flip pancakes

http://www.youtube.com/watch?v=W_gxLKSsSIE&noredirect=1

Robot learns to stack pancakes

http://www.youtube.com/watch?v=v9oeOYMRvuQ

Videos