Machine Learning on a BudgetTitle Machine Learning on a Budget Author Ludovic Denoyer Full professor...

Machine Learning on a Budget

Ludovic DenoyerFull professor at University Pierre et Marie Curie, Paris, France

Quick CV2001: Master in artificial intelligence

2004: PhD in machine learning for semi-structured documents2005: Associate professor, Paris, France

2015: Full professor at University Pierre et Marie Curie, Paris, France

Supervision of a dozen of PhD students

Co-head of the Data Science master (≈ 100 students)Chargé de mission Open Research

Member of the France IA thinking group attached to the french government

July 6, 2017

L. Denoyer Machine Learning on a Budget July 6, 2017 1 / 27

Research Overview - Main TopicsA research focused on the development of new learning models, and driven by real-world applications. A great interest modeling human/machine behaviors.

At the beginning...

Machine Learning with Semi-Structured Multimedia Documents

Reinforcement Learning for Structured Output Prediction

Machine Learning with complex networks (2006-...)Sequential Learning

I Intelligent crawling, active sensors acquisition

Representation LearningI Multi-label ClassificationI Extraction of (multi) RelationsI Heterogeneous Social NetworksI Predicting Information Diffusion on Social Networks

Deep LearningI Spatio-temporal Neural Networks for Forecasting and CompletionI Time series Forescasting with gaussian embeddingsI ...

Budgeted (Deeply-Reinforced) Machine Learning (2009-...)The topic of the presentation today

The new directions (2016 - ...)Learning with incomplete data e.g multi-view, stream, ... – Methods inspired from GAN and VAE

Learning for human-machine interaction – Deep RL

Decentralized and Long-life learning – Deep RL and Meta LearningL. Denoyer Machine Learning on a Budget July 6, 2017 2 / 27

Today

1 Sequential Budgeted Learning (2009–now)

2 Budgeted Sequential Acquisition Models

3 Focus on: Recurrent Adaptive Information Acquisition Models (2016)

4 Focus on: Learning Time-Efficient Deep NN architectures (2017)


Introduction

QuestionWhat is the difference between how a human classifies these pictures and how a learning model(e.g deep neural network) classifies these pictures ?


Introduction



Introduction



Introduction



Introduction



Introduction



Introduction



Introduction



Introduction



Introduction



Budgeted Learning

ML AssumptionsWe assume that the input x is a priori known (e.gfeatures are known)

I In the real life: The information needs to be acquiredThe same computations are applied to all the inputs.

I In the real life: Different computations are applied todifferent inputs

The ”size” of the model is chosen by handI In the real life: The model is able to adapt itself to the

context in which it is used

Research Direction from 2009 to nowObjective: Develop a new family of models able to acquire information by them-selves, tochoose what to compute and to handle operational constraints.

From predictive models to predictive agents....


Sequential Predictive Models

Inspiration: General Diagnostic ProcessAsk general questions to get context.Ask increasingly specific questions based on previous answers (Adaptive Acquisition)Exploit the answers in an adapted way (Conditional Computation)Build the output to predict (Output Prediction)

Key aspects:The model sequentially chooses actions to performActions can be of different types (acquisition, computation, prediction,..)The way actions are chosen is based on a cost-benefit trade-off


Sequential Budgeted Learning

Budgeted Sequential LearningInput: x (a vector, a picture, a state/task, ...), Output: y (a label, a vector, an action, ...)Actions: H = H1, ...,HT a set of actions chosen following Pθ(H/x)Prediction: A function Fθ(x ,H)→ yBudget constraint: C(H) is the cost of H,

I C(H) can be the time spent for prediction, the memory/CPU/energy consumption, etc...

Learning objective:

J(θ) = EPθ(x,H,y)[∆(Fθ(x ,H), y) + λC(H)]

=1`

`∑k=1

EPθ(H/x)[∆(Fθ(xk ,H), yk ) + λC(H)](1)

Now, the learning problem also controls the behavior of the model, not only its predictiveperformance.


Learning problem

Learning

J(θ) =1`

`∑k=1

EPθ (H/x)[∆(Fθ(xk,H), yk ) + λC(H)] (2)

1 If J(θ) is not differentiable ⇒ use your favorite Reinforcement Learning algorithm

2 If J(θ) is differentiable ⇒ adapted gradient approaches

Sequential Acquisition Models [2010-now]

an initial block of information x(0) is sampledthe algorithm chooses which information x(t+1) to acquire nextIf the model considers that enough information has been gathered, it chooses to compute a prediction(a label here)The penalty is on the amount of information gathered or on its price (cost-sensitive classification)


Information Acquisition as a RL problem

Markov Decision ProcessesA state s ∈ S is composed of:

I an input datumI a sequence of previously acquired information

Different types of actions a ∈ A:I Acquisitions actionsI Classification actions

Transitions: Can be stochastic or deterministicReward: −∆(Fθ(x ,H), y)− λC(H) (+ reward shaping)

Other elementsEach state s is described by a features vector

I It corresponds to the aggregation of previously acquired informationI Can be manually defined or learned (e.g recurrent neural networks [ICONIP 2016])

A prediction is a policy : π(s)→ AAn optimal policy is found by using approximating reinforcement learning algorithms


Examples of applications

Textual Reading process

(ECIR 2011) - Approximated RL with RCPI

4 text classification datasets

Performance ≈ baseline methods (SVM)

At a lower price...

Visual Attention Model

(ICLR 2014) - specific learning Algorithm (close to NeuralFitted-Q)

Results on 15-scenes and PPMI


Features Acquisition

Multiple configurations/tasksMultiple MDPs structure studied andmultiple reward functions(Machine Learning Journal 2012) -OLPOMDP and RCPI

Cold Start Recommendation (ICLR2015)

CS-IAMAmerican BeautyBeing John MalkovichLion KingGhostSupermanBack to the FutureFargoArmageddonGet ShortySplash20 000 Leagues Under the SeaBack to the Future Part III

LimitsBut using ”pure” RL for features acquisition is very slow (large number of features)⇒ Can we speed-up the learning when facing particular problems ?


Differentiable Cost Sensitive Features Acquisition Model

TaskConsider a prediction problem where each features i has a particular known cost ci .⇒ How can we learn a good trade-off between prediction quality and cost ?

Two solutions:RADIN: Recurrent Adaptive NN – solved a relaxed version of the problemCSREAM: Deep RL-based approach

PrinciplesThe models can simulatenously acquire multiple features at each timestep.The models are able to learn how to aggregate acquired features in a latent representationspace.The models will be learned by (policy) gradient-descent based approaches (e.g GPUs,parallelization,...)



Notationsx : input datumαt binary vector such that. αt,i 6= 0⇒ feature i of x acquired at step t.x [αt ] = features of x selected by αt

α = (α1, ..., αT ) = features acquired during the whole sequential process

Acquisition process (Inference)procedure Inference(x ,T )

α0 = 0for t = 1..T do

Sample αt from π(αt |x [(α1, .., αt−1])Acquire x [αt ] where new features are such that αt,i = 1

end forreturn y = dθ(x [α])

end procedure



Features aggregationAggregation of collected information is made by a recurrent neural network.

zt is the representation of x after acquisition of α1, ...., αt

zt = f (zt−1, x � αt ) and x [α] = zT

Loss function

J (d , π) = E(x,y)∼p(x,y)[Eα∼π(alpha|x) [ ∆(d(x [α]), y) +λαᵀ.c]]


CSREAM: Stochastic Acquisition Model

J CSREAM(x , y , γ, θ) = Eα∼πγ (α|x) [∆(dθ(x [α]), y)] + λ

T∑t=1

n∑i=1

πγ(αt,i = 1|x).ci

Learning by Policy-gradient

∇γ,θ,βJ (x , y , γ, θ, β) ≈1M

M∑m=1

[∆(dθ(zT +1), y)

T∑t=1

∇γ,θ log πγ(zt )

+∇γ,θ(∆(dθ(zT +1), y) + λ

T∑t=1

n∑i=1

∇γ,θπγ,i (zt ).ci

]

Acquisition policyDifferent possible instances (Bernoulli, Multinomial, ...)


Results I

Corpus Name Nb. Ex Nb.Feat

Nb.Cat

Model Amount of features used (%)

90% 75% 50% 25% 10%

Adult 10708 14 2

SVM L1 84.4 84.3 81.2 77.8 76.3C4.5 85.3 85.3 85.3 84.6 79.0

GreedyMiser 86.0 86.0 85.9 84.8 78.0B-REAM 84.9 84.9 84.6 84.3 81.6RADIN 85.3 85.3 84.9 84.4 82.1

Letter 6661 16 26

SVM L1 48.3 33.0 23.6 14.2 08.6C4.5 82.3 82.3 82.3 48.4 10.2


.....

MNIST 62000 780 10

SVM L1 89.7 89.7 88.2 70.4 57.7C4.5 80.8 80.8 80.8 80.8 80.8


Corpus Name Nb. Ex Nb. Feat Model Amount of features used (%)25% 10% 5% 1%

gisette 6000 5000

SVM L1 97.0 96.8 96.3 91.0DecisionTree 91.9 91.9 91.9 91.9GreedyMiser 88.4 88.4 86.7 78.5

RADIN 95.7 95.7 95.7 94.7

r8 7674 6224


RADIN 96.2 96.1 96.1 95.9

webkb 4162 5388


RADIN 96.2 96.1 86.5 83.1


Results II

Illustration des pixels acquis pour chaqueétape du processus - Modèle RADIN.

Interet du processus adaptatif : performancede RADIN pour différents T .


Transition

From Adaptive Features Acquisition to Conditional computing (2015)Idea: Instead of adaptively choosing which information to acquire, we can adaptively choosewhich computations to perform.

Deep Sequential Neural Networks:An extension of NN where layers arechoosen accordingly to the inputthe model is able to capture complexdistributions...... while minimizing the number ofcomputations

Key Issue (submitted NIPS 2017)Can we automatically learn a Deep-NN architecture that is efficient both in term of predictivequality and computation time ?

Learning problem: Find the best neural network architecture (in a very large set of possiblearchitectures) able to predict well in less than 100 milliseconds on my mobile phone


Learning Time Efficient Deep Architectures

Super Networks

A Super Network is a DAG of layers(l1, ..., lN )l1 is the input layer, lN is the output layerE = {ei,j} ∈ 0; 1N×N is the edge between liand lj associated with function fi,j

Inference:Input: l1(x ,E , θ) = xLayer Computation: li (x ,E , θ) =

∑k

ek,i fk,i (lk (x ,E , θ))

Output: f (x ,E , θ)← lN (x ,E , θ)Learning can be made by back-propagation (GPUs,.....)


Learning Time Efficient Deep ArchitecturesLet us denote H ∈ {0; 1}N×N a matrix such that H � E defines a (sub) Super NetworkLet us denote C(H � E) the cost (e.g time in milliseconds) for computing f (x ,H � E , θ)

Budgeted Learning problem

H∗, θ∗ = argminH,θ

1`

∑i

∆(f (x i ,H � E , θ), y i ) under constraints: C(H � E) ≤ C

Soft version

H∗, θ∗ = argminH,θ

1`

∑i

∆(f (x i ,H � E , θ), y i ) +λmax(0,C(H � E)− C)

⇒ Complex Combinatorial problemL. Denoyer Machine Learning on a Budget July 6, 2017 20 / 27

Learning Time Efficient Deep ArchitecturesStochastic Super Network

We consider a matrix of probabilities ΓAt each inference, H is sampled following H ∼ Γ

Inference1: procedure SSN-forward(x ,E , Γ, θ)2: H ∼ Γ . as explained in Section ??3: For i ∈ [1..N], li ←Ø EndFor . Init layers values4: l1 ← x . Set the input in the first layer5: For i ∈ [2..N],li ←

∑k<i

ek,i hk,i fk,i (lk ) EndFor

6: return lN7: end procedure

Stochastic learning problem

Γ∗, θ∗ = argminΓ,θ

1`

∑i

EH∼Γ[

∆(f (x i ,H � E , θ), y i ) + λmax(0,C(H � E)− C)]

where C is the maximum authorized cost (e.g 200ms)

It can be shown that the optimal solution to this problem is equivalent to the optimal solution tothe original problem.


Learning Time Efficient Deep Architectures

LearningLet us define:

D(x , y , θ,E ,H) = ∆(f (x ,H � E , θ), y) + λmax(0,C(H � E)− C)

We have:

∇θ,ΓL(x , y ,E , Γ, θ) ≈ (∇θ,Γ logP(H|Γ))(D(x , y , θ,E ,H)− D) +∇θ,Γ∆(f (x ,H � E , θ), y)

Note that:The Learning algorithm can handle input-specific costs C(H � E , x)The learning algorithm can also handle stochastic costs functions C(H � E , x)Learning has to be made on the operational system in order to handle the real-cost of themodel (or can use simulated costs)


Experiments (Image Classification)

Model Flop gain ErrorCNF/BNF 0 % 7.99 %

CNF 0 % 9.42 %MaxOut 0 % 9.38 %Drop Con. 0 % 9.32 %

BSN

20.0 % 9.21 %50.0 % 9.20 %80.0 % 10.18 %90.0 % 11.03 %97.0 % 11.58 %98.0 % 13.66 %>98 % 90 %

P-CNF17.5 % 10.29 %28.1 % 19.90 %>28.1 % 90.00 %

Smart P-CNF

0.0 % 8.74 %50.0 % 11.41 %80.0 % 12.45 %96.8 % 13.03 %>96.8 % 90.00 %

(a)Performance on CIFAR - 10

Model Time (ms) ErrorCNF/BNF 68.12 0.39 %

CNF 68.12 0.39 %CKN - 0.39 %

MaxOut - 0.45 %

BSN54.50 0.45 %34.06 0.42 %13.62 0.43 %6.81 0.45 %5.17 0.43 %0.0 89.72 %

P-CNF54.50 0.54 %34.06 1.08 %23.23 1.97 %0.0 89.72 %

(b) Performance on MNIST

(c) Learned architecture on CIFARL. Denoyer Machine Learning on a Budget July 6, 2017 23 / 27

Experiments (Part Labels+Toy)

(a) Architecture on Part-Labels

(b1) Simulating a distributed environment

(b2) Randomly sampled edges costs


Conclusion and PerspectivesAdaptive Budgeted LearningI have proposed models able to:

Learn what to acquireLearn what/when to computeUnder realistic budgeted constraints

It allows to develop systems that can:Deal with expensive informationDeal with incomplete informationReduce time spent for predicting

PerspectivesExpensive information ⇒ Human-Machine Interaction =

I Learning model able to interact with humans/agentsI Decentralized learning

Incomplete InformationI Learning where to locate sensors on dynamic systemsI Monitoring of social networks, ...

Under budgeted constraintsI Memory/Electricity consumptionI Privacy constraints


Questions


Date post:	04-Jun-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times