Multimodal Understanding Group MIT CSAIL Yale...

MLSS 2011, Bordeaux, France

Multimodal Understanding Group | MIT CSAIL Yale Song | [email protected]

The 18th Machine Learning Summer School(MLSS), Bordeaux, France

Yale SongMultimodal Understanding Group

MIT CSAIL1/21

wordle.com

Multimodal Understanding Group | MIT CSAIL


Yale Song | [email protected]

• VTF les Bruyères in Carcans‐Maubuisson

Venue

2/21




• September 4 ~ 17, 2011 @ Bordeaux, France

• ~ 380 applicants, 100 accepted (about 26%)• Application process: Feb 15 ~ April 16, 2011• Decision: May 9, 2011• Poster presentation (optional)

• 11 tutorials, 6 practical sessions, 2 evening talks• 3 sessions per tutorial• 90 mins per session

• 2 social day events (Saint‐Emilion or Medoc winery tour)

By the Numbers

3/21




• Kernel methods• Bernhard Scholkopf, MPI Tuebingen

• Machine learning for robotics• Jan Peters, MPI Tuebingen

• Boosting: theory & applications• Robert Schapire, Princeton

• Monte Carlo methods• Arnaud Doucet, Univ. of Oxford

• Reinforcement learning• Remi Munos, INRIA Lille

• Bayesian inference• Peter Green, Univ. of Bristol

11 Tutorials

4/21

• Bayesian nonparametrics• Yee Whye Teh, UCL

• Sparse methods for under‐determined inverse problems• Remi Gribonval, INRIA Rennes

• Convex optimization• Lieven Vandenberghe, UCLA

• Learning theory: statistical andgame‐theoretic approaches• Nicolo Cesa‐Bianchi, Univ. of Milan

• Graphical models and message‐passing algorithms• Martin Wainwright, UC Berkeley




• Introduction to classification and regression• Luis Montesano, Univ. of Zaragoza

• Convex optimization• Mark Schmidt, INRIA Rocquencourt

• Parametric and nonparametric Bayesian clustering• Francois Caron, INRIA Bordeaux

• (Inverse) Reinforcement learning• Manuel Lopes, INRIA Bordeaux

• (Partially observable) Markov Decision Processes (PoMDP)• Matthijs Spaan, Instituto Superior Tecnico Lisbon

• Gaussian processes and active learning• Roben Martinez‐Cantin, Univ. of Zaragoza

6 Practical Sessions

5/21




6/21




• Speaker: Bernhard Scholkopf, Director ofMax Planck Institute for Intelligent Systems

• Statistical learning theory: empirical risk minimization; VC dimension• Kernels and feature spaces: kernel trick; Mercer’s theorem; positive

definite kernels (PD kernels); reproducing kernel Hilbert space(RKHS); Support Vector Machines; Kernel PCA

Tutorial 1: Kernel Methods

7/21




• Speaker: Jan Peters, Professor, TU Darmstadt

• Model learning: supervised learning for robot control• Policy acquisition: imitation learning• Robot self‐improvement: robot reinforcement learning using optimal

control with either learned model, value function approximation, orpolicy search

Tutorial 2: Machine Learning for Robotics

8/21




9/21




• Speaker: Robert Schapire, Professor, Princeton

• Basic algorithm and core theory: AdaBoost; analysis of training / test errorbased on margins theory

• Fundamental perspectives: game theory; loss minimization; information‐geometric view

• Practical extensions: multiclass classification; ranking problems;confidence‐rated predictions

Tutorial 3: Boosting, theory & applications

10/21

Boosting: Foundations and

Algorithms.

R. Schapire and Y. Freund.

MIT Press 2012




• Speaker: Arnaud Doucet, Professor, University of Oxford

• Topics: rejection sampling; importance sampling; standard Markovchain Monte Carlo (MCMC); adaptive MCMC; auxiliary variablemethods (e.g., parallel tempering, particle MCMC, slice sampling)

Tutorial 4: Monte Carlo Methods

11/21




• Speaker: Rémi Munos, Senior researcher, INRIA Lille

• Introduction to Reinforcement Learning and dynamic programming: DP(value / policy iteration); TD( ); Q‐learning

• Approximate dynamic programming: least‐squares TD; Bellman residual;Fitted‐VI

• Exploration‐Exploitation tradeoffs: stochastic bandit (UCB); adversarialbandit (EXP3); populations of bandits (tree search, Nash equilibrium)

Tutorial 5: Introduction to Reinforcement Learning

12/21




13/21




14/21




• Speaker: Peter Green, Emeritus Professor, Univ. of Bristol

• Topics: Probability, Bayes theorem; likelihood and prior; utility andloss; conjugacy; Bayesian modeling; Bayesian hierarchical models;HMM; Kalman filtering; graphical models; model choice; empiricalBayes; and lots of applications (e.g., protein matching, emissiontomography, sparse latent factor analysis)

Tutorial 6: Bayesian Inference

15/21




• Speaker: Yee Whye Teh, University College London

• Topics: Dirichlet Process (DP); drawing samples from DP (Polya urnscheme; Chinese Restaurant Process (CRP); stick breakingconstruction); DP mixture model; Pitman‐Yor process; hierarchicalBayesian nonparametric models; Latent Dirichlet Allocation (LDA);Chinese Restaurant Franchise; nested CRP

Tutorial 7: Bayesian Nonparametrics

16/21




16/21

• What is nonparametric model?• A really large parametric model.• A parametric model where the number of parameters increases with data.• A family of distributions that is dense in some large space relevant to the

problem at hand.

• Reasons• Model selection. Prevents overfitting and underfitting.• Large function spaces. More straightforward to infer the infinite‐dimensional

objects.• Structural learning.• Novel and useful properties, e.g., projectivity, exchangeability, flexible ways of

building complex models.

• Issues• Developing classes of nonparametric priors suitable for modeling data.• Developing algorithms that can efficiently compute the posterior.• Developing theory of asymptotics in nonparametric models.




• Speaker: Rémi Gribonval, Senior research scientist, INRIA Rennes

• Overview: role of sparsity for compression and inverse problems; introduction tocompressed random sampling

• Algorithms: review of main algorithms & complexity; success guarantees for L1‐minimization to solve under‐determined inverse linear problems

• Deterministic vs. random dictionaries: comparisons of guarantees for differentalgorithms; robust guarantees & restricted isometry property; explicit guaranteesfor various inverse problems

Tutorial 8: Sparse Methods for Under‐determined Inverse Problems

17/21




• Speaker: Lieven Vandenberghe, Professor, UCLA

• Basic theory and convex modeling: convex sets and functions; commonproblem classes and applications

• Interior‐point methods for conic optimization: conic optimization; barriermethods; symmetric prima‐dual methods

• First‐order methods: gradient algorithms; dual techniques

Tutorial 9: Convex Optimization

18/21




• Speaker: Nicolo Cesa‐Bianchi, Professor, Univ. of Milan

• Topics: mistake bounds; risk bounds; empirical risk minimization(ERM); online linear optimization; compression bounds; overfittingand regularization.

• Lots of mathematical proofs.

Tutorial 10: Learning Theory:Statistical and game‐theoretic approaches

19/21




• Speaker: Martin Wainwright, Professor, UC Berkeley

• Introduction to graphical models: Hammersley & Clifford theorem; Markovproperties; factorization;

• Message‐passing algorithms: max/sum‐product; belief propagation; graphs withcycle; tree‐reweighted max‐product algorithm (TRP); LP relaxation

• Variational methods: maximum entropy; conjugate dual functions; geometric view,Kullback‐Leibler divergence; Bethe entropy approximation; Lagrangian derivation ofBP; generalized BP; Expectation‐propagation; mean field theory

Tutorial 11: Graphical Models and Message‐Passing Algorithms

20/21




• Great overview of the field• Good opportunity to know other researchers in the field

• Some useful links• http://mlss11.bordeaux.inria.fr/

• Slides, schedules, general information

• http://videolectures.net/• Video lectures

• http://www.mlss.cc/• Machine Learning Summer School information

Summary

21/21

Date post:	19-Jun-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Multimodal Understanding Group MIT CSAIL Yale...

Documents