Date post: | 15-Jan-2016 |
Category: |
Documents |
View: | 222 times |
Download: | 0 times |
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal Experiment Design (OED)for kernel ridge regression
and
the Minimum Volume Covering Ellipsoid (MVCE)
Tijl De Bie(K.U.Leuven)
Joint work with: Alexander Dolia John Shawe-TaylorMichael Titterington
Chris Harris
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 2
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
The next hour…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 3
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 4
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 5
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design?OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 6
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
NotationOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 7
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
ExamplesOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 8
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
ExamplesOED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 9
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Deliverables in this talk…OED?
Notation
Examples
Deliverables…
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 10
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Least squares regression (LS)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 11
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Least squares regression (LS)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 12
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 13
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 14
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 15
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for LS
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 16
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 17
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 18
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Ridge regression (RR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 19
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 20
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 21
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Optimal experiment design for RR
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 22
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 23
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 24
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel ridge regression (KRR)
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 25
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel D-OED
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 26
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel D-OED
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 27
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
OED: summaryD-OED MVCE
stan
dard
regu
lariz
edke
rnel
Least squares
Ridge regression
Kernel RR
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 28
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Now over to novelty detection!
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 29
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Now over to novelty detection!
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 30
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Regularized MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 31
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 32
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 33
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Kernel MVCE
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 34
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: summary
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 35
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: summaryD-OED MVCE
stan
dard
regu
lariz
edke
rnel
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 36
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
MVCE: dealing with outliers
Novelty detection
MVCE and duality
Regularized MVCE
Kernel MVCE
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 37
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
Linear
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 38
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
Linear, centered
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 39
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
RBF-kernel
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 40
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – MVCE
RBF-kernel
Soft-margin
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 41
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – D-OED
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 42
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Experiments – D-OED
2-norm
infinity norm
1-norm
Costs:
Random vs
Uniform vs
OED (blue)
Eurandom - 20 Dec '05 Tijl De Bie - KULeuven 43
OED for KRR, and the MVCE
Introduction
OED
MVCE
Experiments
Conclusions